Testing for mobile is a minefield. You can be targeting multiple frameworks and then multiple devices within any one of these frameworks. You have the extra variability contributed by the technologies you choose to write your app. For Android alone, there has been considerable device fragmentation. Some reports suggest that there could now be 12,000 different Android devices. Careful consideration needs to be given to how an app appears and behaves on any given device, bearing in mind ever rising user expectations, coupled with a complex and fragmented market for apps. If your app doesn’t work well, there will, with no doubt, be a selection of alternative apps on the market for users to choose from instead.

Xamarin Test Cloud, could be one solution to our mobile testing nightmares. It is a BDD (Behaviour Driven Development) cloud-based testing framework. Anyone who has written tests in WatIn, Selenium or used Cucumber and SpecFlow will find getting to grips with Test Cloud relatively straightforward.

Test Cloud can integrate with your CI system (we use TeamCity here at The Technology Studio). This means Test Cloud tests can be run and can fail your build just like any other type of tests you run. Tests can be written in either C# or Ruby and the user-friendly syntax means anyone involved in the project can write tests, including product owners, testers and developers. Test Cloud runs tests on up to 134 devices, which they claim represents 75% of the mobile market.

Xamarin Test Cloud does have it’s drawbacks though. One being that it is unable to test most hardware features, for example Bluetooth Pairing and NFC technology. In addition, only one set of tests can be run at the same time in the cloud, which means tests will be queued up, potentially blocking CI builds. On the otherhand, if your tests don’t complete within a specific timeframe, they will be automatically ended. So if your app is hardware heavy, this testing framework may not be for you.

Users currently need to sign up for a beta program by contacting Xamarin at hello@xamarin.com. Xamarin is yet to release pricing information. Once signed up, Xamarin will grant access to a dll, so you can either run the tests in the cloud or locally on a simulated device. To find out more, view this webinar: https://xamarin.wistia.com/medias/bdmfr3hk4d.

Mobile devices photo by HLundgaard (Own work). Licensed under CC BY-SA 3.0 via Wikimedia Commons.