I figured that it would be great to kick off my new-found motivation to blog with a post going over how I do mobile testing. You have to do testing, it has a high cost of entry, and can be intimidating. Let’s walk through how I do it as a entrepreneur/consultant:
First, let’s establish some basic ideas:
- If you’re not testing on-device, you’re not testing. While handy for spot-testing as you code, the iOS Simulator is unrealistically fast, and the Android Emulator is slow, buggy, and difficult to use. Also, some things won’t work or break until you test with a real build, like push notifications or Google Maps API keys.
- If you want to test your app on every current iPhone, you currently have 4 devices to test on. If you want to do that for Android, you will want to win the lottery, and rent a warehouse.
- Other things are harder to sim/emulate, like available processor & memory on a phone that has real-world usage. There’s a lot more happening on a phone that has active email/calendar/push connections, and has other apps open in the background.
That being said, all is not lost. I’ve formed an effective strategy to maximize my testing coverage. First off, I buy the latest iPhone when it comes out. That’s the easiest decision, since we’re talking one phone per year. Android requires a bit more clever thinking, though.
- Buy new Nexus devices. Google actually keeps these phones up to date on reasonable schedules, unlike any other Android phones on the market. If you look at the recent Nexus history, the Nexus One, Nexus S and Galaxy Nexus, I bought the Nexus One, skipped the Nexus S, and bought a Galaxy Nexus. This gave me a “latest & greatest” OS to work on, all the time. I’ll probably skip the next Nexus device, unless there’s some significant deficiency, either on the hardware or software side.
- Buy used “carrier” devices. Cracked screen? Big ugly gash in the frame? As long as it’s functional, who cares? Craigslist and eBay are great places to find cheap phones.
- Diversify brands and models. I’m sure you have a favorite, but the sad reality is that the manufacturers like to put their big, ugly thumbprints on their phones. HTC Sense, MotoBLUR, and Samsung TouchWiz introduce nuances and bugs that will be unique to their phones.
Personally, I try to buy one phone from each manufacturer for each major version of Android. I try to stick to the most popular model for each, because the odds are in my favor that I’ll have the phone that my client has. If you’re keeping count, that means that each calendar year I plan on buying one new iPhone, one new Nexus, and 3-4 used Android phones.
The hidden problem I encountered was that it’s hard to carry around so many phones. I looked for multiple-phone carrying cases, but couldn’t find anything. I eventually figured out that Pistol Cases
Yes officer, that’s my pistol case. No sir, I have it because Android is fragmented and I’m a developer.
I just cut out slots in the foam, and used a bit of spray glue to attach the foam to the case. Cost me $10 at my local sporting goods store, plus the glue. You can see it’s holding 8 phones, a MiFi, and still has room for three more. Luckily, while Android phones might do everything else differently, they all share the same USB/Charing cable. I could carry everything I need in that one little case, at least for the next year or so.