Earlier this summer I wrote about our experience running our first usability tests. Last week we ran our second round of testing, this time with an extra challenge: mobile testing. Here’s a quick recap of the options we explored and what we ended up doing.
Our requirements were fairly simple: we wanted to record the screen of an iPhone, as well as audio and video of the user. Some people build wacky contraptions that point a small camera at the screen during the testing session. We feel like this makes the experience less natural for the testers and were intent on avoiding it by capturing the screen directly. We quickly realized that all the available ways for doing this required jailbreaking the phone. We used the JailBreakMe site to do this.
Once we had our jailbroken phone, the solution was fairly simple. On the iPhone we used Veency, a VNC server for jailbroken iPhones. We used Chicken of the VNC on the Mac to mirror the iPhone on the desktop monitor. We then used Silverback to capture both the iPhone output and audio and video of the tester (we had them sit in front of the main computer while using the iPhone).
This isn’t a perfect solution: we couldn’t capture click events and the output from the iPhone was a bit choppy. Also, the video we captured didn’t show the users hands actually interacting with the phone.
We looked at two other options as well:
ScreenSplitr on the iPhone in combination with iDemo on the Mac
Pros: ScreenSplittr transmits the iPhone signal via USB so there is less delay, resulting in smoother animations.
Cons: Having the iPhone connected to the Mac with a USB cable is less natural for user testers. It also doesn’t show where the user touches the screen.
Pros: Records where user touches the screen. Transfers live video right to a browser on a Mac, so no additional software is needed on the Mac side. It also records everything right on the iPhone leading to smooth animations in the final output.
Cons: Requires iPhone 3GS. We weren’t able to test this out ourselves. We would need to take the recorded screencast from the iPhone and match it to the recording of the user.
A big thanks to our intern Matthias for his hard work in tracking down and trying out these solutions.
