Of course, we don't want to become the official naysayers of all studies done in the mobile world, but when doing a usability study on smartphones, it seems a bit silly to limit the test to just 3 tasks: making a call, adding a contact, and sending a text. Those three tasks are probably the least likely tasks for a smartphone user to perform (or at least the first two of the three tasks), especially when there is music to hear, pictures to take, videos to watch, web pages to browse, e-mails to read, and Angry Birds to launch. At the very least, it seems that the study could have added purchasing an app from each platform's app store, or performing a web search.
Each test was rated from 1 to 5 stars, 5 being that the task was very easily performed. iPhone and WP7 each had a total of 11 stars for the 3 tasks, followed by Android with 9, and BlackBerry with 8. Android was the only OS to score a perfect 5 on a task, and that was for making a call, although that was followed by a 1 in adding a contact. There should be a couple of notes made on the study as is: first, the Android phone used was an HTC device meaning it had the Sense UI, and didn't include the Contacts app on the homescreen. Other UIs and phones include that on the homescreen which theoretically could have made this task easier to complete. Alternatively, in the sending a text task, the only user that didn't have to initiate a new SMS was the iPhone user, who merely replied to an SMS (and seemingly didn't know how to get back to the Messaging home screen.)
That said, it seems like the real winner may be Windows Phone, but as mentioned before, there is far more to using a smartphone than the 3 tasks covered here. And, the minimalist icons of Windows Phone did seem to lead to confusion, which may hurt overall usage. As always, UI usability could just be a matter of personal preference, with the exception of BlackBerry, which just needs a major overhaul. Of course, we'd need to see a more complete study to come to any real conclusions, but this was interesting nonetheless. Check out the full video below to see the fumbling noobs!
source: Human Factors via WMPU