Q2 Bug Battle Report: 2009
Battle of the Desktop Twitter Apps
Bug Battle Basics
After tackling internet browsers and social media sites, uTest’s Q2 Bug Battle challenged testers to find bugs in the top five Twitter desktop applications. Combining aspects of functional and usability testing, the uTest community was given one week in June to report the most compelling bugs they could find and to rank these apps based on quality, usability and feature set, with nearly $4,000 in prize money on the line.
The Apps
The “Twitterverse” is big – really big. So when it came to choosing the top applications, we had to be selective. After asking the experts and doing some of our own research, we settled on these popular Twitter desktop apps. Here they are, in no particular order: 1. 2. 3. 4. 5. Tweet Deck Twhirl Tweetr Twitteroo Seesmic Desktop
The Rules
Participating testers were asked to scour these apps for bugs – and then we asked them to submit user feedback – ranking each app they tested based upon usability, feature set and overall quality. After carefully reviewing each submitted bug and survey response, uTest then awarded prize money based on a number of different criteria, including best bugs, best feedback and several Top Tester awards.
The Results
In all, more than 600 testers from 29 countries took part in the week-long competition, finding roughly 320 bugs in the process. What became apparent from the get-go was that three of the Twitter apps – Tweet Deck, Seesmic Desktop and Twhirl – were far and away the most popular in terms of usability, feature set and overall quality (see summaries on the next page). The top tester of the Bug Battle was Bernard Lelchuk of Israel, who reported the most valuable bugs and provided the best testing feedback. Other winners include: Joseph Ours, who received the Top US Tester award and Tyler Ritchie who received the Top Novice Tester award. More details, including a complete list of the winners, can be found at http://www.utest.com/bug_battle. Continue reading…. uTest Inc. | 153 Cordaville Road | Southborough, MA 01772 | www.utest.com | 1.800.445.3914
Survey Rankings (based upon 141 respondents)
Testers’ choice for best overall quality: 1. Tweet Deck 2. Seesmic Desktop 3. Twhirl 4. Tweetr 5. Twitteroo Testers’ choice for best feature set: 1. Tweet Deck 2. Seesmic Desktop 3. Twhirl 4. Tweetr 5. Twitteroo Testers’ choice for best usability: 1. Tweet Deck 2. Seesmic Desktop 3. Twhirl 4. Tweetr 5. Twitteroo Testers level of experience: o 12% had less than 1 year o 41% had 1-3 years o 24% had 3-5 years o 16% had 5-9 years o 6% testers had more than 5 years
Tweet Deck: The Twitter Hat Trick Taking first place in all three feedback categories, Tweet Deck was easily the most favored Twitter app in the Bug Battle. Despite its appeal, testers found a combined total of 77 bugs. Of that number, 13% were classified as “showstoppers” (i.e. bugs in need of immediate attention), while 35% were deemed as high severity. Regarding the types of bugs found, Tweet Deck had 56% of its bugs reported as functional, with GUI (14%) and Technical (30%) rounding out the totals. Common positive feedback from the testing community: o “Easiest access to Twitter actions (i.e. retweet, reply, direct message, etc.)” o “Easy to use – A well-designed application” Common negative feedback from the testing community: o “Inadequate keyboard functionality; menu icons not intuitive” o “Requires too much memory”
Tester Feedback
“With Twitter, I need a third party app that doesn’t take over my desktop, especially while I’m working. And to be honest, I don’t need loads of features; just simple alerts when tweets are updated, and the ability to update them on my own. For me, Tweet Deck is that app.”
Continue reading…
uTest Inc. | 153 Cordaville Road | Southborough, MA 01772 | www.utest.com | 1.800.445.3914
Seesmic Desktop: Top-notch Tweet Management A close second to Tweet Deck, Seesmic Desktop was also well-received by the uTest community, with more than 60% of participants saying they would likely recommend it to a friend. Testers identified 80 bugs, 20% of which were deemed as showstoppers, while nearly 45% of the bugs were classified as “high severity.” And of the total number of bugs, 50% were Functional in nature, with 19% GUI and 31% Technical. Common positive feedback from the testing community: o “Very solid application. Easy to organize, easy to share” o “Provides easy ways to post images and website URLs” Common negative feedback from the testing community: o “Difficulty in organizing tweets; not enough columns” o “Very little useful feedback for new messages”
Twhirl: Up and Coming Ranking third in every category, Twhirl was close to Tweet Deck and Seesmic in terms of maturity, but testers noted that the application has more room for improvement than its competitors. Of the 61 bugs indentified, 16% were deemed as showstoppers, with 33% marked as high severity. In terms of categorization, 38% of the bugs found were Functional in nature, with 18% GUI and 44% Technical. Common positive feedback from the testing community: o “Tweet notification features” o “Compatible with other Twitter apps and social media sites” Common negative feedback from the testing community: o “Placement of the icons confusing to novice Twitter users” o “Inconsistent search functionality”
About uTest
uTest is the world's largest marketplace for software testing services. The company provides real-world testing services through its community of 17,000+ professional testers from 152 countries around the world. More than 1,000 companies - from web startups to global software leaders - have signed up for the uTest marketplace to get their web, desktop and mobile applications tested. More information can be found at http://www.utest.com.
uTest Inc. | 153 Cordaville Road | Southborough, MA 01772 | www.utest.com | 1.800.445.3914