Analytical usability evaluation of the prototype
After we completely redid the prototype using sticky notes, we had two flatmates use it under
our guidance to track down ambiguities and misunderstandings.
We found that the prototype is quite good and our design is also quite intuitive. The main
problems our testers had were caused by them not exactly knowing what was “clickable” and
what was not. This was a problem especially in the Shopping Cart view when they had to
change the quantity of an item. They were lead to rather hit the Edit button than to click on
the quantity itself. Also was it not clear that the little arrow in the bottom right corner means
that you can scroll down.
Another suggestion one of the testers made after they weren‟t quite sure what the minus signs
or cross signs in several lists (like the Bookmarks (2.1) or the Compare (3.1) screens) meant
was to use the same symbol for every remove action and that should be the one used in the
Shopping Cart (6.1) featuring the little bin.
This shows one important thing: it confirms one of the „fears‟ of high-fidelity model users.
The users did actually pick up on little ambiguities like pictures and font size. On the other
hand though, the general idea seemed acceptable; the prototype shows how the device is to
use and works in a clear and simple way and users understood that way immediately.
One user did actually detect a design flaw in the big picture: when you bookmark items, you
cannot choose a colour or a material yet when you compare two items which, after all, you
choose from Bookmarks then they do show these properties. This is just a minor flaw and we
decided to replace „Colour‟ and „Material‟ in the Compare table by „Available colours‟ and
This shows however that our prototype is appropriate: it enables us to check that the device as
it is now can indeed be used and that the whole navigation hierarchy makes sense or – in case
it does not – makes us realise it does not and fix it.
Please consult the prototype while reading this, as different screens are being referred to by
the number on the back of the corresponding sticky notes.
- Starting at screen 0.1 the user was asked what he would do supposing he does not have an
account. All pressed the button saying Guest.
- Then at screen 0.2 they had a look for a few seconds but found out it is a page about
advertisements. We did not treat this page further.
We then supposed the user had found a piece of furniture and scanned its tag.
- Screen 1.1: the users had a look at all the information and found it quite clear and
straightforward. As they were asked to change the colour or the material they pressed the
corresponding buttons on the screen.
- On the same screen, they were asked to scroll down. This created some confusion about
the two pairs of scroll buttons: the hardware ones and the on-screen ones next to the
description. However this would be clarified in real situations by a minimalist trial-and-
error. The user could e.g. press the hardware buttons to scroll the textbox and see the
whole screen scroll instead.
- Users responded to the queries “try to add this to your shopping cart” and “try to
bookmark this” by pressing the Buy and Bookmark buttons on the screen.
- After pressing Bookmark, we find ourselves back at screen 2.1. The users would press the
little cross when they were asked to remove an item from the list and they would just
press the arrow to see what will happen. This would lead them back to screen 1.1.
- If the user pressed Buy then we see screen 6.1. Users would press the bin to remove an
item from the Shopping Cart and assumed that Edit meant changing colours and
- A pressing of Edit in the Shopping Cart would lead them to screen 1.2, followed by a
surprised by pleased exclamation! They would easily press the right buttons in order to
change the colour or material and Update to get back to the Shopping Cart.
- One user thought that the quantity was altered by pressing Edit as well, yet they tried
touching the quantity itself after they realised they were wrong. The other user pressed
the quantity straight away. They were shown the structure that would pop up (can be seen
in brown font on screen 6.1) and said they understood it well.
We then asked the users to try and compare two pieces of furniture they already carried in
their virtual basket. One user had to be told that the device featured hardware buttons above
the screen because he focused too much on the screen; this would probably not happen with
the real device since you can see the buttons as such.
- One user hit the button with the magnifier first because they thought that was the right
one to compare. When the screen showed up, it said Search in the title, so the user
understood this meant searching. On the second try that user pressed the right button, as
did the other user on their first try.
- This led them to screen 3.1. We had to explain them that the list would actually be empty
at the beginning. This would not be necessary had we drawn more screens in the
Compare category. The users pressed the field that says New Item in order to add an item.
Note: listen on whole field to add item, but only on the delete button to delete an item!
- After that users were presented with screen 2.2. They both pushed the right button in
order to select the items to compare. However, they also both wanted to press the add-
button of two different items in one go. If they had used the real device, it would have
loaded screen 3.1 after the first button though. The users were explained this and seemed
okay with it. They repeated the procedure to add a second item to the list and pressed the
Compare button on screen 3.1 to start comparing.
- On screen 3.2 they checked the table out and said they understood it. One user
experimented with the Back button to find out it leads back to screen 3.1 and pressed 3.2
again to come back to screen 3.2.
- Users said if they wanted to buy or see a product they would intuitively touch the name of
it. One user specified they would do it „if it looks like you can click it‟.
Users then were told they had seen a chair they liked and they remembered its name and they
should search for it now.
- Both users instantly pressed the magnifier button above the screen. Screen 4.1 came up.
- Users then simulated typing something on the on-screen keyboard. One user said they
would hit enter afterwards but decided to rather press the Search button when we made
them realise there is no enter key. This feature could however be easily implemented for
the users‟ ease. The other user would hit the Search button after typing.
- On screen 4.2, users quickly read the list and had no trouble pressing the arrow button to
be led to screen 1.1 again.
No tests were conducted involving the calculator. Users handled the device separately, having
no opportunity to talk in-between their consultations.
Finally, we judge that our prototype is very good and quite effective. It is a mid-fidelity
prototype as it is very basic on one hand, leaving no room for real user input but has icons
and a specific design already, making testers pick on these things instead of contemplating
the big picture only.