Today I completed my first evaluation of the paper prototype. It took a bit longer than expected because last monday I did not have the time and because I’m in Leuven during the week I couldn’t test on my target users since I can only meet them in the weekends.
The evaluations went quite well and I received a lot of good feedback. Also all participants thought the application was really nice and usefull. These were the scenario’s I asked them to do:
Scenario’s for the parents:
- Login for your child and set that she/he is capable of reading and writing but needs control
- Set at the permissions everything that has to do with placing comments on “Hide”
- Go to the next screen (not a straightforward way)
- Set a password and continue
- Change the settings so the child cannot create groups anymore
Scenario’s for persons with a mental disability:
- Who was the last one that posted something on your wall?
- Update your status (This is set that the person is not allowed to do this so there will be popup to enter a password => What is the reaction?
- Start a new Facebook chat
- Make a new Facebook group
- Make a new Event
Afterwards I asked some questions to them:
Parents:
- Age?
- Any experience with a touchscreen? Could be a smartphone,…
- Any with the iPad?
- Do you have a child with a mental disability or a child that is too young to realise any dangers in the world like abuse?
- Do you have any experience with Facebook?
- Were the settings clear enough? Any suggestions?
- Do you find the information panel handy?
Persons with a disability:
- Have you ever used Facebook?
- Do you have a Facebook account yourself? If not, interested in one?
- Any experience with a touchscreen? Could be a smartphone,…
- Any with the iPad?
- Do you think it would be handy to have like a button that would read all statusses of your friends aloud or do you prefer choosing the statusses that you want to hear yourself?
The goal of the evaluation was to see whether the application was usable by:
- Persons who might not have experience with multitouch devices and/or social networks
- People with a mental disability who might think totally different and look different at the world. Things that might seem logic for me could not be logic for then and the other way around!
I told the test users that this was a think aloud test so they had to say what they were thinking so I could understand why they were doubting about something, suggestions,…
For the test for the parents I found 5 parents willing to help me age 30-55. There were some general trends that I will shortly talk about:
- For scenario 2, 4 out 5 users tapped hide for every place comment type instead of the general button that would hide them all at once. The reason they said was that it was not really clear that they were grouped. Some suggestions were putting the titles in bold, setting the margin for the subtypes more to the right, put them inside lines, in squares,…
- 2 out of 5 did not find the solution for scenario 3 (they have to swipe to get to the new screen). When I told them what the solution was they said that they just not think of that because it is on paper but that when they see it on the device they might have tried that but they were not sure of it so it is definitely something I still have to check. The interesting here was that the 3 that found it all already had used an iPad and were able to put the link. My coordinator suggested to put a button for the next page permissions or a popup somewhere with some explanation of basic iPad controls for parents who do not have an iPad themselves but want to set the one of their child.
- 2 out of 5 did not find the settings. Feedback was that the icon wasn’t that clear to them but if it was more a button they would probably have noticed it.
Some suggestions I received (also for the facebook part I showed to them):
- Possibility to add standard texts, like for example wishing someone a happy birthday. This way they can be proud they did something on their own AND the parents do not have to worry about it.
- In the application child is not really a good word. It is also for a grown-up person with a mental disability…
- Setting instead of allowing reactions on a status of a friend => More general on a post of a friend.
- Why showing birthdays on the profile of the person? Should it be there?
- Maybe it is better to move the application settings like permissions,… to the general iOS settings => Otherwise child can still see that it does not have access to it. Might concern some children!
- Password in settings => Child could think it is the facebook password
- Like the standard text for wallposts you could do the same for status updates. You could add standard statusses like I’m happy, sad,… and also an option to type it yourself. This could also be very usefull for the third level (persons who need pictograms)
The test for persons with a disability was a little bit disappointing, the problem was that the paper prototype principle wasn’t really that clear for them but the biggest problem was that my paper prototype wasn’t visual enough for them. It was a lot of the same colors, perhaps not a very nice handwriting,… Seeing the difference between what is a button and text, does that represent a picture? And more like that were difficult for them.
For this reason I just did a test with two children without a disability to see whether basic Facebook functionality was still clear. The children had experience with Facebook. Some conclusions:
- Tapping the existing status should allow updating the status
- Sidebar for chat was handy and was immediately used
- One child found that the sidebar draws more attention and was clearer than the tabs (needs further testing)
- Pressed 2 times on text instead of the icon (Events and Notifications)