Evaluation I

Today I completed my first evaluation of the paper prototype. It took a bit longer than expected because last monday I did not have the time and because I’m in Leuven during the week I couldn’t test on my target users since I can only meet them in the weekends.

The evaluations went quite well and I received a lot of good feedback. Also all participants thought the application was really nice and usefull. These were the scenario’s I asked them to do:

Scenario’s for the parents:

  1. Login for your child and set that she/he is capable of reading and writing but needs control
  2. Set at the permissions everything that has to do with placing comments on “Hide”
  3. Go to the next screen (not a straightforward way)
  4. Set a password and continue
  5. Change the settings so the child cannot create groups anymore

Scenario’s for persons with a mental disability:

  1. Who was the last one that posted something on your wall?
  2. Update your status (This is set that the person is not allowed to do this so there will be popup to enter a password => What is the reaction?
  3. Start a new Facebook chat
  4. Make a new Facebook group
  5. Make a new Event

Afterwards I asked some questions to them:


  1. Age?
  2. Any experience with a touchscreen? Could be a smartphone,…
  3. Any with the iPad?
  4. Do you have a child with a mental disability or a child that is too young to realise any dangers in the world like abuse?
  5. Do you have any experience with Facebook?
  6. Were the settings clear enough? Any suggestions?
  7. Do you find the information panel handy?

Persons with a disability:

  1. Have you ever used Facebook?
  2. Do you have a Facebook account yourself? If not, interested in one?
  3. Any experience with a touchscreen? Could be a smartphone,…
  4. Any with the iPad?
  5. Do you think it would be handy to have like a button that would read all statusses of your friends aloud or do you prefer choosing the statusses that you want to hear yourself?

The goal of the evaluation was to see whether the application was usable by:

  1. Persons who might not have experience with multitouch devices and/or social networks
  2. People with a mental disability who might think totally different and look different at the world. Things that might seem logic for me could not be logic for then and the other way around!

I told the test users that this was a think aloud test so they had to say what they were thinking so I could understand why they were doubting about something, suggestions,…

For the test for the parents I found 5 parents willing to help me age 30-55. There were some general trends that I will shortly talk about:

  • For scenario 2, 4 out 5 users tapped hide for every place comment type instead of the general button that would hide them all at once. The reason they said was that it was not really clear that they were grouped. Some suggestions were putting the titles in bold, setting the margin for the subtypes more to the right, put them inside lines, in squares,…
  • 2 out of 5 did not find the solution for scenario 3 (they have to swipe to get to the new screen). When I told them what the solution was they said that they just not think of that because it is on paper but that when they see it on the device they might have tried that but they were not sure of it so it is definitely something I still have to check. The interesting here was that the 3 that found it all already had used an iPad and were able to put the link. My coordinator suggested to put a button for the next page permissions or a popup somewhere with some explanation of basic iPad controls for parents who do not have an iPad themselves but want to set the one of their child.
  • 2 out of 5 did not find the settings. Feedback was that the icon wasn’t that clear to them but if it was more a button they would probably have noticed it.

Some suggestions I received (also for the facebook part I showed to them):

  1. Possibility to add standard texts, like for example wishing someone a happy birthday. This way they can be proud they did something on their own AND the parents do not have to worry about it.
  2. In the application child is not really a good word. It is also for a grown-up person with a mental disability…
  3. Setting instead of allowing reactions on a status of a friend => More general on a post of a friend.
  4. Why showing birthdays on the profile of the person? Should it be there?
  5. Maybe it is better to move the application settings like permissions,…  to the general iOS settings => Otherwise child can still see that it does not have access to it. Might concern some children!
  6. Password in settings => Child could think it is the facebook password
  7. Like the standard text for wallposts you could do the same for status updates. You could add standard statusses like I’m happy, sad,… and also an option to type it yourself. This could also be very usefull for the third level (persons who need pictograms)

The test for persons with a disability was a little bit disappointing, the problem was that the paper prototype principle wasn’t really that clear for them but the biggest problem was that my paper prototype wasn’t visual enough for them. It was a lot of the same colors, perhaps not a very nice handwriting,… Seeing the difference between what is a button and text, does that represent a picture? And more like that were difficult for them.

For this reason I just did a test with two children without a disability to see whether basic Facebook functionality was still clear. The children had experience with Facebook. Some conclusions:

  • Tapping the existing status should allow updating the status
  • Sidebar for chat was handy and was immediately used
  • One child found that the sidebar draws more attention and was clearer than the tabs (needs further testing)
  • Pressed 2 times on text instead of the icon (Events and Notifications)

New storyboard & Meeting

After three really busy weeks packed with deadlines I was able to finish my new storyboard which shows you part 1 of my thesis. A Facebook application for people with a disability. The first two pages of the storyboard are for the parent or guardian of the person with a disability to set what he or she is able to do.

The third page shows how the standard application will look like if everything is enabled and if it is for a person of “level 1” who can read and understand perfectly but has problems with knowing who to trust or not, what is good online behavior and not,… So the parent will be able to restrict the possibilities of facebook. In that way the parent or guardian does not have to worry about those things anymore. So as you can see most of the things are standard things you can do on facebook but with a slightly different interface but the power of the application will be in the ability to block (changed but more on this later) certain features in the application.

For the people with a disability who have problems with reading but are able to understand (almost) everything there will be like planned in the original application text-to-speech options. The idea is that there will be a speaker like icon where the person can tap on and then select the words or buttons that he/she does not understand and those will then be spoken aloud.

The last category will be the people with a disability who have problems with understanding the words. For them I will try to convert statuses etc. to pictograms but for this I still need more information.

Like already mentioned, there is already a small change to the storyboard for setting the permissions. Now this is an ON-OFF button but my coordinator José pointed out that blocking instead of hiding a certain feature could cause frustration for the person with a disability. He/she might start wondering why he/she is not able to do that and others are,… For that reason it might be better to give the options ON | Hide | Block. So hide will really hide that feature in the app so the person with the disability will not see it exists. Block on the other hand will allow the person to see everything and for example allow to type a new status but when it really wants to commit the action then a prompt will show to type in the parent/guardian password.

Any suggestions and/or comments are much appreciated!

Idea’s after last meeting

During the last meeting with my coordinator José this thursday we concluded what the application should do:

  1. Like I concluded from the interview, the students will be able to use text-to-speech in the application. They should be able to select words, sentences or paragraphs in an easy way which the current voice-over functionality on the iPad does not.
  2. The application will gather all the words the students select to be read aloud.
  3. The teacher will be able to see this data in her application. In that way he/she is able to see if there are certain words the students select repeatedly. That way he/she can work on those words.
  4. This data can also be used to make a user profile of the student. This could for example be used in a web search application. When the student looks for certain topics, the application can do a risk evaluation of the resources and recommend or discourage visiting a website because there could be a lot of words in it where the user has problems with. This can avoid frustration. For this the IMS Acc4All (social tagging of the resources) and LIP (this specification defines a user profile) standards can be used.
  5. There will also be Facebook integration. This will be limited to accessing the classgroup’s discussion boards, the accessing of profile pictures for the teachers application. The reason the classgroup discussion board will be integrated is that in this way they will be working in an environment that protects them from some other facebook functionality that some people with a mental disability have problems with like explained in the post about the interview.
  6. Twitter feed for quick announcements from the teacher.
  7. Of course the application will also be able to do the “basic” things we would expect in an application that will be used in a class environment like an attendance checker, grade stats, calendar, assignments checker,…