Evaluation II continued: Feedback of an expert

Since I was only able to test the mockup with one person I contacted an expert to give his opinion on the application and check whether this can be used by most of the persons with a mental disability. When I showed the application the immediate reaction was really positive. He thinks this application can really be a way to close the social gap some more for a lot of people with a mental disability.

For the dilemma for the selection of text and then reading it out loud or the other way around the expert responded that it might be better to choose for first selecting and then reading it loud because in this way it is consistent with other reading applications like Sprint and Kurzweil which some of them already use. For the buttons then it might be a solution when tapped to just read it out loud anyway and also navigate to the page. Facebook does not really have a deep hierarchy of pages so going back to the previous page  is not that difficult in this application.

Some other suggestions were to use sclera pictograms for the statuses with a sub hierarchy, like I feel… and when they select this you go to happy, sad,… The same could be done for some standard texts they could put on the walls of friends like: “I would like to…”, “Let’s go to…”,… Another nice thing would be the possibility to change font and font size, this preference (in the way the person can read it best) is really different from person to person.

Evaluation II

Yesterday I tested the mockup shown in the previous post with a person with a mental disability. I adapted the mockup a little more so the text in the tabbar is replaced by icons. The person I tested the mockup with is 20 years old.

Scenario’s for persons with a mental disability:

  1. Who was the last one that posted something on your wall?
  2. Let something be read out loud (for example a wallpost)
  3. Update your status (This is set that the person is not allowed to do this so there will be popup to enter a password => What is the reaction?
  4. Who’s birthday is it today?
  5. When is event X?
  6. Can you read out loud what person X said to person Y?
  7. Start a new Facebook chat
  8. Can you see who are your friends?
  9. Of which groups are you member?

Afterwards I asked some questions to them:

Persons with a disability:

  1. Have you ever used Facebook?
  2. Do you have a Facebook account yourself? If not, interested in one?
  3. Any experience with a touchscreen? Could be a smartphone,…
  4. Any with the iPad?
  5. Do you think it would be handy to have like a button that would read all statusses of your friends aloud or do you prefer choosing the statusses that you want to hear yourself?
  6. Any remarks?

The goal of the evaluation was again to see whether the application was usable by people with a mental disability who might think totally different and look different at the world. Things that might seem logic for me could not be logic for then and the other way around! This time the evaluation is with a mockup so it should go better than with the paper prototype.

I told the test users that this was a think aloud test so they had to say what they were thinking so I could understand why they were doubting about something, suggestions,…

Results:

Person 1:

I gave the iPad with the application open and the first thing that the person did was trying to scroll through the news feed which, because it is a mockup, is not possible yet but it was nice to see that the person with the mental disability knows how to work with iOS devices.

Then I asked to do the different scenario’s:

  1. The first step was to get to his profile and so the person tried to tap his picture which is quite logic but it was not implemented that way and I asked if there might be another to do this. The person then immediately tapped the profile tab and pointed the correct person that posted something on his wall and even pressed the speaker so it would be read out loud.
  2. For the reading out loud I hadn’t explained anything so when I asked to do this for something where there was no small speaker icon, the person tried to select some text because he thought there might show something then. I then told the person that there might be another way to do this and hinted to the box icon. Then the user tried to select the text again and then pressed the speaker icon. I then explained how it actually works and it also seems logic to the person but he was just thinking in another order.
  3. The person immediately found the update status bar, but when he tapped on it and the password prompt showed he immediately tapped somewhere else to remove it because he thought he did something wrong. I then explained that he was right but that the application might be setup to offer some protection for certain actions.
  4. Has some trouble with finding this, saw a number next to chat and was focused on that being the day and thought first that the pictures from the chat were people with a birthday today. Then he saw the gift and associates that with perhaps a birthday but doesn’t understand the text. He then pressed the text because he thinks that should give birthdays but then I said that it would also be a way to see the birthdays but he should search less far. Then he saw the pictures but said that those persons already had their birthday, so it couldn’t be them… Which is kind of correct since he knows the persons on the pictures. This is an example on the difficulty sometimes to test certain things (like with the paper prototype) because they have trouble living into something not real. They can “only” think really logic. So these persons don’t have their birthday today, so it cannot be that…
  5. Immediately saw the calendar icon and used the speaker to read it out loud.
  6. For this the person again selected the large speaker icon and selected the text. I asked then if there would have been another way and the person immediately responded selecting the small speaker icon.
  7. First clicked on friends icon, then groups and then messages. He did not find the solution for this because the concept of chatting was new for him.
  8. First selected profile but then directly the friends icon and immediately tried to scroll through the pictures with the coverflow mechanism. Said that it was like his music on the iPod.
  9. Tapped the correct icon but when on the page he wasn’t sure if it was the right page.

Then I asked the questions:

  1. Yes, always used it with his mother.
  2. Yes
  3. Yes, has an iPod Touch like already mentioned.
  4. No
  5. No idea
  6. Quote: “I think you deserve a 10/10 for what you have made”, hopefully everyone thinks this?

What I can conclude from the evaluation with this person is that apart from some small interface changes the application is usable by already at least one person with a mental disability. I hope to be able to test the mockup with some more persons with a mental disability very soon.

An update: Mockup

It has been a while since I posted an update because of a lot of projectdeadlines and exams, but the last couple of days I’ve been working on the mockup of my application. I made this with the interface builder in Xcode and used some screenshots from other applications and also from the Facebook website. The plan is to have the mockup finished by tomorrow and then test it with some persons.

The segmented control at the top still needs to be changed because it should have pictograms instead of text.

To create this I had to make Controllers and Views to get some interaction so I designed the following Class diagram. The next step after checking whether the mockup works will be designing a model and further implementing the controllers.

Presentation I: Before holidays

Here is the presentation I gave thursday 16 December:

Feedback I received:

  • More pictures of examples on Facebook where the persons with a mental disability have problems with: For example the top navigation bar: Home – Profile – Account => Really sober
  • Contact the Facebook accessibility team
  • Free more time for the last evaluation of the implemented part I application
  • Number the slides

Other presentations:

  • One of the most interesting presentations in my opinion was the one about Augmented Reality by Niels Buekers. I really think this is an interesting topic and has something futuristic. If this would actually come out in real life I would were glasses even though I don’t need them! The only remark I have on the presentation was that it would have been nice to see some pictures but I understand this is harder then for other topics.
  • The presentation of Michaël Vanderheeren was also very nice. It was really structured and the goals were clear and came from logical conclusions.

Of course the other topics were also interesting, but personally for me the two above really sticked. One because of the topic and the other just because of the great work already done. This proofs how useful those presentations were. The blogs gave some explanation but it was all a bit vague. Thanks to the presentation we all have a better idea now on were the other students stand, feedback on your work,…

Evaluation I

Today I completed my first evaluation of the paper prototype. It took a bit longer than expected because last monday I did not have the time and because I’m in Leuven during the week I couldn’t test on my target users since I can only meet them in the weekends.

The evaluations went quite well and I received a lot of good feedback. Also all participants thought the application was really nice and usefull. These were the scenario’s I asked them to do:

Scenario’s for the parents:

  1. Login for your child and set that she/he is capable of reading and writing but needs control
  2. Set at the permissions everything that has to do with placing comments on “Hide”
  3. Go to the next screen (not a straightforward way)
  4. Set a password and continue
  5. Change the settings so the child cannot create groups anymore

Scenario’s for persons with a mental disability:

  1. Who was the last one that posted something on your wall?
  2. Update your status (This is set that the person is not allowed to do this so there will be popup to enter a password => What is the reaction?
  3. Start a new Facebook chat
  4. Make a new Facebook group
  5. Make a new Event

Afterwards I asked some questions to them:

Parents:

  1. Age?
  2. Any experience with a touchscreen? Could be a smartphone,…
  3. Any with the iPad?
  4. Do you have a child with a mental disability or a child that is too young to realise any dangers in the world like abuse?
  5. Do you have any experience with Facebook?
  6. Were the settings clear enough? Any suggestions?
  7. Do you find the information panel handy?

Persons with a disability:

  1. Have you ever used Facebook?
  2. Do you have a Facebook account yourself? If not, interested in one?
  3. Any experience with a touchscreen? Could be a smartphone,…
  4. Any with the iPad?
  5. Do you think it would be handy to have like a button that would read all statusses of your friends aloud or do you prefer choosing the statusses that you want to hear yourself?

The goal of the evaluation was to see whether the application was usable by:

  1. Persons who might not have experience with multitouch devices and/or social networks
  2. People with a mental disability who might think totally different and look different at the world. Things that might seem logic for me could not be logic for then and the other way around!

I told the test users that this was a think aloud test so they had to say what they were thinking so I could understand why they were doubting about something, suggestions,…

For the test for the parents I found 5 parents willing to help me age 30-55. There were some general trends that I will shortly talk about:

  • For scenario 2, 4 out 5 users tapped hide for every place comment type instead of the general button that would hide them all at once. The reason they said was that it was not really clear that they were grouped. Some suggestions were putting the titles in bold, setting the margin for the subtypes more to the right, put them inside lines, in squares,…
  • 2 out of 5 did not find the solution for scenario 3 (they have to swipe to get to the new screen). When I told them what the solution was they said that they just not think of that because it is on paper but that when they see it on the device they might have tried that but they were not sure of it so it is definitely something I still have to check. The interesting here was that the 3 that found it all already had used an iPad and were able to put the link. My coordinator suggested to put a button for the next page permissions or a popup somewhere with some explanation of basic iPad controls for parents who do not have an iPad themselves but want to set the one of their child.
  • 2 out of 5 did not find the settings. Feedback was that the icon wasn’t that clear to them but if it was more a button they would probably have noticed it.

Some suggestions I received (also for the facebook part I showed to them):

  1. Possibility to add standard texts, like for example wishing someone a happy birthday. This way they can be proud they did something on their own AND the parents do not have to worry about it.
  2. In the application child is not really a good word. It is also for a grown-up person with a mental disability…
  3. Setting instead of allowing reactions on a status of a friend => More general on a post of a friend.
  4. Why showing birthdays on the profile of the person? Should it be there?
  5. Maybe it is better to move the application settings like permissions,…  to the general iOS settings => Otherwise child can still see that it does not have access to it. Might concern some children!
  6. Password in settings => Child could think it is the facebook password
  7. Like the standard text for wallposts you could do the same for status updates. You could add standard statusses like I’m happy, sad,… and also an option to type it yourself. This could also be very usefull for the third level (persons who need pictograms)

The test for persons with a disability was a little bit disappointing, the problem was that the paper prototype principle wasn’t really that clear for them but the biggest problem was that my paper prototype wasn’t visual enough for them. It was a lot of the same colors, perhaps not a very nice handwriting,… Seeing the difference between what is a button and text, does that represent a picture? And more like that were difficult for them.

For this reason I just did a test with two children without a disability to see whether basic Facebook functionality was still clear. The children had experience with Facebook. Some conclusions:

  • Tapping the existing status should allow updating the status
  • Sidebar for chat was handy and was immediately used
  • One child found that the sidebar draws more attention and was clearer than the tabs (needs further testing)
  • Pressed 2 times on text instead of the icon (Events and Notifications)