Over the week, I tested with six people in total:
- The participants are all my friends or their friends. The hypothesis doesn’t put a specific constraint within YouTube user group. However, I do realize that there will be limitations in findings. For example, all the interviewees consider themselves to be tech-savvy people who understand the basics of how YouTube works.
Due to the intimate nature of the concept, I adjusted the prototype to each participants’ own YouTube data.
- I requested to have screenshots of the participants’ YouTube homepage. When they showed it to me, I asked them to click on a video that they wanted to watch the most at the moment, and took screenshots of 1. when it started playing, 2. when it was played more than half way, and 3. when the participants liked a comment below.
- I started the interview on the participants’ habits on YouTube, when sketching out the prototype on their screenshots. I constructed the attribute labels based on their usage and the reasons they picked the video.
The testing focuses on feedback of concept and usability. Here is a summary of the results.
- Concept wise, all six interviewees agree that it’s a bit clearer what the video recommendations are from. Now they know the demographic information and their implicit interactions contributed to the recommendations.
- Usability wise, most of the interviewees couldn’t understand the meaning of the score from the first glance. Without the additional information, they had to guess the meaning through seeing how the score changes after their set-up interactions with the video. (Though the prototype is a set of still images and therefore excluded possibilities for intuitive interactions, i.e. hovering.)
Most importantly, the transparency poses implications on how they might use the service in the future. It could enable a mindful reflection, or it could be a very radical change. When an interviewee who cares very much about her privacy described the approach as “brutal,” I knew that this is something that will initiate uncomfortable conversations that we as customers should’ve had a long time ago.
I’m currently writing a blog post as documentation of the process and official reveal of the project. It will be updated here soon.