Overview
Social media technologies provide an opportunity to involve online crowds in the design process. We developed and tested an approach called Communal Anonymous Feedback with the goal of improving quality of project feedback from social media networks, increase quantity of feedback for people with limited social networks, and increasing access to more diverse user populations. This project was in collaboration between Northwestern University and Carnegie Mellon University to develop instructional material for service design courses at the undergraduate and masters student level.
Understand
To better understand the factors affecting how students perceive social media, we conducted two preliminary surveys with students in a design course focused on needfinding, ideating, and prototyping mobile services. These surveys were distributed before and after an assignment where students were instructed to make a feedback request through their social media connections.
|
SurveysOverall, 30 students responded to the surveys. Results indicated that students were not accustomed to asking for favors, like deisgn feedback, on social media. Interestingly, students reported that they were more comfortable responding to help requests for others on social media. When asked where they would seek feedback, most students said they would prefer to collect feedback data through Facebook.
|
Create
Communal anonymous feedback involves distributing feedback providers between innovators to improve feedback quantity and diversity, and using double-blind anonymity to improve feedback quality.
Process DesignTo carry out communal anonymous feedback, we had teams in mobile app design course develop a prototype and written pitch of their project work. We then took their design material and created a general survey that would ask for feedback for a randomly chosen team. in effect, evenly distributing a the innovators collectively pool of feedback providers. We kept the identity of the poster anonymous by telling feedback providers that they were given a randomly selected project within their friend’s class. This approach leveraged the benefits of social capital (wanting to help friends), while minimizing evaluation anxiety on the feedback seeker’s part
|
|
Test
In order to test this approach, I performed an experiment to measure the effect of anonymity on feedback quantity and quality collected from social media networks on Facebook, Reddit, and Twitter. Contrary to our hypotheses that identifiable users would provide a higher quantity and quality feedback in exchange for social capital, I found that anonymous users were more likely to provide more detailed criticism and praise, which novice designers rated as more useful for their project work.
ExperimentWe performed a content analysis of each feedback comment by dividing them into individual idea units, which were standalone thoughts, and coded these units based on feedback type. We found that Students rated Specific-praise, Specific-criticism, and Directive suggestions as significantly more useful, and feedback providers were more likely to give Specific-praise and Specific-criticism in the anonymous condition.
|
|
Final Approach
These findings add to emerging crowdsourcing research on how to better facilitate and motivate online requests for complex tasks, like providing open-ended feedback. Design activities from this work have been implemented in courses at Carnegie Mellon University, Northwestern University, and University of California San Diego
Publications |
The Team |
|
Hui, J., Glenn, A., Jue, R., Gerber, E., Dow, S. (2015). "Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced Feedback." AAAI Conference on Human Computation and Crowdsourcing.
|
This project was funded by an NSF Cyberlearning Grant (2013-2014)
Julie Hui (Lead): PhD Candidate - Northwestern University Liz Gerber: Professor - Northwestern University Steven Dow: Professor - Carnegie Mellon University Truc Nguyen: Undergraduate - Carnegie Mellon University Amos Glenn: Instructional Designer - Carnegie Mellon University |