Overview

New online technologies are not just changing who gets to innovate, but also for whom we can innovate. The goal of this project is to create activities that teach designers to quickly and more regularly connect with hard to reach populations through online social technologies. I led efforts in this project collaboration between Northwestern University and Carnegie Mellon University to develop instructional material for service design courses at the undergraduate and masters student level.

Understand

We performed an in-depth user study to understand current practices and challenges around using online technologies for user research. Our approach involved surveys, novice and expert task analyses, and developing personas, to inform our initial instruction goals.  
Top ways design students communicate with users. 

Surveys

​We surveyed 27 design instructors from the top US-based human-centered design programs to understand how design students currently perform user studies (see left). Instructors described four primary orchestration challenges when connecting design students and users: overcoming geographic barriers, time constraints and coordination, motivating user participation, and lack of student etiquette

Novice and Expert Task Analysis

 We performed cognitive task analyses with four designers (2 experts, 2 novices) where we asked them to identify user needs using social technologies. For example, we asked designers to identify five opportunities for redesigning the public transit experience using Twitter data. We screen and recorded participants as they described out loud their data collection and analysis process.

​We analyzed data from this task analysis by outlining and comparing the steps novices and experts took. Novices (those with little experience performing online design user research) had particular difficulty identifying what user data to collect and how to analyze the data thematically.

Personas

Given the goal to create a set of activities for service design courses, we developed two personas to highlight the values, personalities, and work habits of our target student population. These personas were ​
inspired from the survey and task analysis, as well as informal conversations with students currently enrolled in Northwestern's service design project courses. See example below of our primary persona.

Identify Learning Goals

We identified six primary tasks involved with performing user research through online social technologies: assess self, identify project needs, choose online platform, collect data, analyze data, 
and inform design. Moving forward, our goal was to develop a series of 30-60 minute activities that taught students how to perform and reflect on each of these tasks.

Create

To develop activities that fulfilled the identified learning goals, all team members came up with activity concepts that would teach students how to leverage online user research methods at different stages of the design process.

Ideate

Because this was an interdisciplinary project between Northwestern and Carnegie Mellon, we had to ideate and prototype concepts online rather than in person. Through independent ideation combined with weekly Skype calls, we came up with over 50 initial activity ideas in a shared Google Spreadsheet. We clustered these ideas based on different parts of the design stage (e.g. Needfinding, Testing). ​

Prototype

All activities were created following a research through design approach, in which we prototyped over 50 activities, internally tested 20 with our research team, and then deployed the best 10 in classroom settings. Prototypes initially took the form of a slide presentation and brief handout. Each activity followed the same general format of 1) Identify the challenge, 2) Identify why online user research is useful for this challenge, 3) Data collection, 4) Data analysis, and 5) Discussion.

Test

We tested our activities in Service Design Courses at Northwestern University and Carnegie Mellon University. We followed an iterative testing procedure where we continued to modify the activities after each testing session.

In-Class Testing

We tested activities with 172 students over 5 sessions (2 undergraduate courses, 2 masters courses, and 1 summer design studio). ​All sessions took place in human-centered design project courses. We collected testing data by ​taking notes during and after in-class testing and reviewing survey responses. 
​
​We found that student faced challenges finding particular user groups online and communicating thoughtfully and effectively. These insights informed design decisions to encourage platform-specific and general etiquette techniques, and to scaffold the creation of online discussion questions (see below).
We found that while students regularly used social media platforms for socialization, they were not particularly skilled at using these platform to researcher and understand user populations for design work. 
​Students were enthusiastic about the activities and some even introduced these methods to colleagues in their industry-based design internships after the class was over. Following the in-class testing, we sent a follow up survey to ask what students liked and disliked about the activity.
"The data amount is bigger than interviewing [sic]. In crowd-driven platforms,we see more about trends of people's opinion."
​“People have this alter ego online, so they feel like they can say anything. You don’t get that face-to-face.”

Activity Design

Our final activity design​s took into consideration insights we gained from iterative prototype testing. To limit the activity to 30 minutes, we broke activities down into data collection and data analysis. Each activity walked students through these two primary tasks using a toy scenario. The activities are currently being used in service design courses at Northwestern and Carnegie Mellon, and we continue to develop these activities based on student feedback. A more in depth overview of the curriculum design can be found here.

Publications

The Team
Hui, J., Gerber, E., Dow, S. (2014). "Crowd-Based Design Activities: Helping Students Connect with Users Online." ACM Conference for Designing Interactive Systems.
This project was funded by an NSF Cyberlearning Grant (2013-2014)

Julie Hui (Lead): PhD Candidate - Northwestern University
Liz Gerber: Professor - Northwestern University
Steven Dow: Professor - Carnegie Mellon University
Truc Nguyen: Undergraduate - Carnegie Mellon University