Jan 6, 2017

Service Design: Verbal Communication and Presentation Practice Service

Team Size: 6 people
Time Period: 2 months
Role: UX Researcher, Technical Researcher
Tools: Interviews, Personas, Scenarios, Service Blueprint, Storyboarding, Speed Dating, User Enactments, Survey Design, Wizard of Oz Testing

We sought to design an on-campus service that aims to improve the communication skills of students at Carnegie Mellon University. This was a final requirement for Service Design, an elective I took when I was completing my Master’s degree in Human-Computer Interaction at Carnegie Mellon University

Overview

Problem

Good communication skills are important to have for any student. From growing your professional network, to sharing your past work during interviews, to collaborating with others in a team, getting your ideas across to other people is crucial to any workplace. For engineering students in particular, communication skills are brushed off as not important and as a result are often underdeveloped. Our goal was to create a service that enables students to practice and improve their communication skills.

Process

Collect

Literature Review
Interviews
Competitive Analysis

Understand

Customer Journey Map
Stakeholder Map

Ideate

Personas
Scenarios

Test

Speed Dating
User Enactments

We first gathered in-depth information on how well students were using their communication skills in and out of class as well as what opportunities and resources for improvement are available to them. Using our models, we focused on identifying pain points that each of our stakeholders have as well as how each stakeholder can directly and indirectly affect other stakeholders. After creating potential communication training services that solve a combination of the different pain points we have discovered, we ran it through a two-stage needs validation process that involves doing a low-fidelity speed dating session for initial feedback followed by a higher-fidelity user enactments session for testing our prototypes.

Solution

Our proposed solution, Echo, uses a combination of virtual reality, speech recognition, and computer vision, to track voice quality and body language while a student conducts to a virtual (through the Echo Chamber practice room) or a live, in-class audience. By providing students with metrics on the quality of their presentation, we hope to make more students aware of these small but important aspects of of their speech that can also serve as a way for them to track their improvement over time.

Phase 1: Research

Highlight: My contributions include interviewing an Alumna from CMU and gathering data for a competitive analysis comparing resources available at Carnegie Mellon to those from other universities

During our research phase, we made some significant discoveries:

Linear Thinking. Engineering students are good at linear thinking, that is, following a set of steps to achieve their goal. Frameworks or reusable templates have been found to be very effective tools engineering students can use for writing but have a tendency to produce very dry writing.
Content Over Delivery. Engineering students tend to brush off communication skills as irrelevant as professors often focus on providing feedback on the content as opposed to the manner on which it was presented.
Spacing: English courses are important in developing students’ communication skills. It is important for academic curricula to evenly space these courses out over several semesters, giving students ample time to absorb and digest these vital communication skills.
Time and Incentive: Since engineering students don’t see communication skills as a priority, they tend to lack the incentive to invest the time needed to improve.

Apart from these findings, we also created a customer journey map and a stakeholder map that we can use as a physical reference in the future:

Phase 2: Brainstorming

Highlight: An idea that I pitched in was a service similar to Facebook Live where students can use the power of social media as an on-demand feedback system. Students can give an impromptu speech and viewers can give real time presentation-related feedback such as “Speak louder” or “Talk Faster” encapsulated as feelings-like emojis.

For our brainstorming session, We were able to generate a total of eight service concepts, four of which are as follows:

In Class Video Recording. This service concept uses required in-class presentations to generate incentive and remove the need to dedicate time outside class hours to record a presentation video for feedback.
VR Practice Room. Since the logistics of gathering an audience to practice presentations is in itself a big challenge, this service concept allows students to practice presentations during their own time.
Feedback through Live Streaming. Like the VR practice room, this service concept focuses on letting students practice presentations during their own time but instead of presenting to an virtual audience, we wanted to leverage social media instead.
Automatic Feedback System. Instead of relying on humans to generate feedback, we explored utilizing emerging technologies such as speech recognition, machine learning and computer vision to gauge a presentation and generate usable feedback.

Phase 3: Needs Validation and Prototyping

Highlight: For the speed dating session, I conducted a speed dating session with the head of CMU’s Global Communication Center.

Step 1: Speed Dating

Utilizing a quick, low-fidelity needs validation technique, we were able to gather feedback for our design concepts:

Practicing on Own Time: Students can have really busy schedules. Our service should be flexible enough so that students can maximize their free time.
Laborious Expert Feedback: The significant time investment and huge disproportionality between students and GCC staff calls for more efficient ways of generating feedback.
Effective Simple Feedback: Despite their simplicity, feedback on one’s tone, pacing and filler word frequency can still be useful as there are still a lot of people who have problems with delivery, according to our language experts.
Peer Feedback Quality: Peers of engineering students are more than likely to be engineering students themselves and would be on the same level when it comes to communication skills.

Taking this new information into account, we refined our final service idea to incorporate the positive aspects from each initial service concept. The most significant aspect of the service that we changed was how we would generate and deliver feedback.

Automated feedback requires the least effort to generate and creates the most basic but useful form of feedback that aims on improving the student’s vocal qualities and body language.

Peer feedback involves having the audience answer a short and simple survey that helps the presenter get an idea on how well he did with his presentation.

Highlight: I was in charge of designing the survey for the user enactments

Instead of asking an open ended question that might possibly take a while to write out, the survey asked the audience to check all of the feelings that they would associate with the presenter’s delivery. The second question aims to give the presenter an idea of how effective he was with getting his main points across. The last question allowed the audience to bring up points that they would like the presenter to expand more on.

Expert feedback would also be an option for students who are still not satisfied with receiving the first two forms of feedback.

To address the problem of time and incentive, we decided to introduce two touch points for our service: a VR practice room who’s primary purpose is to serve as an on-demand venue for receiving automated feedback and an in-class component meant to be used by professors as a supplement for their existing course presentations.

Step 3: User Enactments

Using Wizard of Oz testing, we gathered a small group of students and had them present an impromptu speech in front of our Echo system prototype. The feedback we got from the students are as follows:

Perspective. Students needed some perspective about what success looks like. Although many could recognize a good presentation, students often have trouble understanding what exactly makes it good. Echo helps users parse and understand the factors that manage effective communication with an audience.
Layers. Students appreciated the “layered” nature of evaluation, especially the meticulous reports that allow them to quantify their performance and track it over longer periods of time as well as minute-by-minute. Additionally, Echo empowers students to draw upon two types of support: peer feedback that captures a real audience’s response and expert feedback for communication strategy and guidance.
Speed. Students also appreciated the automated nature of the service, which allowed them to identify and correct their behavior in real-time. Instant feedback promotes the development of good habits, enabling students to recognize patterns in their problem areas and nip them in the bud.

Final Service Proposal and Technical Feasibility

Highlight: Given my software engineering background, I was in charge of gathering information on just how feasible it is to implement our service.

Click on image to see full version

To solidify our service proposal, we conducted research on the technical feasibility of our feedback system.

Commercial products such as Ummo and LumoLift demonstrate how speech recognition and posture tracking exist. Additionally, software development toolkits like CMUSphinx for speech recognition and KinectSDK for body tracking can also be utilized to create the feedback system from scratch.

For complete details on our design process, refer to our final report which can be found here.