The client, Elaine Nichols, ran a special private school for immigrant kids, often from low-income backgrounds. Since learning English is required for jobs, the client was interested in creating a self-learning smartphone app that taught ESL (English as a Second Language). In addition, the client wanted to add gamelike qualities to the app’s teaching, and exploit Immersive media – virtual and augmented reality – in the user interface.
In fall 2018, I was asked to design and develop an application prototype. The goal of the prototype was testing the general concept for ESL learning, along with creating a good simulation of virtual reality in a UX prototype.
Initial Research was conducted into existing language apps, especially those targeted at children and tweens, rather than adults.
A series of visits with the school's students allowed construction of typical users for the app. Based on these discussions, I developed a list of user types for the application. Users were defined by demographics, likes, and dislikes. Since the app had strong gamification, user attributes like "type of battle preferred" during a test, challenge or student duel were defined along with regular UX attributes.
These ESL students are tech-competent, and typically have smartphones capable of playing 3D games. They learn best when they are outside the classroom, e.g. on a boring bus ride. The app is the primary method they are using to improve their language skills.
These younger ESL students are just starting school, and not very competent with technology. They attend a mix of on-ground ESL teaching, whith the app acting as a supplement.
Parents are assumed to be busy, and have low-end Android phones that won't display good graphics. They need an SMS version of the app, employing "conversational design" to monitor and communicate with their kids and the Instructor
Teachers are assumed to have good smartphone and desktop access, and communicate with both students and teachers.
In the first stage of research, I did a study of various learning styles. The goal was determining what kinds of things engage, and actually teach, for casual access. We envisioned ESL students, often on slow bus trips, going to the app to pass the time. The app had to enable casual learning. After analyzing the literature on casual learning, along with problems with current language apps, we came to the following product specifications:
Research and User Models resulted in our choosing a "virtual world" model for the app, as opposed to the "flashcard" model common in other apps. Instead of memorization, learning a language let students manipulate an avatar in the virtual world. Similar to Pokemon, the avatar could be entered in challenges with other learners, and its success in battle depended on the student's understanding of the language.
Since the app's virtual world had to function on both high-end and low-end devices, we implemented a "Responsive Virtual Reality" using Progressive Enhancement, a model developed by Arturo Paracuellos of Unboring.net (his example below):
(Source: unboring.net. Click image to link to original article)
We modified his Progressive Enhancement model as follows; VR, "magic window" smartphone VR, or an SMS conversation between a user and the app system or app avatars.
After defining user stories, defining interaction began. Since we were building a prototype, interaction focused on onboarding.
I used Conversational Design to define interaction in low resolution. Writing interaction as a screenplay between users and game avatars had ensured that many virtual world interactions could be accomplished by "texting" an avatar or student in the virtual world.
I also created a conversational diagram for a challenge between two students
Medium resolution design consisted of simple wireflows, mostly defining the user task completed on each screen during onboarding.
High resolution was a standard Interaction Design Flowchart, incorporating conditional branches within onboarding for all users. Each step in the interaction was matched by number to the conversational design model created earlier.
The client felt that paper prototypes would not work for the target student and parental audiences. So, we needed to create an app prototype that simulated Virtual Reality, as well as SMS communication and voice control. To create the app, we used ProtoPie (http://protopie.io). Unlike most other UX prototyping tools, ProtoPie supports smartphone sensors including tilt/movement, camera, SMS, compass, and voice.
Simple Avatars and scene UI elements were created for the grey-box wireframes.
A series of onboarding screens were created. ProtoPie allowed us to make the screens actually work. For example, users actually had to type into the registration and login fields, and ProtoPie validated their input. Controls were touch-driven and animated. We were able to make "grey-box" wireframe screens of the app, and make elements onscreen interactive. The app could also simulate VR in "magic window" format. Student testers could move their smartphones around to explore the virtual world, interact with avatars, and manipulate objects.
Live tests of the avatar were conducted with the primary "tween" student audience, ESL students aged 9-14. Usability tests were conducted for the onboarding strategy, and students were asked to give their subjective evaluation of the app's value to them.