-
Dr Pete Markiewicz Portfolio
Dr Pete Markiewicz Portfolio
The client, Elaine Nichols, ran a special private school for immigrant kids, often from low-income backgrounds. Since learning English is required for jobs, the client was interested in creating a self-learning smartphone app that taught ESL (English as a Second Language). In addition, the client wanted to add gamelike qualities to the app’s teaching, and exploit Immersive media – virtual and augmented reality – in the user interface.
In fall 2018, I was asked to design and develop an application prototype. The goal of the prototype was testing the general concept for ESL learning, along with creating a good simulation of virtual reality in a UX prototype.
Initial Research was conducted into existing language apps, especially those targeted at children and tweens, rather than adults.
A series of visits with the school's students allowed construction of typical users for the app. Based on these discussions, I developed a list of user types for the application. Users were defined by demographics, likes, and dislikes. Since the app had strong gamification, user attributes like "type of battle preferred" during a test, challenge or student duel were defined along with regular UX attributes.
These ESL students are tech-competent, and typically have smartphones capable of playing 3D games. They learn best when they are outside the classroom, e.g. on a boring bus ride. The app is the primary method they are using to improve their language skills.
These younger ESL students are just starting school, and not very competent with technology. They attend a mix of on-ground ESL teaching, whith the app acting as a supplement.
Parents are assumed to be busy, and have low-end Android phones that won't display good graphics. They need an SMS version of the app, employing "conversational design" to monitor and communicate with their kids and the Instructor
Teachers are assumed to have good smartphone and desktop access, and communicate with both students and teachers.
In the first stage of research, I did a study of various learning styles. The goal was determining what kinds of things engage, and actually teach, for casual access. We envisioned ESL students, often on slow bus trips, going to the app to pass the time. The app had to enable casual learning. After analyzing the literature on casual learning, along with problems with current language apps, we came to the following product specifications:
Competitive analysis of language app learning models.The leftmost screen shows Stage 1 style learning (memorization). The middle screen shows less common Stage 2 learning (context). The rightmost screen shows Stage 3 learning challenges, which are currently found in only a few language apps at present (communication)
Research and User Models resulted in our choosing a "virtual world" model for the app, as opposed to the "flashcard" model common in other apps. Instead of memorization, learning a language let students manipulate an avatar in the virtual world. Similar to Pokemon, the avatar could be entered in challenges with other learners, and its success in battle depended on the student's understanding of the language.
Since the app's virtual world had to function on both high-end and low-end devices, we implemented a "Responsive Virtual Reality" using Progressive Enhancement, a model developed by Arturo Paracuellos of Unboring.net (his example below):
(Source: unboring.net. Click image to link to original article)
We modified his Progressive Enhancement model as follows; VR, "magic window" smartphone VR, or an SMS conversation between a user and the app system or app avatars.
After defining user stories, defining interaction began. Since we were building a prototype, interaction focused on onboarding.
I used Conversational Design to define interaction in low resolution. Writing interaction as a screenplay between users and game avatars had ensured that many virtual world interactions could be accomplished by "texting" an avatar or student in the virtual world.
I also created a conversational diagram for a challenge between two students
Medium resolution design consisted of simple wireflows, mostly defining the user task completed on each screen during onboarding.
High resolution was a standard Interaction Design Flowchart, incorporating conditional branches within onboarding for all users. Each step in the interaction was matched by number to the conversational design model created earlier.
The client felt that paper prototypes would not work for the target student and parental audiences. So, we needed to create an app prototype that simulated Virtual Reality, as well as SMS communication and voice control. To create the app, we used ProtoPie (http://protopie.io). Unlike most other UX prototyping tools, ProtoPie supports smartphone sensors including tilt/movement, camera, SMS, compass, and voice.
Simple Avatars and scene UI elements were created for the grey-box wireframes.
A series of onboarding screens were created. ProtoPie allowed us to make the screens actually work. For example, users actually had to type into the registration and login fields, and ProtoPie validated their input. Controls were touch-driven and animated. We were able to make "grey-box" wireframe screens of the app, and make elements onscreen interactive. The app could also simulate VR in "magic window" format. Student testers could move their smartphones around to explore the virtual world, interact with avatars, and manipulate objects.
Interactive prototype screens for onboarding. Left: select language, Center: select user type, Right: select Avatar.
Interactive prototype screens for onboarding. The prototype challenges students with basic (Level 1) questions in order to help them learn the interface. Questions are functional and scored by the prototype.
Interactive prototype screens related to the virtual world. The left screen is a functional microphone that allows students to test voice control of virtual world elements. The center screen allows them to choose the virtual world landscape. The right screen is a leaderboard showing challenges and other students.
Interactive prototype screens for advanced questions. To the left, a draggable element allows students to choose endings for worlds. In the center, students must create a sentence telling their avatar how to plant a crop. On the right, phrase choice to answer questions.
Interactive prototype screen for SMS communication. Users can address their avatar within the virtual world, or by SMS. Parents can check up on student activity via SMS, and teachers can send messages to students.
Live tests of the avatar were conducted with the primary "tween" student audience, ESL students aged 9-14. Usability tests were conducted for the onboarding strategy, and students were asked to give their subjective evaluation of the app's value to them.
Students also completed surveys which helped to quantify their experience with the app prototype. Questions were designed to see whether testers were able to recognize specific user interface (UI) elements.
Based on the results, I constructed a "pitch deck" for the client to show to investors along with the interactive app prototype.
(Click above to access the pitch deck)
The resulting app prototype is unique in re-creating a virtual world/game VR environment without hardcore coding. The app's use of smartphone sensors allowed testers to explore the virtual world created, and interact with it. In contrast to most UX prototypes, the logins and questions actually worked! This was essential, since the testers (9-15 years) were reluctant to comment on static wireframes or comps.