Kids Learning Math & Learning from Kids: Lessons from User Tests

Today we bring you the first in a series of STEM related blog posts — starting with last year’s Cooney Center Prize finalists for the Motion Math — Gabriel Adauto and Jacob Klein.

Throughout the process of creating Motion Math, our bouncing star fraction game for the iPhone and iPad, we’ve greatly benefited from conversations with our primary users: kids. The game was just released, and you can buy it here. We were honored this past June to be a finalist for the Cooney Center’s inaugural Innovations in Mobile Learning Prize, and we wanted to share five key take-aways from dozens of rounds of user-testing with kids aged 6-14 in homes and schools.

1. Kids Grasp Technology
One danger with creating educational games is that we all have some common sense about how to teach; this intuition can blind us to the ways that kids (and especially today’s digital natives) are different. To take one striking example, many young kids now expect all screens to be touchscreens; we’ve heard from many parents that their young children now pinch and tap the TV!

Adults are more articulate about what they want in an interface, but often struggle in ways that kids never do; we’ve learned to be wary of adding features that adults ask for.  A great example is text instructions – kids get the symbolic hints whereas adults prefer explicit language.

Also, kids are fantastic bug finders! If you want your game to be played upside down, sideways, and in ways you never intended, give it to a child. We encourage this creative testing by asking students to find bugs. In the rare 😉 cases that they do, we reward them with a collection of hand-picked bug stickers.  It’s charming to watch them wear around a sparkly bug sticker on their shirts as a badge of their technical prowess.

2. The Deeper You Look, the More Complex the Subject Matter
We decided to tackle fractions in our first game: we knew it was notoriously difficult for many learners, but didn’t comprehend its complexity before talking to kids. The fact that 1/2, 50%, 0.5, and 3/6 all mean the same thing is odd; at the same time “1/2” is very different from “1/2 of 2.”  Fractions can mean a distance on a number line, selecting parts of a group, parts of a whole, breaking something into parts, and many other physical metaphors. Competence in one metaphor doesn’t necessarily mean understanding (or even awareness) of another. When first exposed to fractions, kids don’t necessarily come home and start converting measurements in the kitchen. Games are unique device to show patterns among concepts because the game environment can be constructed in a way that surfaces these relationships. As we start on new math topics, we’ll assume there are more wrinkles in the idea than we expect; the challenge is to surface this complexity and demonstrate patterns while preventing beginners from being overwhelmed.

3. Show, Don’t Instruct (And Iterate!)
The part of our game that has undergone the most profound change is the introduction.  Even when testing our original paper prototype, we realized our unique game interaction would require some explaining: our game doesn’t rely on the more common interaction of tapping, but rather tilting.  In addition to novel gameplay, we also introduce a new form of educational content: placing a bouncing fraction-star on a number line. It’s potentially overwhelming.

Initially, we wrote carefully-crafted instructions: “Move the fraction star to the right place on the number line by tilting.” We added italics on the last word to really drive home the game mechanic, and we were pretty proud of ourselves. One day of classroom testing proved we had perhaps the most useless sentence ever written. Most kids didn’t read the instructions; even ones that did forgot them as soon as they moved to the next screen. Words failed. After many attempts and revisions, we settled on the current version: we introduce tilting by putting the player on an empty screen with nothing but a bouncing star and targets to land on. Some adults get confused, but 95% of kids immediately explore the device and figure it out.  Only after the game interaction has been mastered do we introduce the idea of a star holding a fraction.  We find some validation for this approach from education pioneer Maria Montessori, whose “auto-educating apparatuses” were designed to allow children to individually explore concepts deeply by leveraging their senses. “This makes it possible for the child to work by himself, and to accomplish a genuine sensory auto-education, in the visual perception of form,” she wrote. Our users are now able to understand the application controls without verbal clarifications.

4. Multiply the Motivation
One second grader struggled to make it past the first level, and we expected her to stop playing. Instead, she bragged to her friend: “I got 6 stars!” and immediately played again. Minor bits of motivation can maintain engagement, so we kept adding different forms of feedback: progress stars for each correctly solved problem, a cumulative score, new level indicators, bonus effects, and a game-over indication of overall progress – presented as a fraction, naturally. Different levels of positive feedback motivate learners at different skill levels. At the same time, we didn’t want to create a game that overly congratulated our players or made victory too easy: that’s why we’ve made the last few of our 24 levels quite difficult.

5. Doubt Your Own Awesomeness
After working for many months, our natural bias was to love our game, and it was easy to go into user-testing inclined to hear affirmation.  When we go into classrooms and the teacher announces that we are app developers testing a new game, the kids get very excited and put themselves on best behavior. (The Hawthorne effect is the psychology term that describes this phenomenon.) In general, we must remain skeptical about our customers’ reactions to the game despite their praise and enthusiasm.  More telling are the subtle indications of their liking, such as picking to play our game again rather than other iPhone games on the same screen. When a kid presses “Play Again” without asking for permission, they are showing real engagement. They have found the game easy enough to begin, but they also believe that they could improve over their last performance, and when they play again over and over they are in a state of flow: continuous engagement. This is an ideal state for maximal learning and enjoyment.

Just hanging out with children reminds us of our important role in maintaining their excitement about learning. With focused observation and questioning, and letting go of our own biases, user testing can provide the feedback and inspiration to create great learning products.

Gabriel Adauto and Jacob Klein are co-founders of Motion Math. Their first game, a bouncing star fractions game for the iPhone, iPod Touch, and iPad, can be purchased here.

More to Explore:

Motion Math recently won an Editor’s Choice Award for Excellence in Design from Children’s Technology Review!