App Reviews to Help Choose Ed Tech this Holiday Season
December 19, 2014
flickr/Sean Dreilinger
If you’re a parent or educator thinking of purchasing some apps this holiday season, you may decide to consult one of a growing number of lists and websites highlighting educational apps for young children. It makes sense to consult the reviews: With thousands of educational apps filtering into the market, it’s increasingly difficult to wade through and find high-quality resources.
But do you know who is behind those lists? Are you confident the reviewers have selected the best apps for children, let alone for a child’s learning?
To help, we’ve conducted interviews with founders and developers at some well-known review sites (listed below in alphabetical order) that live, eat, and breathe apps and other ed tech products. This is by no means a complete list of all the resources out there, but we wanted to get a sense of how reviewers choose which apps make the cut. Like the apps themselves, these review sites vary in audience, funding, and philosophy, but they all share the common goal of making the marketplace a little more manageable.
Karen Mahon, founder of Balefire Labs, spent more than 20 years in the ed tech field as an educational psychologist and trained instructional designer. As the market exploded, she said she couldn’t find an easy way to compare apps using science-based standards.
Balefire Labs was launched as a consumer report–style service to compare apps and see what each one contains. Each app is rated on 12 criteria stemming from peer-reviewed research on the effects of interactive features built into educational technology. For example, the service shows which apps include error remediation, adaptable difficulty, or clearly-stated learning objectives. Twenty percent of apps are checked by two reviewers to ensure they’re staying consistent on each criteria point.
This post is part of Seeding Reading, an series of articles and analysis by New America’s Ed Policy Program and the Joan Ganz Cooney Center at Sesame Workshop. See also the Learning Tech section of EdCentral.org and the JGCC blog.
It’s tough to get a high score on Balefire Labs. Of the approximately 3,300 reviews posted on the site, only 10 percent have received an A or B grade.
But Mahon stresses the letter grade isn’t the most beneficial part of the review. Rather, it’s the ability to compare on a giant chart which apps offer specific research-based learning tools like progress reports or mastery-based instruction.
The team uses a number of methods to find apps, including searching what’s new in the app stores and in Top 100 lists—although Mahon has found that an app’s being on the Top 100 list is “pretty much meaningless” in terms of instructional quality.
The site is funded by subscriptions starting at $3.99 per month, which Mahon said is a way to avoid taking money from developers for the bulk of reviews. However, developers can pay a $250 fee to have their product reviewed and to receive feedback reports on how to improve. The site is also funded by professional development for teachers and other consulting services.
“Sixty-five percent of all of our apps that we review are free. Only 35 percent are fee-based,” Mahon said. “That’s intentional on our part. We’re not trying to get [parents and teachers] to spend more money on apps. We’re trying to get them to spend more wisely.”
The ed tech field looked very different in 1993 when Warren Buckleitner started the Children’s Technology Review. But the site and service has adapted alongside the industry, writing more than 11,000 reviews along the way and publishing them in monthly issues. This year, it launched the CTREX database of all its reviews with searchable tags and filters.
A four-person in-house staff scores apps using a detailed rating criteria that’s standardized for consistency and occasionally cross-checked by reviewers. That consistency and cross-checking ensures that if you compare two very similar products, the one with a slightly higher rating is, in fact, going to be just slightly better.
All reviews are written from the viewpoint of a “picky teacher”—or a teacher who’s skeptical of public relations jargon and loves tools that foster active learning.
“I call myself a magic hunter,” Buckleitner said of the rare app that scores a home run. “I’m looking for good pedagogy and innovation. And I really like to see technology being harnessed for the benefit of teaching.” Buckleitner started his career as a elementary school teacher in Michigan and originally created Children’s Technology Review as a thesis project while pursuing his master’s degree in human development. He later earned his PhD in educational psychology from Michigan State University.
The site is funded by subscriptions to the service, which start at $8 per month. And although Buckleitner said it’s impossible for any service to be completely bias free, a subscription fee means the site has no financial ties with developers or publishers.
Buckleitner added a tip for anyone on the hunt for a go-to review site: Pick an app you’re already very familiar with and check how different sites review it to see the different approaches and emphases.
Common Sense Media’s Best Apps and Games
Common Sense Media, a national nonprofit known for its work in helping parents navigate the sticky digital landscape, has two sites that sort and highlight apps. One is Graphite, the classroom-focused tool we describe below. The other is the “Best Apps and Games” section of Common Sense Media’s website, which is designed primarily for parents. It presents multiple ways to browse or search for apps: by a child’s age, by type of device, by subject (math, science, etc) and by skill (such as problem-solving or creativity). The site also compiles “best of” lists, such as “Best Roadtrip Apps,” or “Best Puzzle Apps for Kids.”
In the same way that Common Sense Media rates movies and video games, apps are rated according to various dimensions. Reviewers are staff members at Common Sense Media, led by an editor with background in education. They review apps to determine whether the app is appropriate for certain ages. They use a red “off” icon to signal when something may be inappropriate for, say, a 5 year old, and a yellow “pause” button to signal that parents may want to be cautious. The site also uses a rating system to identify apps along a five-point learning scale, with a 5 signaling that an app is “really engaging” and has an “excellent learning approach,” to a 1 meaning that the app is “not recommended for learning.”
Parents and kids are also invited to review apps, and often the parent reviews spotlight dramatically different viewpoints, with one parent effusive in her love of an app and the next complaining that the app was boring her children and not worth the money. (The kids’ reviews can be comical, such as the one-star review for Elmo Potty Time from someone who gave it a red “off” icon for 17-year-olds.)
The site displays no ads. As a nonprofit organization, Common Sense Media’s funding comes from individual donors, as well as a variety of foundations and other organizations.
Carisa Kluver and Marc Kluver cofounded Digital Storytime in 2010 to help fellow parents and educators discover picture books for iPads and promote early literacy. Carisa, who reviews and rates the stories herself, has a background as a school counselor, health educator, and researcher in child and maternal health.
Each review covers the general plot, any necessary context about the author, and the reviewer’s general takeaways and impressions. But the site also displays how well an app measured up across nine rating categories. For example, are the e-book’s interactive features well integrated into the story? Or do they interfere with reading comprehension? How’s the audio quality? Is the book’s content appropriate and not too overstimulating for bedtime reading?
Other guest reviews, which include text but not a rating, come from other sites and blogs around the web that Digital Storytime reposts with permission. To date, there are over 970 reviews up on the site.
Digital Storytime is funded by ads that are all run through third parties like Google Ads to avoid conflicts of interest. Kluver never has direct contact with advertisers. Kluver also writes a blog called the Digital Media Dietthat covers digital media, kids, and technology.
Common Sense Media launched the free website Graphite last year. Jeff Knutson, senior editor of education reviews, said the two-part review system was designed to give teachers and parents “the best of both worlds.”
A product’s first “learning review” is written by a team of educators who are coordinated by the in-house editors at Graphite. The educators, many of whom are National Board Certified in a variety of fields, are trained on a set of standards that examine a product’s engagement, pedagogy, support for teachers, and feedback.
Hovering right beneath the learning review is the “teacher review,” a crowd-sourced review that averages feedback from teachers. The lively “field note” section details teachers’ firsthand experience using the product in classrooms.
In other words, if the “learning review” is like a restaurant review from food critics, the “teacher review” is like Yelp.
Knutson said Graphite takes a “whole picture” approach, including the nitty-gritty app details and tools to help teachers integrate the apps into the classroom. For example, its App Flows feature helps teachers visualize how a tool could fit into a lesson, whereas its Common Core Explorer helps teachers match reviews with specific standards.
“[All parts] of the site are intended to work together,” Knutson said. “It should really give teachers a wide idea about how to use these tools in the classroom.”
So far, Graphite holds approximately 1,700 educational app, website, and game reviews. Graphite is supported by the SCE Foundation and a personal investment from Bill Gates.
Another source of information is Moms with Apps. Founded by four mothers who were developing family-friendly apps and connected over social media in 2009, the site now features a robust community of developers committed to creating apps that protect kids’ privacy. Moms with Apps recently released the results of a survey of more than 400 parents about how they find apps for their children: while 96 percent of respondents report that their kids have benefited from using apps, nearly half (49 percent) said the process of finding good apps is “moderately” to “very hard.”
The Moms with Apps site allows the developer members to provide their own descriptions, and provides additional insight about what’s inside the app. The site enables parents to filter by age range, device, subject, and other app characteristics, including the ability to play without an Internet connection, whether there are in-app purchases, or ads. The “What’s Inside” sidebar that accompanies each description provides further information, such as whether the app collects information or connects to social networks.
The site also features developer profiles so parents can learn more about the people who are creating these apps.
In 1978, educator, mother and children’s book author Diana Huss Green published the first issue of Parents’ Choice. With the aim of guiding parents to books and toys that encourage a love of learning, the publication eventually attracted reviewers from a variety of fields.
Today, Green’s daughter Claire S. Green is the president of the foundation that’s now the country’s oldest non-profit guide for children’s media and toys. A core team of five, plus a number of reviewers with specialized experience, use the same guiding principles to rate educational technology products that were used to review Legos and building blocks.
Products that receive a Parent’s Choice Award go through a multi-tiered evaluation process. Companies submit their products along with a processing fee that varies depending on the type of product being reviewed (the fee is $250 for apps.) Then the Parent’s Choice committee considers how that product helps a child grow “socially, intellectually, emotionally, ethically, physically,” according to their website. The judges also weigh factors like cost, originality, and play value.
Only about 1 in 5 products submitted to the Parent’s Choice Awards receive recognition in any of the six award levels.
Educators Jayne Clare and Anne Rachel founded Teachers With Apps in 2010 after creating their own reading app, ABC Shakedown Plus, only to see it drowned out by apps they didn’t consider high quality. Today the blog is updated constantly, sometimes several times a day.
“One of the things that makes us unique is we field test all the apps with real students,” Clare, who now runs the site, said in a recent video interview with an app developer. She retired last June after teaching special education for the last 30 years across preschool to eighth grade. “We don’t just cut and paste from the iTunes store. We don’t just play a game ourselves and write a review. We feel that you don’t know the magic of an app until you put it in the hands of a child.”
Teachers With Apps’ reviews cover the general experience of using the app. The site also publishes reviews from a team that includes a speech pathologist, an occupational therapist, primary and secondary teachers, and a few college level professors who field-test apps and write the reviews.
The site is funded by ads that, Clare says, are only sold if they “respect the developer’s product.” And, if you buy an app using the link on the site, Teachers With Apps receives a small percentage of the sale.
“You’re only going to be on our site if you’re high quality and educational,” Clare said in the video interview. In other words, there are no poor ratings on the site—it only features products the team would recommend.
Kathleen Costanza is a writer and editor in Berkeley, CA. She covers education and digital media and learning for HiredPen Inc.
