Animated vs. Static Learning Agents – My M.Ed. Capstone Research

You never know what’s going to happen when you go to a DevLearn session with Cammy Bean, folks. In her excellent session on Avoiding the Trap of Clicky-Clicky Bling-Bling, someone asked about the value of onscreen characters and Cammy mentioned the research I did for my M.Ed. capstone project this year… which reminded me that I still haven’t shared it here!

So, for regular readership and those who might have wandered here because of that session, here’s the super-short version:

I created two versions of a course with only a slight difference in multimedia, then invited participants to take the course, tracking behind the scenes which participants were given which version. I reported on the differences in:

immediate understanding of the content (level 2 for those of you who roll that way)
self-reported engagement measures (level 1)
LMS-reported engagement measures (whether the learners finished the course, how much they explored the optional areas)

And what aspect of multimedia did I test? I specifically focused on the use of a fully animated learning agent* — one that moves her lips and gestures with her hands — versus a static (image-only) one. And I found no statistically significant difference in positive outcomes. Zero. Zip. Nada. Bupkus.

So basically, if you’re thinking about sinking a lot of development time or money into developing fully animated learning agents in hopes of increasing engagement or retention, I think this is research that you will want to read. Some vendors of those tools distribute research that others have done, supporting the use of learning agents in some situations, but nowhere in my background research or in this project did I find evidence that animation — lip synchronization, hand gestures, body movements — increases positive outcomes.

You can read the full report here, and I would love to receive your feedback, comments, and questions below.

* A learning agent, also called an animated pedagogical agent (and sometimes dubiously called an avatar), is a character that guides the learner through the course, somewhat like a teacher, although learning agents can be interactive or merely multimedia.

http://twitter.com/ljwp Laura Payette

Saw you present at DevLearn — nicely done! I wondered why so many vendors there seemed to be pushing their new animated characters. It just doesn’t seem relevant to me for most learning applications. Would love to look at your full report when I get a chance. Thanks for posting!

http://onehundredfortywords.com Judy Unrein

Thanks for stopping by, Laura! I would love to get your feedback.

I’ve noticed this trend as well, and so it’s an issue I’ll be posting about more in coming months. Stay tuned!

http://twitter.com/megbertapelle Meg Bertapelle

I’m totally not surprized – I would think that it would actually be more distracting, but did you find the outcomes were the same?

http://onehundredfortywords.com Judy Unrein

Across the board, the objective results were very close, but there’s no doubt that there were individual participants who reacted very positively to the animated LA and then some that reacted quite negatively. One of the results that I found particularly instructive was a conversation with one of the learners a few days after she took the course. She was 19 — solidly in the demographic that we’re often told we have to impress with multimedia — and she was completely unable to recall which version she had taken. (And I checked afterward… she had the animated version.)

Now, you could definitely take that to mean that this particular multimedia that I created was unimpressive. I can absolutely concede that… but it was also created using some of the most expensive software that is marketed to our industry. If even that software, used fairly skillfully, can’t create good enough multimedia to make the training memorable, I would humbly submit that our time and money is better spent designing and developing more meaningful kinds of engagement and interactivity.

Anonymous

A while back I saw a question on LinkedIn about whether animated or photo avatars were better. Your research doesn’t directly address that, but it does make me feel hard pressed to justify doing more than something static. Thank you for sharing this. I definitely agree with your conclusion that there are tons of other variables that could be researched in this area.

http://onehundredfortywords.com Judy Unrein

Absolutely. The original intention was to have seven versions of the course, actually, including video, still images, animations, and illustrations… each with human speech vs. text-to-speech (except no video + text-to-speech, because that would just be stupid). But… that just wasn’t feasible. I’d love to do it someday, though.

I did stake out a web presence for elearningresearch.org, where people can take the native files from this study and reuse them, and while for the time being, I’m not really going to have time to put a lot of work into that project, it’s there if someone wants to do it.