The Wicked Effectiveness of Retrieval Practice

This is part 3 in a series on what we know about how we learn and how this knowledge should inform how we teach. The series is intended for teachers, students, and developers of education technology who want to be more informed about their practice. Part I and Part 2.

Studying is not the same thing as taking a test, is it? Studying is when we learn something. Taking a test just evaluates whether we’ve learned that something.

The research on retrieval practice turns this logic on its head. What we tend to think of as “studying” — books out, highlighters in hand, re-reading that tricky paragraph for the fifth time — is not nearly as effective for long-term learning as taking repeated tests on what we know. Tests don’t just evaluate learning — they accelerate it.

This is why a lot of educational psychologists are in favor of more testing (not less) in schools. They’re not talking about the kinds of high-stakes, standardized tests present in every state and nearly every country in the world. They’re talking about low-stakes tests designed to facilitate learning.

What it is.

Retrieval practice is all about trying to remember what you’ve read, seen, heard, or otherwise experienced before. It’s also often referred to as “the testing effect” because this kind of practice usually comes in the form of tests.

Retrieval practice can take several different forms.

Cued retrieval is like a flash card. The learner gets a hint or a word or something to prompt her to remember what she learned before.

Free recall is “free” from hints and prompts. Here’s a blank piece of paper: remember everything you can.

Elaborative retrieval involves something more than just vomiting out information or giving the right answer to a prompt. Maybe you organize what you know into a concept map or an outline. Maybe you build a timeline or some other kind of organizational framework.

Nearly every test is also an example of retrieval practice, regardless of format: true/false, multiple-choice, short answer, open-ended responses.

What unites all of these forms is they involve the process of trying to remember information. It may just be about trying to remember the information (e.g., cued retrieval, free recall of a word list or passage), or it could be about remembering and organizing the information (e.g., elaborative retrieval of connected concepts), or it could be about remembering and applying the information (e.g., a test that asks students to solve physics problems).

How it works.

The brain takes in a breathtaking amount of information over the course of a lifetime. We don’t need to remember all of it in every situation. So our brains provide differential access to different memories. I’m anthropomorphizing the brain here, but it’s almost as if, by trying to remember something, the brain says, “Oh, you need access to this information. I’ll make it easier for you to access later.”

Actually, access is the wrong word. Remembering modifies memories; it doesn’t just access them. Sometimes we tend to think of brains like computers. When a computer “remembers” something it’s accessing memory. The memory is there (barring some kind of corruption) and the computer finds that information and pulls it up. The hundreth time that you ask a computer to access information is the same as the first time.

Our brains don’t work like that. For us, remembering something changes the memory. One aspect of this change is that our memories for events can be frightfully wrong, even if they feel so real and true. Another aspect of this change is that remembering something now can make the it more memorable in the future. Which is essentially what’s going on with retrieval practice.

The effect of retrieval practice is also an example of two larger principles that come up over and over again in the learning literature: 1) actively thinking about ideas is better than passively accepting them and 2) practice what you want to get better at.

Actively thinking about ideas is better than passively accepting them. Or “cognitive effort increases learning.”

If we return to our “memory as network” metaphor (the previous article), one way that retrieval practice seems to help is that it strengthens the relationships between the “right” pieces of information. With cued retrieval, for example, the link between the cue and the target memory is strengthened.

The other thing that seems to be happening is that retrieval reduces the interference of other memories on the target memory; in some sense “tightening” and “organizing” the network. So you’re trying to remember the word “bird”, but you also saw a bunch of other related words: nest, feather, beak, etc. Retrieval practice prevents you from building links to unrelated words (a distinct advantage over elaboration as a memory technique).

This idea also comes into play when thinking about when retrieval works well and when it works poorly. Suppose I give you a list of words to remember. And then ten minutes later I ask you to tell me all of the words on that list that you can. Your memory for the last words on the list (e.g., the last words you studied) will be pretty good. They’re still fresh in your mind.

But then I ask you to take a delayed test three days later. Now, your memory for the last words you saw when you first studied will be worse. In other words: for immediate tests, your memory of the last words on the list will be better than the earlier words; in delayed tests, your memory of the last words on the list will be worse than the earlier words.

Why? Remember: tests are learning events. When you took that immediate test, which words were the hardest to remember? The early words on the list. Which words were the easiest to remember? The words at the end of the list.

So which words are you going to remember better later? The early words. Your brain worked harder to remember the early words on the immediate test — it was effortful and active. So you remembered more of those on the delayed test.

A final theme on this principle comes from the effectiveness of different forms of retrieval practice. Recall (“Tell me what you remember.”) beats recognition (“Have you seen this word before?”). Recall is harder than recognition. Short answer test questions beat (simple) multiple-choice questions. Questions that ask students to apply information beat questions that ask students to simply regurgitate information. The pattern is the same: in all these cases, cognitive effort pays off.

Practice what you want to get better at.

This principle is a little deceptive. Of course you should practice what you want to get better at. But people often think they’re practicing one thing when, in fact, they’re practicing something else. Students study to perform well on tests. But when students “study,” they often re-read. So much so that “studying” is essentially synonymous with re-reading in the psychological literature. But re-reading is a completely different activity than taking a test. Taking a test involves remembering relevant information and applying it correctly to a specific question. Re-reading doesn’t necessarily require that you remember information or apply it correctly; you just see the information again.

As with many of these principles, there are some exceptions to this general rule. But if you want to get better at taking the test, one of the best ways is… to practice taking the test. If you want to recall information, practice recalling information.

This idea applies to different forms of retrieval practice as well. Cued retrieval practice involves providing students with a cue for remembering. You say, “espinaca,” I say “spinach.” You say, “Newton’s second law,” I say “F = ma.” Cued retrieval is like giving students a hint. And this kind of practice makes people better at giving the right answer when provided a hint.

But free recall works a different way. You say, “sit down with a blank piece of paper and tell me the meaning for every Spanish word that you can remember.” Free recall is harder. Without a specific cue, I might not remember that espinaca means spinach. This kind of retrieval gives students a practice at recalling information without a hint, which is usually more realistic. In most situations, no one is giving us the one crucial hint we need to recall the right information.

How we know it works.

Research on this idea stretches way back into the early 20th century, which explored memory for word lists and passages. But modern interest in the idea was revived in the early nineties by a pair of researchers who pinned down that it was really the retrieval part of taking tests that was improving memory. Since then a wave of experimental work and classroom implementation have established the power of the effect.

The basic research approach asks one group of students to “study” (in other words: “re-read”) a previously studied word-list or passage and another group of students to take tests that ask them to recall what they read before. Researchers have performed all kinds of variations within this basic model. You can give students many repeated tests at different intervals. You can give students different kinds of tests. You can give students feedback on their performance or not.

The upshot of all of this research: retrieval practice just works. It works with word lists, passages, and more complex information. It’s not particularly sensitive to the study design or the context or the kind of thing you’re learning. Study after study after study just says, “yep, it works this way, too.”

Surprisingly, tests can enhance memory for information learned after the test. So if you’re tested on the four word lists you’re trying to remember you tend to perform better on a fifth word list than if you just re-studied those initial four word lists. This “forward effect” of testing is a reasonably well-replicated finding in the lab, but it’s less well studied than the effectiveness of retrieval practice on what you learned before.

What about classroom studies? In the past ten years or so, several researchers have tested the testing effect in the classroom. One of the most extensive research projects took course over 5 years and involved 1,400 middle school students. The upshot: it works there, too. Students who took delayed quizzes on material performed about a letter grade higher than their peers. These frequent tests also reduced test anxiety.

How effective is retrieval practice compared to other learning techniques, like elaboration? Elaboration is helpful. But retrieval practice is ridiculously helpful. Head-to-head competitions between the two show that retrieval practice is simply more effective on delayed tests (immediate tests are often deceptive measures of long-term learning).

How much more effective depends upon the forms of elaboration and retrieval that are compared. But just to give you some sense of the numbers, at least from one study: if students using elaboration got 60% correct on average, students using free recall got 75% on average. If students using elaboration got 50% correct on average, students using free recall got nearly 70%. (For those of you concerned with variation, we are talking about effects on the order of one standard deviation — a pretty massive, consistent effect). Again — this is compared to a learning strategy that we already know is more effective than typical “re-reading” study practice.

How to implement it.

There are so many forms of retrieval practice that implementing it well can be tricky.

Flashcards are a form of cued retrieval practice, but they have to be used in a specific way to be effective. Suppose you’re learning a language. On the front of the card, you put the foreign language word; on the back, the English word. You always study the cards front to back. That’s a problem. You’re reinforcing only one direction — from the foreign language word to the English word. But you probably want practice going the other way as well (from the English to the foreign language word). This kind of cued practice can be helpful in remembering the large body of vocabulary necessary for fluency. But it can’t replace practice at creating sentences and integrating words in the absence of these specific cues. It can’t replace conversational practice.

Simple, short quizzes are an easy way of incorporating some retrieval practice in the classroom. But the implementation matters. Suppose you gave a ten-question quiz once a week over the previous week’s material over the course of the semester. And you returned each quiz a few days after your students took it. This is good, but it could be improved.

The first problem is that students are only retrieving information once and on a predictable (and relatively short) schedule. A better way is to incorporate a mixture of questions from previous lessons on the quiz. You might have one question that refers to material a couple of months ago. Another question that deals with material you just went over yesterday. And yet other questions from times in between. Now, students are going to do worse on this kind of quiz than on a quiz that only covers material from the week before. But it’s better for long-term learning. Which is the main point of doing a quiz like this in the first place. These quizzes are meant to help students learn the material; they aren’t intended for students to ace.

The second problem is that students aren’t getting feedback about their answers in a timely fashion. We’ll talk more about feedback later. But in this case, it would probably help to go over the answers briefly right after the quiz. If students made mistakes, not correcting these mistakes can lead them to remember the mistaken information over the correct information.

The surprise of a pop quiz is not particularly beneficial, especially if you’ve attached any significant grade to it (generally speaking, heightening anxiety is not great for learning). You can tell them a quiz is coming. But knowing exactly what’s going to be on the quiz can diminish the effectiveness of the practice. Suppose you tell them exactly what’s going to be on the quiz. Then the students study right before class to ace the quiz. This is almost the same as giving them an immediate test after learning the material.

“High-level” questions — questions that ask students to apply, analyze, or evaluate information — seem to be better than “low-level” questions — questions that merely ask students to remember a fact. In one study, students who took “high-level” tests outperformed students who took “low-level” tests for all kinds of questions (both “high-level” and “low-level”).

Another effective approach would be to use a free recall exercise. Put a blank sheet of paper in front of you and write down everything you can remember about a specific topic. This is approximately the approach taken in several of the more rigorous tests of retrieval practice. A side benefit is that you’ll get a much better sense of what you know, which can make later study more effective.

It can be even more effective to incorporate other learning mechanics. Rather than simply listing what you know, why not try creating a concept-map, timeline, or other visualization? This can help organize what you know.

A final consideration is spacing: how long should you wait before testing yourself (or your students) after initially studying the material? One of the keys to making retrieval effective is to delay the first test long enough for it to be “cleared” from your short-term memory. Immediate tests aren’t nearly as helpful. After that, how long to wait between subsequent rounds of retrieval practice seems to depend on a number of different factors (but when the retrieval is “easy,” it’s probably too soon). Research suggests, however, that we should be thinking more in weeks and months rather than in hours and days.

If you’re a teacher and you want to figure out ways to incorporate retrieval practice into your classroom, I recommend checking out www.retrievalpractice.org.