The “Wrong Tool” Problem

One of the most common challenges that students face is applying the right concept or procedure at the right time. When solving a physics problem, for example, we have to apply the right equation or equations to solve the problem. There are several kinds of mistakes we might make:

1) Not knowing the key equations. We come to the problem with a set of ideas — tools — that might be helpful. Could be we just don’t have the right tool for the job. We needed hammer, but all we have is a screwdriver and a wrench.

2) Choosing the wrong equations. This is about using the wrong tool. We have a hammer, but for some reason we’re using a screwdriver to pound the nail in.

3) Not performing the procedure correctly. We’ve got the right tool for the job, but we make a mistake in applying this tool. We have a hammer and we need to pound in a nail, but we’re holding onto the head of the hammer and using the handle to hit the nail.

Learning science can help us address each of these kinds of mistakes. But this article focuses on number (2): using the wrong tool.

The opportunity to misapply categories, concepts, and procedures comes up in almost every discipline. In science and mathematics courses, students often need to apply the right kind of analysis for a given problem. When learning languages, students have to apply the right grammatical structure for the expression they want to make. In law, students have to apply the right legal framework to the current case.

The typical approach to teaching someone a new principle goes like this:

1) Explain the principle.
2) Provide an example.
3) Move on.

But this approach is insufficient for addressing the “wrong tool” problem. The reason is simple: from only one principle and one example the student has no idea when to apply the principle. How different can a problem be from the example to make it inappropriate to apply the principle? The student has no idea.

When we are novices at something, we pay attention to superficial features. If you are a go novice, the following image just looks like a bunch of random stones. To an experienced go player, however, the stones have meaning: there’s a fight in the lower left corner with multiple groups pincering each other, and it looks like the white stone on the right-hand side and the black stone in the top right corner are going to play roles in the outcome.

Plenty of research supports the idea that novices organize information in their heads according to superficial features and experts organize information according to deep, structural features. One classic study asked novice physics students and expert physics students to classify a number of problems. The novice students paid attention to the superficial situations presented in the problems: these problems have ramps in them; those problems have a rope in them; those other problems involve free-falling bodies. The expert students paid attention to the underlying ideas beyond the problems: these problems involve conservation of energy; those problems involve Newton’s third law; those other problems involve Newton’s first law. The structural understanding of the experts explains why they can solve problems much more effectively: they know their tool box well.

The novices, however, don’t really understand the relationship between tools and problems. Another study taught middle-schoolers permutations and combinations using, as examples, dice and cars. One group of students only saw dice permutation problems and car combination problems. The other group only saw car permutation problems and dice combination problems. The principles and procedures behind counting permutations and combinations, however, were taught to both groups. In spite of all students getting both 1) the principle and 2) a good example, they could not apply the principle effectively when tested. The first group tended to think, “oh, this is a problem about cars, that means it’s a permutation problem. I’ll apply the permutation procedure.” The second group tended to think, “oh, this is a problem about cars, that means it’s a combination problem. I’ll apply the combination procedure.” The problem was that students had associated the superficial features of the problem — the kind of object featured in the problem — with the underlying tool. But permutation problems ask a different kind of question than combination problems.

Two tools in our own toolbox of teaching strategies can help us solve the wrong tool problem: analogies and contrasting cases. Analogies are situations that seem superficially different, but share an underlying structure. A dice permutation problem and a car permutation problem may seem different: one’s about dice and the other one’s about cars. But they actually share an underlying structure: they’re both about permutations. This is true of even simple situations that we, as adults, take for granted. Three fish and three rocks seem different — after all, one involves living creatures and the other inert stones — but they share an important underlying property: three. Analogies show us how the same underlying property can show up in very different situations.

Contrasting cases are situations that seem superficially similar, but have an important difference. A dice permutation problem and a dice combination problem seem similar: they’re both about dice. But there’s an important difference between them: one involves permutations and the other involves combinations. Three fish and five fish seem more similar to each other than three fish and three rocks, but mathematically there’s an important difference: one involves a group of three and the other involves a group of five.

These two tools can be used together to teach students the variety of situations where they can apply a principle and the variety of common-looking situations where they cannot apply a principle. This helps shape the boundary of the concept for new students.

This leads to two rules of thumb when teaching new concepts: “Two examples are better than one.” And, “Non-examples can be as important as examples.”

Consider the following research on analogies:

Psychologists asked participants to address the following problem: a patient has a tumor that needs to be irradiated, but a beam strong enough to kill the tumor will kill healthy cells, too. On the other hand, a weaker beam won’t kill healthy cells, but also won’t kill the tumor. What should the doctor do?

To teach people how to solve this problem, the psychologists gave the learner either zero, one, or two analogies, and either told learners about the principle behind the solution or not. So among the conditions was the “one example and the principle” approach as well as a “two examples and no principle approach”.

The analogies were different superficially, but shared the same underlying solution structure. A general wants to attack a fortress, but the roads leading to the forest are laced with mines that will detonate if large numbers of troops walk over them. A fire is raging; there is plenty of water, but the villagers only have small buckets to put the fire. The solution to all of these examples is convergence: split the beam up into weaker beams that converge on the tumor. Split the forces up into small groups to converge on the fortress, and avoid triggering the mines. Give each villager a bucket of water and coordinate their efforts to pour lots of water on the fire all at once.

How did the participants do? Participants who saw an example and read the principle solved the radiation problem 32% of the time. Participants who saw two analogous examples but did not read the principle solved the radiation problem 52% of the time. Participants who saw two analogous examples and read the principle solved the radiation problem 62% of the time.

Knowing the principle helped a little bit. But seeing two examples helped a lot: students who only saw two examples did almost as well as students who saw two examples and the principle. The use of multiple examples is critical for learners to grasp the deep principle that all of the analogous examples share.

Now consider a study on the use of contrasting cases. A psychologist had students learn new letters from a foreign alphabet under three conditions. In the first condition, the students took 12 minutes to rewrite the new letters. In the second condition, the students took 10 minutes to examine important differences between contrasting sets of letters. In the third condition, students didn’t study the letters at all. Every student then learned the name for each letter. Students in the second condition learned the names for letters significantly faster than both the control and rewriting conditions. Why? Because the letters were already distinct in this group’s mind.

Contrasting cases prevent confusion; studying the differences between similar-looking things helps learners draw clearer boundaries between those things.

A common approach when students aren’t solving problems correctly is to give them more practice. Have them solve more problems! But if they’re applying the wrong tool, just having them practice more problems might not help because they’re spending a lot of their effort repeating a procedure (that they may already know). They can get far more practice in the key skill by just making the decision about which tool to apply. They don’t have to solve the problem, just match the problem with the right concept.

So: the next time you go about teaching a new concept, procedure, or principle, use analogies and contrasting cases to help students avoid the “wrong tool” problem.

Notes:

The contrasting letters research comes from Gibson, E. J. (1969). Principles of perceptual learning and development, and is discussed in Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and instruction, 16(4), 475–5223, available here.