Concept maps are rubbish

For many years, I have given the following advice to students who are preparing for an exam.

“There are two ways to revise; answering questions or turning information from one form into another. Both mean that you have to process information in some way. On the other hand, reading your notes or just copying them out can be done without really thinking and so these aren’t very effective.”

I don’t know what I based this advice upon, other than my own intuition. I was quite late to education research, as I suspect many teachers have been. So I was particularly interested when in 2013, Dunlosky et. al. published their findings about the most effective study strategies. Testing was in there – and that was the main message that seems to have come out from the various articles and blogs that have been written on the back of this paper (as an aside, it is also worth mentioning the recent debate as to whether testing works equally well for complex items as it does for simpler ones).

Rereading and similar strategies, such as highlighting, were found to be ineffective. However, the report is more ambiguous when it comes to my advice about, “turning information from one form into another.” By this, I meant making revision notes, flashcards or posters and Dunlosky et. al. found that ‘summarising’ had low utility because many students don’t do it very well.

One information processing strategy that is particularly popular amongst educational researchers and some teachers, and that is not directly addressed in the Dunlosky et. al. study, is the notion of constructing ‘concept maps’.

Concept Map for Energy produced by NASA - Public Domain

Concept Map for Electricity produced by NASA – Public Domain

I speculate that concept maps are popular because they apparently mirror the ‘schema’ that psychological theories propose for the way that information is organised by the mind. You might expect that constructing a concept map of an area of study would require you to rehearse the different relationships between the concepts and so remember and retrieve that information better.

Which all sets the scene for a fascinating piece of research conducted in 2011 by Jeffrey “Testing Effect” Karpicke and Janell Blunt. They compared testing-based study strategies with concept map construction. In all of their experiments, testing was superior. This was true even when the final test – the “post-test” which we analyse to determine the effect of the experiment – required students to construct a concept map. So concept mapping is not even the best preparation for concept mapping.

This is quite a clever experimental variation because there is a general principle in education research that students tend to learn what you teach them. So, with a cunning experimental design, you can bias the results. If you read much research you start to see it everywhere. For example, imagine you were comparing an inquiry learning approach to gravity, where students perform experiments with balls on ramps, make hypotheses and so on with an explicit approach that teaches them about gravity. If your post-test focuses on hypothesising, results tables and so on then you’ll get a different outcome than if your post-test contains physics questions.

So the fact that Karpicke and Blunt found that testing beat concept maps under conditions most suited to concept maps is significant. However, one objection was raised; the students in the study had little experience in concept mapping and so perhaps they were not doing it very effectively.

Fortunately, we now have a 2015 replication of the Karpicke and Blunt study which also deals with the experience issue. Lechuga et. al. conducted their study in Spain and essentially found the same as Karpicke and Blunt. Crucially, they also assessed students who had prior experience with concept mapping. Retrieval was still more effective than concept mapping for this group, although by a smaller margin.

This illustrates how educational research should work. We have a replication and we have variations to the experimental design to take account of different explanations and hypotheses.

On this basis it is clear that investing time and resources in concept mapping is the wrong choice. It may be better to advise your students to test themselves and answer trial questions instead.

Advertisements

15 Comments on “Concept maps are rubbish”

  1. Reblogged this on Century 21 Teaching and commented:
    A very interesting read.

  2. This is an interesting article, but does it really tell us anything useful? Any teacher worth their salt surely uses a range of delivery and embedding methods during a curriculum lifespan, including different methods of preparing for and using assessment as a key learning tool.

    Please keep these research commentaries coming, but let’s also remember that much of the research only serves to present interesting angles on what we already know and use effectively.

    • stan says:

      I suggest Nick read Super Forecasters. While the title may seem unrelated the history of medical science and the resistance by doctors who knew what they were doing is very relevant.

      It also covers the psychology of over confidence. You know, we are all better than average drivers. It’s abnormal not to have it when it comes to what we care about most and we are drawn to those that have more than most. The book provides a great study in how to combat it.

      Anyone who says we already know what is optimum and do that should ask themselves how they know and how they update that reliably. With limited time a shotgun approach of using a range of methods is not obviously a good one.

    • Josh Fisher says:

      This is a reason why education research should be made more freely accessible: researchers often have to answer the “usefulness” question before they publish (or before they even run an experiment/get funded) and they speak directly to the question *in the paper*, while providing references to related work addressing related questions.

  3. julietgreen says:

    I’m glad the findings have now reached a more general audience. When I was carrying out my Master’s research, prior to this publication, I came across Roediger’s work (e.g. http://ow.ly/U4z5) in this field, but few would accept my view on the positive effects of testing at a time when assessment gurus were fairly united in their condemnation. The thing is, this is nothing new. Anyone who went through public exams in the first 7 decades of the 20th Century, would have very much used the effect of testing and self-testing as a major strategy in terms of learning and recall of information. I think we’ve sought more ‘user-friendly’ and possibly lazy methods; they may be less arduous or challenging, but they don’t work as well.

  4. dodiscimus says:

    My instinct about useful revision techniques mirrored yours and I would originally have agreed that because concet maps involved reorganising information they were good revision tools. However, I came to the conclusion that they were limited because students (I’m mainly referring to A-Level here) could only do one of two things. Either they worked from memory, in which case they only included what they could easily remember; or they worked from notes/textbooks, in which case they only did a small amount of thinking about structure and a lot of copying. I think there might be some merit in getting students to fill in blanks in a concept map (e.g. a concept map of themes in a play with illustrative quotes to be added, or a physics concept map with all the formulae missing etc), and I think there may also be merit in giving students the concepts and asking them to do the mapping (not as revision but as a way of helping them to appreciate the deeper structure within part of a subject), but my impression is that actually creating the concept map from scratch is only valuable if understanding is already nearer the expert than the novice end of the spectrum. I think it’s quite a good thing for teachers to do for students as a knowledge organiser for the beginning of a topic for example, not only for students to have but also as a way for the teacher to think carefully about how to organise the teaching and learning. However, with those provisos I agree that concept mapping is not very useful and the papers you reference seem to support this.

  5. If I’ve understood their method correctly, what Karpicke & Blunt found was that *repeated* retrieval was better at helping students recall information than *a single episode* of concept mapping. That’s not surprising given that it’s repeated recall that reinforces memory.

    Their experiments don’t show that investing time and resources in concept mapping is the wrong choice. Concept mapping and recall serve two different functions. Concept mapping makes explicit the way students organise their knowledge about a particular topic and can be a useful way of identifying and correcting misunderstandings Recall reinforces their memory of particular facts. Ideally you need both.

    • stan says:

      Logical
      The study participants did spend equal time in both cases and those using retrieval practice did better at producing a concept map in the test.

      So it seems for these students (undergrads) retrieval practice was the best use of their time even when it comes to being able to produce an accurate concept map.

      So while you are correct that testing for the students ability to produce a concept map will provide information about mistaken views about the concepts, spending time on concept maps is not the optimal way for them to practice for such a test.

      Keep in mind the context of Greg’s title here is exam prep.

      • Yes the conditions were time-matched but not matched for frequency of rehearsal – which is what reinforces memory.

        Karpicke & Blunt’s main point is that the advantage retrieval practice has over concept mapping for recall is the opposite of what people intuitively expect. But they also point out that concept mapping resulted in higher initial performance and that students could use concept mapping plus retrieval practice.

        Interesting findings but I don’t think they quite justify the title of the post.

  6. Greg,

    I wouldn’t go as far as saying “rubbish”, rather that only in certain cases it might have a function (it’s now seen as an all-purpose strategy). First off, it depends on whether the concept map is made while reading or after reading, in which case it could function as a testing event. More important might be what the examination is. A concept map can represent hierarchies or heterarchies. If that is what is to be learnt- in other words there’s a proper constructive alignment – and the student has to first try to create a hierarchy from memory and may then use the materials to complete it to fill in the gaps, then it could be similar to “Read-Test-Reread-Test”.

    paul

    • gregashman says:

      Hi Paul – these are good points and worth bearing in mind. I am interested in the idea that they might support the learning of hierarchies and so on. The title was meant to be a provocation. 😉

  7. […] This article below was originally posted on my old blog. It is relevant to my recent discussions of critical thinking and concept maps: […]

  8. Ryant says:

    The thing is Greg the evidence doesn’t actually support your conclusion. You quote one Karpicke study about concept maps but overlook a follow up where he found that concept maps were valuable tools if used for retrieval practice. The point being that it is how you use them that makes them effective or not. I’ve sent you a link to the follow up study on twitter but easy to find.

    • Here is a direct link to the Karpicke study showing that concept maps are effective tools if used for retrieval practice. http://learninglab.psych.purdue.edu/downloads/2014_Blunt_Karpicke_JEDP.pdf

      And here is the relevant section from the conclusion:
      “It is important to note that the results show that concept mapping can indeed serve
      as an effective task when it is implemented as a retrieval-based learning activity. The key element for promoting meaningful learning was not the format of the activity; it was the requirement to engage in active retrieval practice during learning.”

      The evidence does suggest that concept maps (if used for retrieval practice) are a valuable tool in toolbox.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s