Engagement is a poor proxy for learning

We’ve probably all heard a colleague say something like, “I did a great activity today. It worked well. The kids were really engaged.” We even have professional development based on this premise: A consultant will come in to a school and promote a drama-based activity or project-based learning and everyone will conclude how effective it is because the students are really engaged.

I think the term ‘engagement’ has two meanings when people use it in this way. The first is that students are motivated by the activity and the second is that they are actively doing something. Perhaps the latter is seen to imply to the former because, in many classrooms, full participation in the activity might be optional.

Professor Robert Coe claims that engagement is a poor proxy for learning.

Source: Professor Robert Coe


It is important to understand what ‘poor proxy’ means. It doesn’t mean that engagement is undesirable or in conflict with learning. Rather, learning is invisible. It’s not something that we can directly observe. So if we want to conclude that learning has taken place then we look for a proxy. A good proxy might be something like a delayed test. If students score well then we can infer that learning has taken place. Coe is claiming that engagement is a poor proxy.

This is quite reasonable. We can imagine students busily and enthusiastically doing stuff but learning little; or at least learning little of what is intended. For instance, a Macbeth diorama could engage students for hours without improving their ability to analyse the play.

We can also imagine students staring pretty blankly while a teacher talks and yet learning loads. Certainly, the mind has to be active for learning to take place but this doesn’t mean that the body has to be physically doing something. Richard Mayer calls the conflation of the two the, ‘constructivist teaching fallacy‘.

So why has Coe’s perfectly sensible suggestion provoked so much argument on Twitter recently? 

Let’s take another of Coe’s poor proxies: classroom is ordered, calm, under control. I view an orderly classroom as highly desirable. I think that, in many cases, it is a necessary prerequisite for learning to occur. But I would not argue that we can infer from an orderly classroom that learning is taking place. Students could be behaving very well and learning little. Conversely, it is quite possible for students to be learning whilst some members of the class are misbehaving.

In my first year of teaching I had a difficult Year 10 science class for 80 minutes on a Friday afternoon. If we worked hard for the first 40 minutes I let the students research a science topic in the computer room for the last 40 minutes. They learnt little science from this research but many teachers dropped by and commented on how well behaved the students were.

I think that the problem people have with Coe’s analysis is that much of the evaluation of educational innovations never gets past the question of whether students are engaged. If this engagement leads to better learning then we could and should use better proxies to measure this learning. If a consultant comes in to your school and tries to sell you engagement then the professional response should be to ask for something more.

Advertisements

13 Comments on “Engagement is a poor proxy for learning”

  1. This post reminds me of an example that Daniel Willingham gives of a teacher that wants to teach about the slaves fleeing to the northern states. They often had biscuits for food with them. To engage the students the teacher had them bake biscuits. They learned little about the slaves but lots about baking biscuits.

    • Tara Houle says:

      Yes, that was an excellent illustration that Willingham gave. My daughter learned about the Ancient Civilizations of Egypt by making a glitter book (lots and lots of gold dust and glue) and then cutting out pretty pictures of Egyptian queens/princesses all over. She was very engaged and got an “A”.

      Too bad she didn’t learn about any historical facts though. Ah well. Her art skills are coming along splendidly though. Oh and this was when she was 13…

  2. David Jatt says:

    Is an “engaged class” more likely to be learning than a class where the students are disinterested and not paying attention?

    It’s not a good proxy but being engaged is a positive thing and will be an aide to learning or a step on the path to learning.

    There’s too much edubollocks point scoring going on. “But what does engaged really meeeeeeeeaaaaannnnnn,” Etc. I blame the internet. Every ten pence teacher turned wannabe academic’s got a f*&%ing opinion now,

    Sorry. Winds me up.

    Note: This post has been edited to remove a swear word

  3. A while ago I had a brief exchange on Twitter. I would like to elaborate on my thoughts on engagement….

    I agree that engagement is a poor proxy for learning. Additionally, learning is not engagement. Engagement can and is oversimplified. (It has taken me 4 years of reading and writing and I’m still confused). The best definitions recognise that engagement is a meta-construct, i.e., it contains multiple dimensions and it is context specific.

    A long time ago I went on an excursion with Year 10 students to the Gold Coast theme parks. They were totally engaged in the rides and sights. I would describe this as adolescent engagement in theme parks. From what I could observe and hear they enjoyed it, it was memorable, and they participated. From a teacher perspective (mine) they were also exceptionally well behaved.

    Onto schools and classrooms…. The first part is easy. The teacher perspective: Are the students present and more or less well behaved? But then the students’ subjective perspectives are crucial. Are they interested, do they see the work as relevant, do they like the work and school?

    Next, I return to the context. What is expected to be learnt? If I taught a concept or fact to Year 12s that has little life utility but was crucial for marks in an exam I wanted the students to be able to write it on an exam sheet two months later. But if I was teaching an important skill, eg. how to structure your work in a science lab report, then I would want the students to be able to transfer that skill elsewhere in school. To labour my point, if it was a crucial piece of knowledge eg. how to have safe sex I would be pleased to know that an adolescent was having safe sex 10, 20 years later (not literally pleased to know!). These are examples of context specific engagement and learning and the importance of what I suspect was Coe’s point about learning.

    Pulling it all together. I don’t think it is desirable to have any one part missing. It is possible to coerce a student into learning. It is possible to have behavioural adherence. It is possible to learn irrelevant content. Each of the teacher’s perspectives, students’ perspectives, and the learning context are crucial components.

    Finally, I am proud of some work I have published in this area in which I have more carefully structured my ideas around engagement: http://rer.sagepub.com/content/early/2016/09/16/0034654316669434.abstract

    • gregashman says:

      It is interesting that you have introduced the idea that content may be relevant or irrelevant. How do you draw this distinction?

      • Two part answer.
        1) Does the student think it is relevant to them?
        This is always subjective and is challenging in a classroom with diversity. I suspect this is one reason why engagement is problematic in adolescence. Adolescents are becoming autonomous, independent, etc.
        2) Does society think it is relevant to the student?
        This is the contested bit. My point (above) about safe sex is my favourite example. I think this is an essential skill or knowledge that a trained professional (teacher) should assist students to know, even if they don’t want to know it at the time. Some adults disagree with my view.
        Another example: In Australia some people would assert that christianity is an essential piece of knowledge that should be taught in schools. I think it is mostly irrelevant.
        Somehow policy makers and educators decide what is relevant to the student. We are never all going to agree on what is relevant. But I do think that at some level society (adults) should decide what is relevant.

        Curriculum is not my particular area of interest and knowledge but I think that currently adults decide too much. Students should have more say.

      • gregashman says:

        The danger with this approach to curriculum is that you potentially entrench elitism. Shakespeare plays, for instance, are unlikely to appeal much to adolescents until and unless they know quite a lot about them, and perhaps not even then. However, in order to access careers in the theatre then this kind of knowledge might be necessary. So what? Perhaps our students express no interest in working in the theatre and prefer to focus on other areas of work. This may be a valid argument but we should not then be surprised if occupations such as those at the top end of the arts are populated by individuals educated in elite institutions which insist on teaching Shakespeare regardless. And we also don’t know how many disadvantaged students, if required to interact with Shakespeare, might have developed an interest.

        E D Hirsch Jr makes a more practical argument about access to democratic discourse. In order to read a paper of record such as the New York Times you need quite a lot of general knowledge: writers of such papers necessarily assume some knowledge on the part of their readers otherwise the writing would be difficult and convoluted. Yet this expected knowledge – eg of the Israel Palestine conflict, world geography, that ‘The Kremlin’ means the Russian government – is not obvious and many children from impoverished backgrounds may never come across it. If we don’t teach this knowledge then we essentially prevent access to sources of political argument and disenfranchise already disadvantaged students. Hirsch makes such a case here:

        http://atlantaclassical.org/wp-content/uploads/2014/08/Reading-Comprehension-E.D.-Hirsch-article.pdf

  4. […] themselves more and more? On a separate note, as people such as Greg Ashman have pointed out, engagement is a poor proxy for learning, so reduced enthusiasm does not necessarily translate into reduced […]

  5. […] ever, Tytler’s argument for inquiry is made on the basis of engagement. Engagement is a poor proxy for learning and creating short-term situational engagement is no guarantee that students will develop a love of […]

  6. […] The above question was posted by Ken, which is a challenging question. PD is a requirement in education. We need to ensure that we remain up to date with emerging pedagogies and technologies, however, we need to revisit the way in which PD is run. There are some good examples of useful, relevant and practical PD, however, anecdotally, I know that there are also still a large number of schools delivering PD via lecture or PD that is fun and engaging, but that will not actually change practice. As Greg Ashman (@Greg_Ashman) often comments, engagement is a poor proxy for learning.  […]

  7. […] learning science? [Hint: student enjoyment, activity, collaboration, and technology use are poor proxies for learning.] Ask yourself why there are proficiency gaps when teachers and administrators are in agreement […]

  8. […] for the same reason that many progressive methods fail: students might engage enthusiastically (and engagement is not the same as learning) but they won’t learn very much because learning builds on prior learning and if students spend a […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s