Engagement is a poor proxy for learning

We’ve probably all heard a colleague say something like, “I did a great activity today. It worked well. The kids were really engaged.” We even have professional development based on this premise: A consultant will come in to a school and promote a drama-based activity or project-based learning and everyone will conclude how effective it is because the students are really engaged.

I think the term ‘engagement’ has two meanings when people use it in this way. The first is that students are motivated by the activity and the second is that they are actively doing something. Perhaps the latter is seen to imply to the former because, in many classrooms, full participation in the activity might be optional.

Professor Robert Coe claims that engagement is a poor proxy for learning.

Source: Professor Robert Coe

It is important to understand what ‘poor proxy’ means. It doesn’t mean that engagement is undesirable or in conflict with learning. Rather, learning is invisible. It’s not something that we can directly observe. So if we want to conclude that learning has taken place then we look for a proxy. A good proxy might be something like a delayed test. If students score well then we can infer that learning has taken place. Coe is claiming that engagement is a poor proxy.

This is quite reasonable. We can imagine students busily and enthusiastically doing stuff but learning little; or at least learning little of what is intended. For instance, a Macbeth diorama could engage students for hours without improving their ability to analyse the play.

We can also imagine students staring pretty blankly while a teacher talks and yet learning loads. Certainly, the mind has to be active for learning to take place but this doesn’t mean that the body has to be physically doing something. Richard Mayer calls the conflation of the two the, ‘constructivist teaching fallacy‘.

So why has Coe’s perfectly sensible suggestion provoked so much argument on Twitter recently? 

Let’s take another of Coe’s poor proxies: classroom is ordered, calm, under control. I view an orderly classroom as highly desirable. I think that, in many cases, it is a necessary prerequisite for learning to occur. But I would not argue that we can infer from an orderly classroom that learning is taking place. Students could be behaving very well and learning little. Conversely, it is quite possible for students to be learning whilst some members of the class are misbehaving.

In my first year of teaching I had a difficult Year 10 science class for 80 minutes on a Friday afternoon. If we worked hard for the first 40 minutes I let the students research a science topic in the computer room for the last 40 minutes. They learnt little science from this research but many teachers dropped by and commented on how well behaved the students were.

I think that the problem people have with Coe’s analysis is that much of the evaluation of educational innovations never gets past the question of whether students are engaged. If this engagement leads to better learning then we could and should use better proxies to measure this learning. If a consultant comes in to your school and tries to sell you engagement then the professional response should be to ask for something more.

Advertisements

21 thoughts on “Engagement is a poor proxy for learning

  1. This post reminds me of an example that Daniel Willingham gives of a teacher that wants to teach about the slaves fleeing to the northern states. They often had biscuits for food with them. To engage the students the teacher had them bake biscuits. They learned little about the slaves but lots about baking biscuits.

    1. Yes, that was an excellent illustration that Willingham gave. My daughter learned about the Ancient Civilizations of Egypt by making a glitter book (lots and lots of gold dust and glue) and then cutting out pretty pictures of Egyptian queens/princesses all over. She was very engaged and got an “A”.

      Too bad she didn’t learn about any historical facts though. Ah well. Her art skills are coming along splendidly though. Oh and this was when she was 13…

  2. Is an “engaged class” more likely to be learning than a class where the students are disinterested and not paying attention?

    It’s not a good proxy but being engaged is a positive thing and will be an aide to learning or a step on the path to learning.

    There’s too much edubollocks point scoring going on. “But what does engaged really meeeeeeeeaaaaannnnnn,” Etc. I blame the internet. Every ten pence teacher turned wannabe academic’s got a f*&%ing opinion now,

    Sorry. Winds me up.

    Note: This post has been edited to remove a swear word

  3. A while ago I had a brief exchange on Twitter. I would like to elaborate on my thoughts on engagement….

    I agree that engagement is a poor proxy for learning. Additionally, learning is not engagement. Engagement can and is oversimplified. (It has taken me 4 years of reading and writing and I’m still confused). The best definitions recognise that engagement is a meta-construct, i.e., it contains multiple dimensions and it is context specific.

    A long time ago I went on an excursion with Year 10 students to the Gold Coast theme parks. They were totally engaged in the rides and sights. I would describe this as adolescent engagement in theme parks. From what I could observe and hear they enjoyed it, it was memorable, and they participated. From a teacher perspective (mine) they were also exceptionally well behaved.

    Onto schools and classrooms…. The first part is easy. The teacher perspective: Are the students present and more or less well behaved? But then the students’ subjective perspectives are crucial. Are they interested, do they see the work as relevant, do they like the work and school?

    Next, I return to the context. What is expected to be learnt? If I taught a concept or fact to Year 12s that has little life utility but was crucial for marks in an exam I wanted the students to be able to write it on an exam sheet two months later. But if I was teaching an important skill, eg. how to structure your work in a science lab report, then I would want the students to be able to transfer that skill elsewhere in school. To labour my point, if it was a crucial piece of knowledge eg. how to have safe sex I would be pleased to know that an adolescent was having safe sex 10, 20 years later (not literally pleased to know!). These are examples of context specific engagement and learning and the importance of what I suspect was Coe’s point about learning.

    Pulling it all together. I don’t think it is desirable to have any one part missing. It is possible to coerce a student into learning. It is possible to have behavioural adherence. It is possible to learn irrelevant content. Each of the teacher’s perspectives, students’ perspectives, and the learning context are crucial components.

    Finally, I am proud of some work I have published in this area in which I have more carefully structured my ideas around engagement: http://rer.sagepub.com/content/early/2016/09/16/0034654316669434.abstract

      1. Two part answer.
        1) Does the student think it is relevant to them?
        This is always subjective and is challenging in a classroom with diversity. I suspect this is one reason why engagement is problematic in adolescence. Adolescents are becoming autonomous, independent, etc.
        2) Does society think it is relevant to the student?
        This is the contested bit. My point (above) about safe sex is my favourite example. I think this is an essential skill or knowledge that a trained professional (teacher) should assist students to know, even if they don’t want to know it at the time. Some adults disagree with my view.
        Another example: In Australia some people would assert that christianity is an essential piece of knowledge that should be taught in schools. I think it is mostly irrelevant.
        Somehow policy makers and educators decide what is relevant to the student. We are never all going to agree on what is relevant. But I do think that at some level society (adults) should decide what is relevant.

        Curriculum is not my particular area of interest and knowledge but I think that currently adults decide too much. Students should have more say.

      2. The danger with this approach to curriculum is that you potentially entrench elitism. Shakespeare plays, for instance, are unlikely to appeal much to adolescents until and unless they know quite a lot about them, and perhaps not even then. However, in order to access careers in the theatre then this kind of knowledge might be necessary. So what? Perhaps our students express no interest in working in the theatre and prefer to focus on other areas of work. This may be a valid argument but we should not then be surprised if occupations such as those at the top end of the arts are populated by individuals educated in elite institutions which insist on teaching Shakespeare regardless. And we also don’t know how many disadvantaged students, if required to interact with Shakespeare, might have developed an interest.

        E D Hirsch Jr makes a more practical argument about access to democratic discourse. In order to read a paper of record such as the New York Times you need quite a lot of general knowledge: writers of such papers necessarily assume some knowledge on the part of their readers otherwise the writing would be difficult and convoluted. Yet this expected knowledge – eg of the Israel Palestine conflict, world geography, that ‘The Kremlin’ means the Russian government – is not obvious and many children from impoverished backgrounds may never come across it. If we don’t teach this knowledge then we essentially prevent access to sources of political argument and disenfranchise already disadvantaged students. Hirsch makes such a case here:

        http://atlantaclassical.org/wp-content/uploads/2014/08/Reading-Comprehension-E.D.-Hirsch-article.pdf

  4. Agree entirely. Project learning, which is supposed to engage actually disadvantages those that it is supposed to help – the disadvantaged. Currently in UK there is an argument about making all children study an academic curriculum when some of them do not have the capability to understand it. This is particularly fraught with early years education where, though teachers believe reading and writing to be important, they believe narrowing the curriculum to prioritize these things disadvantages children – but early years teachers seem unaware of the disadvantage of being a poor reader by secondary level: a situation that narrows the curriculum far more than it ever is in an early years classroom dedicated to teaching SSP, And the children in this position are most often those from disadvantaged homes, who were supposed to be helped by spending less time on phonics and maths in early years so they could enjoy a wide curriculum.

    1. Pretty dumb statement – and then people who don’t think use the statement against those who consider intrinsic motivation and engagement essential for meaningful and purposeful learning and also for the well being of children. When I hear the word engagement , I hear that in addition to meaningful learning , the kid finds the learning engaging and it is interest, curiosity , his questions that are the source of his engagement . The teacher is providing the conditions for intrinsic motivation to flourish and is supporting this motivation with good teaching and resources. A more accurate and not misleading statement would be – Engagement is not always …….. Prof Coe then says ” A good proxy might be something like a delayed test. If students score well then we can infer that learning has taken place” . Well if the only type of classroom learning is the traditional type this could be true , but the statement implies that teachers can’t assess learning while she is interacting and facilitating learning and the only exhibition of a child’s learning is a test , that kid’s questions, the dynamic back and forward between kids , the demonstrations of sophisticated thinking , the passion , the way learning touvhes their heart is irrelevant because it can’t be reduced to a number. We need to remember the words of Rice University Profess Linda McNiel that test scores tell us the least that matters about a child’s education. Our concern for engagement comes from a recognition that that we are not dealing with robots and teaching outomes go far beyond test scores

      1. But children may not be engaged by, for example Shakespeare, but of they do not learn about his work they never will be engaged by it, and many careers will be closed to them. That teaching outcomes go beyond test scores may be a truism but if engagement and adventurous questions have not lead to being able to read fluently (phonics) and with understanding (good and increasing knowledge base) you will not be able to engage in anything.

  5. Is it not the teacher’s job to create interest and promote engagement that will drive meaningful learning. We may be able to teach kids to read , but helping them to be inclined to read is a different story. Nobody is saying that phonics is a bad thing , just that it needs to done in a context of story so we promote understanding together with decoding , and understanding helps us to decode as well.What we have in education and we find this even on an undergraduate level is that students are only motivated to learn to meet extrinsic performance goals rather than seeing value or being intrinsically motivated. The only difference between high and low achievers is that high achievers see the value in education in getting a good job and making money and know they have ability and low achievers don’t. I am not worried that a progressive educator won’t teach a kid to read . What worries me is the lack of long term retention of knowledge and skills ( I studied for the test , got a 100 , then forgot everything after the test ), real understanding of ideas along with critical thinking , creativity, the capacity to apply skills to different types of tasks and other sophisticated intellectual outcomes or students’ interest in what they are doing and the liklihood that they will come away with a continued motivation to learn that characterises Old school education. Old school sees no value in adventurous questions and engagemnt because they ignore intellectual skills https://deyproject.files.wordpress.com/2015/04/dey-lively-minds-4-8-15.pdf

  6. I am pretty concerned if children don’t learn to read since most knowledge and most employment is closed to them. As to critically thinking about a subject or becoming interested in it, these are dependent on knowledge. As to forgetting facts learned for an exam, they will be embedded in long term knowledge, allowing understanding when new information related to that knowledge comes along.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.