Experts and the elements of understanding

This is one for the cognitive science nerds. A paper has just been released pre-publication by Ouhao Chen, Slava Kalyuga, John Sweller*. It attempts to unify the well-known ‘expertise reversal effect’ with the ‘element interactivity effect’. I’ll try to put this succinctly but I’ll surely miss some important nuances in the process.

Imagine we are solving a problem and that different parts of the problem rely upon each other. A typical example would be rearranging an equation: We can’t do the steps in isolation because the second step depends upon the first and so on. Similarly, construction of a paragraph will draw upon different elements that each depend upon the other. Contrast this with learning a list of labels. The example Chen et. al. use is learning the Periodic Table in chemistry. It would be a complicated task to complete but learning the symbol for fluorine is not dependent upon learning the symbol for iron so the element interactivity is low (no pun intended).

Van Gog and Sweller have previously used this idea to explain experiments which seem to show that retrieval practice works for low element interactivity learning but studying worked examples is better when interactivity is high. The new paper uses the same idea to reconcile the seemingly contradictory ‘generation’ and ‘worked example’ effects and draws on Chen’s experiments to bring this into focus: For novices, generation works better for low element interactivity and worked examples work better for high element interactivity and for more knowledgeable learners, generation was best all round.

Element interactivity is not straightforward to define. Chen et. al. themselves suggest that processes such as making inferences to construct mental representations, integrating them with prior knowledge, or blocking irrelevant information are hard to define in terms of interacting elements and it is worth pointing out that Van Gog and Sweller’s findings are contentious and a matter of some debate.

If we accept the notion of element interactivity – setting aside concerns that it might sometimes be hard to identify the elements – then it should change as we become more expert at a task. Once we have completed a certain kind of process often enough, that routine becomes like the subroutine of a computer. We can pull it out of long term memory as and when required. We don’t have to deal with all of the interacting elements in working memory and it consumes fewer working memory resources. Element interactivity effectively reduces with increased expertise. This could account for the expertise reversal effect where experts learn more by solving problems than by studying worked examples – the opposite of what we see for novices.

Those who follow the rhetoric of mathematics education reform will also be interested to find that Chen et. al. have a go at defining ‘understanding’:

“Element interactivity also can be used to define “understanding.” Information will be fully understood if all interactive elements can be processed in working memory simultaneously (Sweller et al. 2011). Nevertheless, the term understanding tends to not be used when dealing with low element interactive information. If someone deals with information low in element interactivity, such as “Cu” stands for “copper”, we would not refer to them understanding or failing to understand the relation. If we fail to recall this relation, we will attribute the failure to forgetting or having no prior knowledge rather than failing to understand. Therefore, understanding is only used for materials high in element interactivity.

The distinction between learning by understanding and learning by rote is also related to element interactivity. Learning by understanding increases the number of interactive elements that must be processed in working memory simultaneously. However, if a large number of interactive elements cannot be handled simultaneously, learning by rote reduces the number of interacting elements albeit at the expense of understanding. Of course, learning by understanding is the ultimate goal of instruction.”

Some of this is similar to ideas that I have tried to articulate before about ‘subjective understanding’ – we feel that we understand something when we can manipulate the elements in our working memory. I am a little bit uncomfortable with the second paragraph because I see no reason why we cannot both memorise something and also see how it fits into the bigger picture or ‘understand’ it. So I don’t see that memorisation is necessarily at the expense of understanding. I can memorise a multiplication fact such as 7 x 8 = 56 whilst also understanding why this is the case and how I could demonstrate it to be true should the need arise.

I think that all educators need a basic theory of expertise given that this is what we are aiming at for our students and the idea of element interactivity could add something here. I might also take this opportunity to shamelessly plug my new book (which you can buy here) which addresses the issue of expertise and the misconceptions that we have about it.

*Disclosure: Kalyuga and Sweller are my PhD supervisors

Advertisements

The bad idea that holds teachers back

We have all probably heard a common trope along the lines that, ‘I don’t teach content, I teach children.’ Another such example popped-up in my timeline via @Borto74:

students curriculum

It is an odd idea to emphasise. I would expect that any teacher who is doing the job that he or she is paid for will be teaching students the curriculum. So why does this come up?

It is to do with the cult of the individual that permeates our schools. Even supposedly sober commentators such as Ben Jensen and Jacqueline Magee perpetuate this idea. Whilst noting the profound difficulties required in attempting to meet every student’s individual needs, they do nothing to dispel the notion that this is the way to go.

Differentiating work is difficult and time-consuming. It can also potentially lead to invidious outcomes as students who are identified as struggling get watered-down content, increasing the achievement gap. It is not something that higher performing states seem to do. The graph below might come as a bit of a shock to anyone who thinks that differentiation is proven to be best practice:

PISA vs TALIS graph

I write about differentiation at some length in my new ebook. The evidence just isn’t there to support the idea. It seems truthy enough that catering to individual needs will be better for students but it ignores the realities of the classroom; a teacher cannot simultaneously individually instruct 30 students. Even when researchers have tried to make it work, they have complained that the teachers weren’t doing it right. So it is either something that works if you have particularly talented teachers who can implement it – although this has not been demonstrated – or it is an idea that doesn’t work at all. You decide.

What is clear is that it is not an approach that is grounded in solid evidence.


Ouroboros

Edit 25th April 2016: Ouroboros is now available on Amazon.

I recently launched my new ebook, “Ouroboros”. Since then, I have been pleasantly surprised by the reaction. I have received positive comments on Twitter from David Didau, Mike Beverley and John Walker. David was also kind enough to write a positive review which you can read here and that includes the following:

“Although I consider myself more than a little familiar with Greg’s thinking, I still found plenty of fresh thought and experienced a couple of ‘A-ha!’ moments as concepts I’ve been mulling over for some time settled neatly into place. The point being, however much you think you know, you’ll probably learn something from this book. And if you’re a new-comer to the Ashman oeuvre then you’re in for a treat. Best of all, it costs less than a fiver!”

It is a short book at around 30,000 words. That just happened to be how it turned out. You can therefore probably read it in one concerted go. It includes my thinking on educational trends and some possible solutions for moving teaching forward.

I made a conscious decision to not publish the book via Amazon. This is a punk project and I wanted to see what I could do myself. I guess this restricts the potential audience but I’m after quality rather than quantity. My research suggested that Amazon take away a lot of control from authors. By publishing my book as a pdf, the reading experience is better; you can print it out or copy and paste from the file. I will review my position on this if there’s enough demand for a Kindle version.

You can purchase the book by clicking the “Add to Cart” button below and checking out via PayPal. It costs $15.00 AUD which, at the time of writing, is just over $11.50 U.S. dollars and just over £8.00 British pounds.

Add to Cart

You will then be redirected to a page where you can download the pdf. You will also get an email with a link to this page. The pdf comes stamped with your email address and transaction ID to discourage unauthorised redistribution. The distributor is a site called e-junkie – ask me about it if you’re interested in doing something similar yourself.

I hope your enjoy the book. Let me know what you think!

I also had a query from a teacher educator about using the book on a course. Feel free to put it on your reading list. However, it’s also probably a good idea have an institutional version that may be distributed through libraries. I have decided to charge $120.00 AUD for this so it makes sense for you if there are more than 12 people at your institution who might read it. If this is of interest then please contact me via the form below:


ouroboros


The secret to a strong department

I started my career as teacher in North-West London in 1997. I began work in a science department where every topic at each year level had a lesson-by-lesson scheme of work. These were variable. At best, a scheme would give detailed guidance on what to teach, what activities to use and so on. At worst, there would be a lesson name, an associated practical activity and nothing else. As a new teacher, I found the former far more helpful than the latter. Faced with a blank page, I was overloaded. ‘What do the experienced teachers do here?’ I wondered. In these circumstances, I sought advice; a time-consuming and inefficient process of catching someone in a corridor or office and getting them to tell me something that could easily have been written down. I was quite capable of planning a lesson myself but I knew that it might not be as good or might not achieve quite the right objectives.

Since then, I have had the opportunity to work with a number of different departments. The best ones have planning documents that include a high degree of lesson-by-lesson detail. The worst ones give  teachers a list of five vague themes to cover in a term and let them get on with it with very little guidance. In my experience, this is not something that divides along teacher-led versus child-centred lines. The schemes of work in my first science department were distinctly constructivist in their approach whereas other well-planned departments have used more explicit approaches.

This experience has informed my own practice. At the start of 2015, I began teaching a VCE subject for the first time. I was teaching alongside a colleague who had a long and successful track record with the course. I wanted to capture and document exactly what he did in as much detail as possible because I wanted to do the same. Why wouldn’t I want to do that?

I used to think that departments that did not plan in a systematic way were simply disorganised; a bit of a shambles. This may be the case. However, I now realise that teachers have developed all sorts of rationales (or rationalisations) to explain this. These come under two themes that are usually linked in any discussion.

Firstly, teachers claim that they need a certain amount of autonomy. This is related to flawed notions of what it means to be a professional. It is often claimed that this autonomy is required in order to respond to the particular needs of the children in any given class; needs that may be different from the students in other classes. Joint planning is characterised as marching students through a curriculum that may not be suitable for their current stages of development.

I also suspect that many teachers like to do what they like to do and herein lies the danger: Imagine I am not comfortable with a key part of the curriculum and I have enough autonomy to de-emphasise it or even miss it out. Things might not get taught.

Joint plans actually have many advantages. For instance, they help to manage workload. If there are four teachers in a teaching team, and they have nothing to start with, then each teacher needs only plan a quarter of the lessons. This is a big gain on planning all of them. The other teachers may then focus on improving these lessons or writing quality lessons of their own rather than recreating each lesson, on their own, late at night, from scratch.

Children are not as different from each other as is sometimes supposed. A really good explanation of chemical equations is a really good one for pretty much all students. We could say the same about a great passage to read on the battle of Bosworth or a way of practising the construction of complex sentences. Teachers may vary the pace or give additional guidance – this is not an argument for fully scripted lessons – but on exactly what basis does a teacher decide to ignore or change something that has been jointly planned? What evidence would he collect and how would he know that his alternative is better? If it really is better then it is likely to be better for all the students in the cohort and so this activity should replace the activity in the joint plan.

The autonomy argument is a castle built on sand. The cost is everyone spending loads of time planning their own idiosyncratic lessons from scratch. The gain is unclear.

However, it is important to note that joint planning needs to be effectively led. You cannot rely on teachers passing resources to each other on the basis of goodwill. Inevitably, someone will end up doing more than their fair share, resentment will build and this may eventually lead to people keeping their stuff to themselves. Strong, joint planning is therefore a sign of a strong department. It is the sort of thing that senior leaders should be looking for instead of bowling into lessons demanding card-sorts, group-work and ‘engagement’.

This is why I was heartened to read a young maths teacher explain the advantage of joint planning. Freed from scrabbling to create last-minute worksheets, she can focus instead on the craft of the classroom; on building productive relationships:

“After a term, I was amazed by how much maths knowledge the kids had retained. It, of course, came down to how hard the kids have worked, but it also came down to consistently good teaching. Yet this was only possible because I had not planned anything. My focus has been on developing my subject knowledge and teaching the kids, not writing three-page lesson plans, or making resources that would have been of sub-standard quality anyway because I do not have the expert knowledge of my HoD.”

I cannot think of a better way to induct a new teacher into the profession. It is hard to develop expertise when you have to focus on everything at once.

Does your department require you to constantly reinvent the wheel or does it provide robust joint planning so that you can benefit from the wisdom of your colleagues?

Planning


Is teaching a profession?

I think that most teachers assume that being a professional means that we should have a certain level of autonomy. We close the classroom door and make our own decisions about how best to serve the needs of the students in front of us. Yet I don’t think that this is how other professions necessarily operate. If I went to the doctor with a nasty rash then I would expect to get the same standard treatment, regardless of who that doctor is. I would expect a lawyer to use a standard approach to applying for a court order or an engineer to use standard principles in order to design a bridge.

Perhaps this is what sets teaching apart. Perhaps it is simply different to other professions in this way. After all, attempts at standardising teaching practices can be terrible. Think of Ofsted, the English schools inspectorate, patrolling the country enforcing group work and limiting teacher talk. We don’t want this. And the standard of evidence in education is different even to that in medicine. It is harder to be certain that a particular approach is the right one.

However, there are a couple of problems with this argument. Firstly, are we comfortable that a child in Mrs Black’s class could get a very different experience to a child in Mr Brown’s class, even if it’s the classroom next door? At least on some dimensions, one of the classrooms is likely to be better. Is this fair? And if we argue that teachers should make their own decisions then does this not imply a kind of Darwinian logic to policymakers?: Teachers may choose their methods and then we may reward the ones who get the best results and punish or demote the ones that don’t.

Let’s conduct a thought experiment. Siegfried Engelmann’s Direct Instruction programs use scripted lessons. I don’t claim that this is true but let’s just imagine that there was overwhelming evidence that students in such programs outperformed their peers who were taught using different methods and let’s assume that there are no negative consequences, such as a loss in motivation or a lowering of self-esteem. Would we all start to use these programs? I think that the answer is a resounding, “No.”

The reason for my confidence is that we have the same situation already in the form of systematic synthetic phonics. The evidence is compelling that this is the best approach to teaching early reading (see here, here and here) and yet the model that is still promoted is a ‘mixed’ or ‘balanced’ approach that de-emphasises the importance of phonics and that may include the use of multiple cues, a strategy that is potentially harmful. Misconceptions about phonics abound e.g. that it is about single letter-to-sound correspondences or that English is mostly non-decodable.

Oddly, the fact that teaching practice is so vaguely defined seems to support a whole industry built around telling teachers what to do, from education schools to consultants to purveyors of edtech and so on. The recent attempt at creating a “College of Teaching” in England demonstrates the problem well. It doesn’t seem to have occurred to the organisers that the college should be all about classroom teachers and the sentiment has been returned in the form of monumental indifference. With days to go in their fundraising appeal, they have only raised £20,000 of the £250,000 that they said that they needed in order go ahead. How much of this £20,000 comes from actual teachers is unclear and they now seem to have shifted the goalposts to 1000 donors rather than £250,000.

Contrast this with the overwhelming success of teacher-led, grassroots movements such as researchED, Northern Rocks, WomenEd, teachmeets or the many online forums, chats and blogs that are transforming the ways that teachers see themselves in the UK and around the world. Much of this is fueled by teachers taking charge of the agenda and taking a critical stance towards what the experts proclaim. It is a welcome development that Scotland now makes educational research freely available to teachers. It’s odd to contemplate that this is not standard practice and we should be arguing for such access everywhere else.

If teaching is a profession then it is certainly not the same kind of profession as medicine. I am not sure that it can be but I am growing more certain that it will be up to us to define what teaching is in the future.

Display of eurythmics by students of the Queensland Teachers Training College 21 April 1937

Display of eurythmics by students of the Queensland Teachers Training College 21 April 1937


The evidence for Dan Meyer’s TED talk

Those of you who are familiar with this blog will be aware that I have highlighted differences between my position and that of Dan Meyer. In particular, I have criticised some of the ideas that Meyer presented in his 2010 TED talk. The claims that he made can be typified by this extract from the transcript:

“So 90 percent of what I do with my five hours of prep time per week is to take fairly compelling elements of problems… from my textbook and rebuild them in a way that supports math reasoning and patient problem solving. And here’s how it works. I like this question. It’s about a water tank. The question is: How long will it take you to fill it up? First things first, we eliminate all the substeps. Students have to develop those, they have to formulate those. And then notice that all the information written on there is stuff you’ll need. None of it’s a distractor, so we lose that. Students need to decide, “All right, well, does the height matter? Does the side of it matter? Does the color of the valve matter? What matters here?” Such an underrepresented question in math curriculum. So now we have a water tank. How long will it take you to fill it up? And that’s it.”

Meyer goes on to talk of his use of video to create engagement with the problem. It is clear that he sees motivation as important. He also speaks as if these are ideas that he has developed and evaluated himself, in his own classroom – “It’s been obvious in my practice, to me.” – rather than anything that is the product of educational research. He does not refer to evidence that his is a more effective form of teaching. This would be fine if he was simply presenting ideas for us to contemplate but, instead, he is making a strong claim; that we need a general change to the way that we teach mathematics.

I have queried this lack of evidence before and I have suggested that it is a good example of the problem with how we talk about education. It is hard to imagine this kind of a discussion about the practice of any other profession (if we can class teaching as a profession – this might disqualify it).

In my view, Meyer’s ideas are at odds with cognitive load theory. The type of instruction he describes might be effective for students who are already relatively expert but the evidence suggests that it is unlikely to work with novices who haven’t had much instruction in the topic. A lot depends upon the enactment. If the open-ended task comes at the end of an extended period of instruction or it is brief and is followed by comprehensive, explicit instruction then this might work reasonably well. If we are hitting relative novices with such problems then it would seem to provide too little guidance.

In a recent exchange with Meyer on Twitter, I suggested that the difference between us was that I had evidence for my position and that he did not have evidence for his. He did not agree with this and when I asked for the evidence to support the claims in the TED talk, he offered the following:

Meyer evidence

I thought that I should look these up. They are not full references and so I entered the terms into Google Scholar and read the first paper that seemed relevant. I expected the evidence to be about problem-based learning because that is the way that I would broadly categorise Meyer’s method.

I was familiar with Mayer’s work on multimedia and it seems that this supports Meyer’s suggestions about using multiple forms of presentation, although it must be pointed-out that textbooks with diagrams do also fit the definition of ‘multimedia’. It seems sensible to use arresting graphics, visuals and video as long as we pay heed to cognitive load theory and its implications which, according to Mayer, include:

“(1) the presented material should have a coherent structure and (2) the message should provide guidance to the learner for how to build the structure. If the material lacks a coherent structure — such as being a collection of isolated facts — the learner’s model-building efforts will be fruitless. If the message lacks guidance for how to structure the presented material, the learner’s model-building efforts may be overwhelmed. Multimedia design can be conceptualized as an attempt to assist learners in their model-building efforts.”

This would not seem to support the idea of using distracting information with novice learners and I would argue that many textbook questions have substeps in order to assist learners in their model-building efforts. [I will also take this opportunity to add that Mayer has written an excellent paper on the repeated failure of discovery learning]

The paper that I found by David and Roger Johnson on ‘constructive controversy’ doesn’t seem to have much to do with the effectiveness of maths teaching, focusing largely on variations of social studies lessons. It does have some data and reports effect sizes. However, many of the dimensions on which constructive controversy is assessed are things like attitudes and motivation. When academic performance is reported, constructive controversy seems to lead to ‘higher-level reasoning strategies’. I think it would be important to look at exactly how this is measured, particularly since much of the discussion is about understanding the position of opponents. It is not obvious how this might apply in maths.

It also seems that  constructive controversy is not compared with explicit instruction in these studies (the comparison conditions are group ‘concurrence-seeking’, ‘debate’ where a judge decides on winners and losers and ‘individualistic efforts’). I am willing to concede that controversy may make something memorable and interesting but I am not sure that we have to follow the Johnson’s model in order to introduce it or that explicit forms of instruction cannot make use of controversy.

And so, finally, I reviewed an article by Kasmer and Kim on ‘The nature of student predictions and learning opportunities in middle school algebra.’ I can see that prediction might be a component of problem-based learning and that it might help to activate prior knowledge or give knowledge-gap experience to students. Yet the study in question seems to have no control group and so we can’t really establish the extent of the advantages of predictions over any other type of instruction (I did briefly look for controlled studies but couldn’t find any):

“In order to investigate the value of using prediction, prediction questions were developed and posed in one middle school mathematics classroom throughout one school year. The target content area was algebra: linear and exponential relationships.”

Let us assume that there are significant advantages to making predictions. This still does not imply the use of problems with distracting information and substeps removed. Predictions could as easily be incorporated into a period of explicit instruction as one of problem-based learning. We could ask students what they think will happen before demonstrating the correct method. In fact, I do this a lot.

In my view, the papers that I have presented do not provide compelling evidence that maths teachers should follow the approach to maths teaching that Meyer describes. Instead, there’s seems to be a much stronger case for using explicit instruction. You can find some of the evidence to support my view here and I intend to have an ebook available soon where I will discuss this at greater length.


The skill of competence

Making Learners Extraordinary ™

It is becoming increasingly clear that none of the jobs that people do today will even exist in five years time (apart from maybe undertaking). Therefore, there is no point in teaching students to regurgitate rote, disconnected facts. We cannot predict which facts they might need because we don’t know what they will be doing and, even if we did, the jobs of the future will not require low-level cognitive skills like fact-knowing. Instead, these jobs will be replaced by computers – Google will know facts for us. Instead, we will need to use higher level cognitive processes. Neuroscience shows that these are the executive skills that coordinate the brain.

At the Extraordinary Learning Foundation™, we have been working on better ways of developing the higher level skills of comprehension and communication. When you look at the performance of experts and scan their brains in a scanner then different areas ‘light-up’…

View original post 304 more words