What would you conclude if, after persuading a school district to adopt your preferred model of maths education and studying a self-selected number of teachers in a couple of schools identified by the district as exemplars, you found that it wasn’t working out the way that you imagined and students told you that they found elements of the program frustrating? Well, you should perhaps conclude that there is a problem with your model. It would be a stretch, don’t you think, to blame the state’s maths standards? And yet that is the finding of a new study by LaMar, Leshin and Boaler which you can read open access in all its unfalsifiable glory.
Following the usual practice of one of the study’s authors, we are not told the real name of the district where the study was conducted. Instead, it is referred to as ‘Gateside’.
The U.S. has a different way of teaching mathematics to the Australian and English systems that I am familiar with. In the U.S., students typically follow a set sequence of maths courses: Algebra 1, Geometry, Algebra 2, Pre-Calculus and if they can fit it in, AP Calculus. This sequence means that students need to study Algebra 1 in the Eighth Grade if they are going to complete AP Calculus before the end of high school. AP Calculus is effectively, if not explicitly, an entry requirement for many top American colleges.
This tends to lead to a split where more advanced students take Algebra 1 in Eighth Grade, with their less advanced peers waiting until the following year. Gateside had decided to disrupt this model by making all students wait until Ninth Grade to study Algebra 1 and then teaching them in mixed ability classes. It would only be in Eleventh Grade that students could choose to either do Algebra 2 or Algebra 2 plus Pre-Calculus. The district also decided to eschew ‘procedural teaching’ in favour of a form of group work, the Platonic ideal of which the authors refer to as ‘complex instruction’. A dash of mindset theory was also supposed to be added somewhere.
The researchers approached the district and the district provided them with a list of seven out of their fourteen high schools that had, “fully implemented Complex Instruction and the district’s core curriculum.” The researchers then chose two of these schools to study. They approached the ten Algebra 1 teachers and eight of these agreed to participate in the study.
Apparently, one of the features of complex instruction is that students work together on ‘groupworthy’ tasks. The idea is that these tasks should only be possible to complete as a group. Moreover, every member of the group should be able to contribute in a substantial way to the overall solution and nobody in the group would finish the task before everyone in the group finished the task.
Clearly, such an idealised maths task is hard to design. Students who are more advanced at maths are going to be able to contribute more to a maths task than students who are less advanced. So, both groups became frustrated. The more advanced students were frustrated by having to constantly explain the maths to the less advanced students – i.e. by doing the teacher’s job – and by having to wait for them before they could move on. The less advanced students felt under pressure to work quickly. The teachers charged with managing the ensuing chaos developed a system where they would stamp the work so they could keep track of who had finished what and therefore decide when each group could move on. The researchers disapproved of these stamps.
This is a predictable clash between researcher idealism and teacher pragmatism. The researchers gave the teachers an impossible task. The teachers then tried to make this practical. The researchers then disapproved of the teachers’ solution.
The authors suggest that the main problem with the district’s approach – the one that caused all the frustration – lay in the tasks the teachers were setting and the fact that these tasks were driven by the curriculum standards. For instance, asking students to solve is something the authors consider to be only a ‘beginning’ level of task. Finding the ‘zeros’ of the graph of is only slightly better and classed as ‘developing’. Instead, what we really want students to be doing is an ‘expanding’ task such as working out the pattern in a growing sequence of squares like this:
To be frank, this is the sort of task I would expect to see in maybe a Fifth Grade classroom. It has some connection to algebra, but not much. Most of the time that students are engaged in a task like this, they will not be developing their understanding of algebra. The first two tasks pay forward to later study of algebra, including calculus, in a way that the squares problem really does not.
So, if the state standards are forcing the teachers into tasks like the ‘beginning’ and ‘developing’ ones, they appear to me to be correct to do so. Attacking these standards seems like an excuse.
The real lesson from this study is something quite different: If you are able to persuade a district to adopt this model, they adopt it and then point you towards the best implementations of it, you will find it does not work. If that’s the case, what are the chances of getting this model to work at scale? I suggest they are very low.
It is worth noting that the researchers cite evidence that the new approach is superior than the old one. Apparently, the algebra failure rate dropped across the district. However, it is my understanding that U.S. schools do not use common, standardised assessments to determine who passes such courses and so this could simply be a function of applying a lower standard. We cannot check, because we don’t know the actual name of the district.
And the authors make a number of strong claims throughout the paper – claims that are potentially falsifiable such as, “When mathematics is taught as a set of procedures to follow, many students disengage, and various studies have shown that procedural teaching is particularly damaging for girls and students of color.” However, the reference provided is often then to another non-randomised study similar to the present one and involving some of the same authors.
One such claim that particularly drew my attention was that, “…procedural teaching encourages students to take a ‘memorization’ approach to mathematics, which has been shown to be associated with low achievement.” This refers to a paper from Scientific American that I have examined before and that I do not believe demonstrates this finding.
Nevertheless, people will point to this paper and districts will embark upon similar reforms, thinking they are evidence-based.