Is Maths Pathway evidence-based?

A number of people have recently mentioned to me an Australian teaching programme known as Maths Pathway. The ‘co-founder and chief visionary’ is Richard Wilson, a former management consultant who appears to have taught for a period of time before starting the enterprise with Justin Matthys, a Teach for Australia alumnus. Interestingly, the advisory board is a firmament of Australian educational luminaries.

The programme has it all including growth mindsets, personalised learning, rich learning, differentiation, ‘targeted explicit teaching’ and data-informed practice. However, it also makes a claim to be ‘evidence based’. Which is a strong claim.

For instance, large scale studies of growth mindset interventions have failed to show much of an effect and differentiation lacks a clear definition and supporting evidence. If you look at the images used on the Maths Pathway website, you see students mostly sat at computers while the teacher is interacting with a small group. This is the kind of personalised learning that has led to fairly mixed results.

Personalised learning is also not new. When I was in primary school, I did SMP Maths. This involved taking a laminated card from a box, completing it, and then going back to the box to swap it for another laminated card. It was pretty boring and I still remember the day that the teacher abandoned the boxes and actually taught us something. There is an emotional connection when a person tries to communicate with you that is absent from cards and computers and that probably accounts for the ‘embodiment effect‘ found in multimedia learning.

It is pleasing that the evidence for explicit teaching is sufficient for an enterprise like Maths Pathway to use it as part of its pitch, but we need to bear in mind that the evidence base for explicit teaching comes from studies of whole-class explicit teaching. At first look, it seems obvious that a more tailored approach will work better for students, but you have to consider the trade-offs: A whole class receiving instruction together that may not be tailored to their precise individual needs may well be more effective than far shorter periods of more personalised instruction interspersed with lots of time alone in which to become distracted.

When it comes to rich learning, this strikes me as the kind of thing that some in the maths teaching world like to stroke their beards about:

Rich Learning is an engaging experience for the whole class, where mathematical concepts are explored using open-ended activities with multiple entry and exit points. This type of learning focuses on developing critical thinking and reasoning skills and encourages students to think like mathematicians with no assessment involved.

The evidence-base for asking relative novices to behave as experts is very weak, at best.

No matter my doubts, Maths Pathway claim to have evidence that their approach works. They seem to have developed a rubric with levels going from 1 to 10A, which I assume is intended to correspond with the year level objectives of the Australian Curriculum. When they assess students prior to starting Maths Pathway, they find they have made a lower rate of progress along the Maths Pathway levels than after they have been in the programme for a while. Interestingly, they then use this to generate a massive ‘effect size’ of 1.42 which they then compare favourably with John Hattie’s effect sizes for things like direct instruction (0.60).

I’m not convinced Hattie’s effect sizes are a particularly sound way to gauge the effectiveness of different educational approaches, but used in this way, I can’t imagine even Hattie would approve. Clearly, testing students using a rubric that the programme is designed around will show a benefit for participants in the programme. The assessment questions are likely to be similar to the course materials.

Unfortunately, school leaders are probably unaware of these kinds of criticisms. Coupled with the fact that Maths Pathway is aligned with rhetoric from the recent ‘Gonski 2.0’ review about personalised learning and mapping individual progression, you can see why the programme is popular and has grown to include over 35,000 students as of December 2017.

Although I have my doubts, Maths Pathway could potentially be an effective teaching programme. It may be a positive step in the right direction, particularly for schools whose current maths programme is weak or who lack specialist maths teachers.

Is Maths Pathway evidence-based? That depends on the standard of evidence you apply. I would use the term to refer to programmes that are supported by evidence from randomised controlled trials or, at the very least, lots of high quality epidemiological studies using standardised tests as the outcome measure. I would not apply it to a before and after study using a test designed by the programme developers.


28 thoughts on “Is Maths Pathway evidence-based?

  1. Tom Burkard says:

    Somehow, I can’t see that a programme where “mathematical concepts are explored using open-ended activities with multiple entry and exit points” would really be suitable for schools that lack maths specialists.

  2. Easy study:
    Pick 30 maths pathways secondary schools.
    Pick 30 non pathways schools.
    Control for Index of Community Socio- Educational Advantage (ICSEA) Compare 7 to 9 NAPLAN growth data.
    If Maths pathsways school’s numeracy growth is statistically significantly higher than non maths pathways schools that would be compelling evidence it works.
    The school down the road from us uses it. Their 7 to 9 growth is way worse than ours. My sample size of 2 schools is lacking.
    Someone should do that study.

    • Sue says:

      Agree. Furthermore, this year’s NAPLAN data could be quite revealing in all schools that have been using MP for the past 2 yrs (ie. comparing the Yr 7 & 9 data for each individual student)…. In Yr. 7, MP Diagnostic Testing placed our son (admittedly gifted at Maths) at approx. Yr 9 level – so MP (in all its wisdom) set him practically NO Yr 7, or 8 content (which he and we were not fans of). He has just done the Yr 9 NAPLAN and for the first time ever, did not complete the whole paper, despite having been Band 10+ in his Yr 7 NAPLAN (ie. early in Yr 7, prior to much MP)! I fear his Yr. 9 NAPLAN results might be quite damning for MP…

      • Not finishing is not an issue if he was doing NAPLAN online. The adaptive nature of the online test means stronger students will be under time pressure.

      • Sharon says:

        Hello Sue,
        My son was in a similar place when he did his Naplan test in year 7. He has had a year of MP, unfortunately due to Covid 19, there will not be a Naplan test for him this year. What were your son’s year 9 Naplan results for maths like?
        I’m looking for the evidence, but I think it is a handbrake and it kills extension learning and investigation that might have been done in a normal maths class

  3. Mitch says:

    I do like that in the picture of ‘the perfect classroom’, computer screens are easily seen by the teacher. Such an easy change if you have space in your classroom to have tables facing away when on computers and facing towards you when listening to you.

  4. Jenny says:

    My children are at a school which now uses Maths Pathways – it has been devastating. The students hate it and are crying out for someone to ‘teach’ them something!! I could not be less impressed.

    • Tom Middle says:

      Or is it the parents who are unfamiliar with style of learning who are ‘crying out’ for them to teach them something?
      The role of the teacher hasn’t disappeared. it has been transformed. Whats the point in teaching Pythagoras to a class of students when only a third are ready to learn it?
      Unless you have a class where every student has the requisite knowledge of triangle properties, square numbers, and algebraic manipulation, your amazing lesson will fall on 67% deaf ears.

      • Sue says:

        Tom Middle, my children are not at the same school as Jenny and I do not know her, but I can answer your question – it is the students hating it. Parents are understandably going to be concerned when their children hate something at school and will investigate and attempt to share their child’s voice. Our gifted son, who has always loved and excelled at Maths and is incredibly computer savvy, hated Maths Pathway and following his own repeated requests for the past 2yrs, has finally left MP and is in a class with a Maths teacher this year and no MP (he is loving it and thriving). Others we know have changed schools solely because their kids hated MP, others we know have started paying expensive maths tutors as their kids hate MP and are not learning on it, others do the modules with/for their children because their children hate MP, others are complaining that they are having to teach their children Maths, as the kids hate MP and are not learning on MP (MPs takes the credit for all such student progress, as these confounding variables are not accounted for)…

        Your Pythagoras scenario is easily rectified by streaming (which many schools do) and good Maths teachers (which many schools have). MP is no doubt going to be useful in a school that has no Maths teachers (eg. very remote), as it is preferable to nothing – but in a school with good Maths teachers, it is both insulting to the teachers and devastating to the students (as stated by Jenny). The role of teaching had already been transformed – our older son has always had great Maths teachers, who provided both whole class and small group teaching and utilised ‘Maths Space’ for both homework and for working ahead/behind in class at their own pace (which the majority of kids loved). I could elaborate further, but my son’s and many other student’s amazing observations of MP will fall on 100% deaf ears, as 11,12 and 13yo children don’t communicate in arenas as this, it is their parents who share their children’s voices… yet it appears, are questioned when they do so…

    • Angie says:

      This program is destroying children’s learning. Kids in my daughters class are cheating just to get through each testing section. This is providing teachers and parents with fake results in terms of the academic level of the child. From the words quoted directly from my daughter ‘this course is great for dum dums who hate maths and just not want to learn.’. All the hard work my daughters year 6 teacher did with her is now being destroyed by this computer program. It turns teachers into supervisors and I feel that the maths faculty should feel that it is an insult that their classroom has turned into a babysitter service where they have no need to do anything but walk around the classroom while the kids do irrelevant testing. How can you test a kid on something they have never been taught? Get this out of schools!

  5. Annette says:

    Introduced at my daughters school and absolutely hate the program. Seems to promote lazy teaching, in one term groups received 1 rich learning small group tutorial th rest of the time spent on computer by themselves. If they strike a problem they have a 4 step solution, look at answer, watch video, ask a friend and then ask teacher. Steps my child took and the teacher sent her to start steps again. Children need to complete and achieve 6 modules a fortnight to be on task for one years growth. This is happening, when l asked for results on average was told it was confidential. My child already struggling, Grade 6 at Grade 5 levels but when l checked on modules to complete she had some Grade 2 modules still to achieve. Education department needs to step in and stop this.

  6. Campbell says:

    MP has destroyed my son’s interest in maths. It tends to work for those students who are interested in maths to start with and have a natural aptitude. I suspect the MP model prescribed by the MP people is not adopted at our school. It is supposed to include significant 1:1 teacher time and small group work. We are told that our son’s teacher sits at the front of the class doing nothing. The 4 step problem solving approach is an absolute farce (check the answer, watch the video, ask a friend, before asking the teacher). What a joke.

  7. milla says:

    i had a question: if 4 cups of water is 1 bottle and it takes 14 bottles to fill a bucket how many cups are there? so the question was basically 4 times 14 but it turns out the bucket was had ten letters and the cup was a measurement and every bottle was a litter how? and ten litters was under the question!

  8. Andrew says:

    My son has experienced 2 years of Maths Pathways in yrs 7-8. Initially we thought it was a great concept, to identify areas where he was not strong and help build this knowledge before progressing to more complex topics. However after two years he has learnt very little. The content is very limited: the online videos are 1-2 minutes long, and if meant to replace a teacher instructing on a topic, lack both detail and the ability for students to ask questions if not clear. And the exercises are shoddy at best: 1-2 pages of exercises are not in my opinion sufficient for a student to retain a topic, particularly given the nature of the program allows students to shift between core streams of maths, rather than systematically building on knowledge in previous exercises.

    The idea of the teacher assisting small groups in the class in non-existent in my son’s school – the teacher sits at the front of the class and if the students are having issues, directs them to use the MP program to find the answers.

    On one module my son is now on his 8th attempt, every time he does a test he cannot work out where he has gone wrong, there appears to be no feedback in MP on where mistakes are made, and there has been no intervention from the teacher.

    Discussions with his teacher have not been much help – again the teacher suggests my son either use the program to find that answers, or attend a lunchtime Maths Club – rather than actually helping him in class
    Contacting the MP developers seeking answers has drawn a blank too – they pretty much suggest its the schools responsibility.

    We have of course tried extracurricular tutoring to help my son improve, both through myself and private tutoring. However I feel there is too much complacency that MP is a one-size-fits-all, and the school is putting too much faith in a program that appears to be very lacking in depth.

    Reminds me of a dodgy maths software package that some door-to-door salesman tried to sell my wife and I once – tried to make us feel guilty that we wouldn’t pay him 10K a year for what was obviously a few scanned exercise sheets and no real teaching.

    • Lisa says:

      This is definitely a situation where your Maths teacher is negligent. Maths Pathway is a (potentially useful) tool, but it is up to teachers to actively continue teaching in order for it to be effective.

  9. Sarah Borg Chaves says:

    Maths Pathway is cheap for schools to run and I have yet to meet a student who likes it. My son now hates Maths more than ever. I came to read this article as a parent and a teacher of 40 years because my son hates Maths now and I am trying to get him to turn around. Maths Pathway appeals to schools as it is cheap teaching. Larger class sizes, no streaming needed (which means fewer teachers needed as lower class sizes should have a low teacher to student ratio). I teach IT but found the parent info hard to understand. Son was progressing over 122% but gets a D term after term. If I couldn’t see that – how do parents with poor literacy understand? School said he got individual teacher help but son says teacher never taught one lesson, one concept but sat at desk whole lesson. For this to happen, a whole lot of people in a school are enabling this company. Thankfully, the school I work in did a lot of investigation and rejected it, I walk around seeing teachers working with students and support Maths boys getting help in very small class sizes – and progressing to General Maths in the year. If you are a poor reader, you have to use the text to voice option but comprehension is still the problem.

  10. A concerned student says:

    lol this is old but anyway as a student at a school which uses mathpathways i can testify that it is a program the hinders the students mathematically and is bad for education overall if you wish to hear more details my contact information

  11. Tim says:

    Ditto my children don’t like MP. At first novel but now they miss the teacher interaction.
    Teacher sits at the front engaged on the smart phone and any question is referred back to MP on the computer. I think both students and teachers are disadvantaged and unfortunately our children are the experiments. When ever does real life go at our own pace?

  12. thomas says:

    Math Pathways is the worst thing ever introduced . Independent learning is for home schooling and lockdowns . We need more zDirective learning

  13. I came here looking for feedback about MPP after a search on Maths Pathways and have to say ‘great article’.

    Based on the comments made beneath the article, I have this to add.

    I have a daughter who used to love maths until MPP came along. She’s bored and hates it but does the work and still manages to get a high grade. She says her friends who used to hate maths now like it because of MPP. She admits *most* kids in her class prefer it.

    While I would love to campaign against MPP because it killed my daughter’s enjoyment and enthusiasm for maths, it does (grudgingly) appear to be good for the majority of kids. Some kids are outliers. It makes sense for a school to implement a method that catches as many of the pool that it can. And even though my daughter hates MPP, she’s still hitting above the standard as she always has been.

    Kids are not going to like every subject, and sometimes their enjoyment of a particular subject will lapse for a multitude of reasons. Learning in spite of enjoyment is a skill they will need in the world of academia. Just like working a job in the real world in spite of enjoyment is a skill we all need.

    While MPP can promote ‘lazy teaching’, I’m not of the opinion “teachers should be insulted” because a teacher that wants to teach will do so regardless of what program is in place. They’re not going to do nothing all day. I find that idea more insulting than MPP. If a teacher is happy to sit behind a desk all day and do/say nothing, I doubt the quality of their usual teaching was any good to begin with.

    I agree that testing a program for suitability should not come from the founders of the program itself (duh), but there were government-run comparative tests performed with different state schools using control groups and those that implemented MPP. I didn’t even have to search very hard to find it. The Australian education board is not going to let some random program interfere with its syllabus without taking a long, hard look at it.

    Here’s the 2007 study that I found to have the most comprehensive data online for the public to view, There were other studies done in 2002 and 2000.

    Click to access ED498258.pdf

  14. Louise says:

    This is a reply to Strangewriter. The report you have provided a link to refers to the MPP program which is a K-8 maths intervention program run in the U.S. It has nothing to do with the Maths Pathways being purchased by schools in Australia. The government has not completed any studies to confirm the claims made by this company as in Australia, the mode of curriculum delivery is totally up to each school. This is why it is so frustrating for parents. I find that the program over-inflates growth and will ultimately set students up to fail. Judging how good the program is by how a student feels does not cut it for me. If it was so good, why do students have to return to tradition instruction for Yrs 10, 11 and 12?

  15. Daniel Burns says:

    Strangewriter, the report you refer to is based on an American system and has nothing to do with the Math Pathways used in Australia. The Education Board has nothing to do with how schools deliver the curriculum as it is up to each to figure out how best to present it. This program sets students up to fail as it does not replicate the environment in Yr 11 & 12. It does away with a consolidatory exam which students should be learning how to study for.

  16. Justin Mahon says:

    “Destroying children’s education”, “videos replacing the teacher”, “the children hate it”, “why don’t they actually teach”. What a lot of pearl clutching, generalised nonsense. This article facilitates the hysteria in the comments by failing to fully explain the program in all its dimensions, acknowledge that implementation (school and teacher) is critical, explain the rubric to curriculum mapping properly and conflates data across various impact assessment studies casually. Many students and schools are getting a great deal out of this program, but I suspect the fact that it identifies learning gaps and provides hard data about effort and growth is confronting for many instead of being understood as the start of success of you frame appropriately. Furthermore, mathematics learning should challenge students if it is to be effective. It’s time for parents to help children with the discomfort of growth and have higher expectations of them. My God children are entering a world where they will one day wish their parents had high expectations….

  17. April says:

    My children hated maths pathways from the start, but one of my girls had completed the year 11 equivalent in year 10, always scored above the mean in Japan and has never struggled a day, until she hit year 11 and now she hasn’t passed a maths test yet. No maths, or any other subject, should be able to use their own evaluation tool, such as the modules test, to gauge their own success. If my girl has already completed year 11 maths, why is she struggling to complete basic functions like linear and quadratic equations…

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.