A first look at TIMSS 2015

Embed from Getty Images

TIMSS is a series of international tests in maths and science that first took place in 1995 and that has been repeated every four years since then. The 2015 data has just been published and I have been trying to quickly digest the Australian version of the report. This post therefore has a bit of an Aussie slant – I will comment on other countries but I lack data on statistical significance.

It’s worth noting that TIMSS is a more abstract kind of assessment than the better known PISA. PISA sets questions in contexts, for instance by using mathematics to solve a practical problem. This means that there is quite a heavy reading load for PISA test items. In comparison, TIMSS has a more traditional feel, asking some context-free textbook questions such as 42.65 +5.728 = ?.

TIMSS assessments test maths in Grades 4 and 8 as well as science in Grades 4 and 8. The headline for Australia is that its overall performance is pretty stagnant :

  • Grade 4 Maths – mean of 517 – Significant improvement on 1995 but no significant change since 2007
  • Grade 8 Maths – mean of 505 – About the same as in 1995
  • Grade 4 Science – mean of 524 – About the same as in 1995
  • Grade 8 Science – mean of 512 – About the same as in 1995

So we haven’t gained much traction in these areas in the past 20 years. Why not? This is the kind of question that education research should be addressing.

It seems reasonable to look at the performance of a single country over time like this and try to draw a few inferences but I am more sceptical about comparing the performance of different countries. For instance, Shanghai is often cited for its PISA results but this is a city and not a state. In Australian terms, it would be fairer to compare Shanghai with Canberra. Similarly, it seems unfair to compare countries with smaller and more homogeneous populations with places like the United States. However, I still find the following results to be quite stunning:

  • Singapore, Korea, Japan, Hong Kong and Chinese Taipei are the only East Asian countries represented. They take out the top five places in both maths assessments and five out of the top six places in the two science assessments
  • In the Grade 4 maths test, the highest performing country outside East Asia is Northern Ireland (which did not take part in the Grade 8 assessments).
  • In the Grade 8 maths test, the highest performing country outside East Asia is Russia.
  • Not only did the countries listed above significantly outperform Australia in these areas but countries such as the United States, England, and Kazakhstan all significantly outperformed Australia in all areas.
  • 30% of Australian students – nearly a third – hit the ‘low’ or ‘below low’ benchmark in Grade 4 maths compared to 21% in the U.S., 20% in England, 14% in Northern Ireland and just 2% in Hong Kong (Hong Kong has none in the ‘below low’ category).
  • Interestingly, Finland was outperformed by the United States and England in Grade 4 maths, although I don’t know if this was significant. However, Finland did better than these countries in Grade 4 science (Like Northern Ireland, Finland only entered students at Grade 4).

Perhaps we might pause before sending more delegations of worthies to Finland to marvel at phenomenon-based learning. Instead, Australians might be better to head to Kazakhstan.

I await next week’s PISA results with interest.


9 thoughts on “A first look at TIMSS 2015

  1. Great idea would love to visit Kazakhstan. Though I’m guessing they don’t have the diversity of learners we have in Australia to compare. But a nice trip it would be!

    • Mike says:

      Oddly enough, earlier this year at an international competition that some of my students were attending, I eavesdropped on a conversation between one of those students and one of the competitors from…Kazakhstan. And they were comparing their respective education systems.

      The system as she (the Kazakh student) described it appeared to me to be both academically rigorous and adequately flexible. Needless to say, there was a foreign language requirement way beyond the pathetic 100 hours that we have in Australia. (I might add that this Kazakh student spoke near-flawless English.)

      Next stop Astana!

    • Why would they not have the diversity? It’s a massive assumption about a country that is little known. In addition, Finland is a relatively homogenous country and yet the lack of diversity has not stopped people going there to see what is special about the system – though it appears they have gone on the usual progressive trajectory of dipping results with greater progressive teaching.

  2. New Zealand also remains pretty stagnant and sitting below 500. https://mathmo.co.nz/2016/11/29/have-new-zealands-timss-maths-scores-really-improved/

    I agree that one should be careful when comparing the performance of different countries, but in a global economy, stagnancy is a concern when the leading countries are increasing their lead. We don’t like to compare ourselves to the Asian countries, but it’s worth noting that England’s scores have improved, following their respectable effort to ditch fuzzy maths, bring back traditional methods of computation and adopt teaching methods from Shanghai and Singapore. Northern Ireland has also done very well, their higher score possibly explained by a more homogenous population.

  3. Tempe says:

    One does wonder what methods of teaching and curriculum is Kazakhstan using. The only info. I could find is that they have a set curriculum and they use textbooks.

  4. Pingback: What should @Birmo do? | Filling the pail

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.