A first look at TIMSS 2015Posted: November 29, 2016
TIMSS is a series of international tests in maths and science that first took place in 1995 and that has been repeated every four years since then. The 2015 data has just been published and I have been trying to quickly digest the Australian version of the report. This post therefore has a bit of an Aussie slant – I will comment on other countries but I lack data on statistical significance.
It’s worth noting that TIMSS is a more abstract kind of assessment than the better known PISA. PISA sets questions in contexts, for instance by using mathematics to solve a practical problem. This means that there is quite a heavy reading load for PISA test items. In comparison, TIMSS has a more traditional feel, asking some context-free textbook questions such as 42.65 +5.728 = ?.
TIMSS assessments test maths in Grades 4 and 8 as well as science in Grades 4 and 8. The headline for Australia is that its overall performance is pretty stagnant :
- Grade 4 Maths – mean of 517 – Significant improvement on 1995 but no significant change since 2007
- Grade 8 Maths – mean of 505 – About the same as in 1995
- Grade 4 Science – mean of 524 – About the same as in 1995
- Grade 8 Science – mean of 512 – About the same as in 1995
So we haven’t gained much traction in these areas in the past 20 years. Why not? This is the kind of question that education research should be addressing.
It seems reasonable to look at the performance of a single country over time like this and try to draw a few inferences but I am more sceptical about comparing the performance of different countries. For instance, Shanghai is often cited for its PISA results but this is a city and not a state. In Australian terms, it would be fairer to compare Shanghai with Canberra. Similarly, it seems unfair to compare countries with smaller and more homogeneous populations with places like the United States. However, I still find the following results to be quite stunning:
- Singapore, Korea, Japan, Hong Kong and Chinese Taipei are the only East Asian countries represented. They take out the top five places in both maths assessments and five out of the top six places in the two science assessments
- In the Grade 4 maths test, the highest performing country outside East Asia is Northern Ireland (which did not take part in the Grade 8 assessments).
- In the Grade 8 maths test, the highest performing country outside East Asia is Russia.
- Not only did the countries listed above significantly outperform Australia in these areas but countries such as the United States, England, and Kazakhstan all significantly outperformed Australia in all areas.
- 30% of Australian students – nearly a third – hit the ‘low’ or ‘below low’ benchmark in Grade 4 maths compared to 21% in the U.S., 20% in England, 14% in Northern Ireland and just 2% in Hong Kong (Hong Kong has none in the ‘below low’ category).
- Interestingly, Finland was outperformed by the United States and England in Grade 4 maths, although I don’t know if this was significant. However, Finland did better than these countries in Grade 4 science (Like Northern Ireland, Finland only entered students at Grade 4).
Perhaps we might pause before sending more delegations of worthies to Finland to marvel at phenomenon-based learning. Instead, Australians might be better to head to Kazakhstan.
I await next week’s PISA results with interest.