Regular readers will be familiar with the phenomenon of Finland’s continuing decline on international assessments such as the Programme for International Student Assessment (PISA) run by the OECD:
Commentators seek to explain this decline in a number of ways. Pasi Sahlberg, former policy advisor in Finland and now with the Gonski Institute in Australia, attributes it to the distraction of digital devices (In August I debated Sahlberg on the educational lessons we may draw from COVID-19 and you can catch that debate here). And in this post, I critically examined the suggestion that a decline in effort from Finnish students is the cause.
A newly published PhD thesis by Aino Saarinen sheds more light on the possible causes. Saarinen examined data on Finnish students from the 2012 and 2015 rounds of PISA with a few questions in mind: Were scores correlated with self-directed learning practices, use of digital learning materials in schools or participation in an early learning programme? Also, was variance in learning outcomes, which is apparently increasing in Finland, associated with any of these factors?
Similar to findings made by the OECD themselves, as well as by researchers using OECD data to examine a number of different countries, Saarinen found:
“Frequent use of self-directed teaching practices or digital learning materials at school were associated with students’ weaker learning outcomes in several knowledge domains. Instead, frequent teacher-directed practices were related to students’ higher learning outcomes. Moreover, frequent use of self-directed teaching practices or digital learning materials had more negative impact on students’ learning outcomes in students with (vs. without) risky background.”
Saarinen found little impact from participation in early learning.
Of course, these results do not directly address the question of what has caused the decline. It is possible, for instance, that there was even more self-directed learning in, say, 2006. However, this seems unlikely given the qualitative accounts we have of how Finnish education has changed over the last 20 years or so (see e.g. here and here) and the more recent turn toward approaches such as ‘phenomenon-based learning‘.
And there are limitations on drawing too many inferences from the questionnaires that PISA uses to assess the practices that students are exposed to. They strike me as a little eccentric, with potentially overlapping concepts often present in different constructs. But then, nothing is ever perfect. It would be great to run a statewide randomised controlled trial, but that’s not going to happen and so we have to make our inferences as best we can from the data as it stands.
If we do consider the data as it stands, the discussion highlights – again – that constantly pointing to Finland as an example of educational excellence is flawed. It is a sign either of ignorance of the last fifteen years of data, self-deception or worse.
Filling the pail 
Am I right in reading the PISA data that, while there has been a decline over the years, Finland still performs above average in most categories?
Yes, but so what?
So…
You say, “the discussion highlights – again – that constantly pointing to Finland as an example of educational excellence is flawed.” Meanwhile, it is an approach that has led to (significantly) above average results year after year.
I’m not denying the decline. That is its own issue to be discussed, for sure. But if you are going to use the PISA results as a benchmark, then it seems a bit disingenuous to say Finland is a bad example when it consistently performs above average in the very test you are using to demonstrate its weakness.
So… that.
Why do you think its position in the league table is important?
Especially given that you do not mention that fact once in your article. Seems a bit like a lie of omission.
Especially given that you do not mention that fact once in your article. Seems a bit like a lie of omission.
So… actual competency is irrelevant in a discussion about the quality of a system designed to create competency? It’s an intriguing position, Greg.
You’re trying to suggest that Finland’s education system is poor. And to back that claim up, you’re pointing to it’s results on a system in which it has consistently been amongst the top performers. The only way you can make Finland look bad by pointing at its PISA scores is by removing the context.
Let’s take reading… From 2012 to 2018, the average reading score in Finland has fallen from 524 to 520. A decrease. That’s bad. Finland’s education system cannot be very good.
Let’s add some context… In the same time frame, the OECD average fell from 496 to 487. That means Finland is significantly higher than the average both then and now, and also that the decline in Finland (4 points) is significantly more slight than the average decline (9 points).
So actually, perhaps Finland is not doing such a bad job after all? Perhaps we should be looking to Finland, since it is performing significantly better than average both today and over time.
With context, your point doesn’t even exist.
Also, lest I appear to be cherry-picking, let’s look also at Maths, where Finland fared worse. From 2012 to 2018, the average Maths score fell by 12 points, while the OECD average fell by only 5 points. So, not great. But in 2018, Finland was still averaging 507 compared to the international average of 489. So, if both continue to decline at the same rate, Finland will still be above average until at least 2033.
Being above average means that Finland is doing better than most countries, Greg. So to say that it is not doing a good job is utterly meaningless. Is it doing a perfect job? No. Can it improve? yes. But does that make it a bad example? No. Clearly not. It is a better example than most. By reference to the measuring stick that you have chosen.
Furthermore, one of the few countries that has been consistently increasing is Mexico. From 2012 to 2018, its average in Science has increased by 4 points, while Finland has fallen by 23. So, presumably you would suggest that Mexico is a better example that Finland? The fact that Mexico is fourth from the bottom, and far below average, is apparently irrelevant? It is a better model than Finland, by your logic.
I think you have a point clearly the Finns are doing something right. The issue is that what they have been changing in the last 15 years may be undermining that.
So is it increased distraction by iphones or is it new directions in teaching?
Evidence suggests the latter is what I am reading from Greg.
Specifically the increase in self directed learning.
Given it is hard to know relative to other countries what else is going on the measure of a decline correlating with an increase in some approach to education is the point Greg is trying to extract.
The risk with following the Finn’s lead is you might find without the other things they are doing and picking up the newest trends only from them (those that are in the news) will give you the same slope as they found when applying them.
This all seems obvious to me and Greg has pointed it out before and I have been following along.
But you can’t take anything from a dataset if you only look at one point. Finland is going down. Sure. We can talk about that. But we can’t conclude that it is a “bad example” if, in the light of context, it is better than almost every other country.
At this point, I should point out that I am not really a fan of OECD in any case. But if that’s the data we’re referring to, then we should refer to it honestly and transparently. Isolating one datapoint renders it close to meaningless.
Yes we can say that Finland is in decline. And as such we can ask why and consider how to repair. But as soon as we start to talk about it as a good or bad example for the rest of the world, then we need to look at it in light of the rest of the world. And in that context, while it may be in decline, it is still one of the top performers in the world.
Thank you for all your comments.
“if, in the light of context, it is better than almost every other country”
Looking at its rank compared to other countries is meaningless. Countries vary on a wide range of factors that affect educational outcomes, such as wealth, demography and even the orthography of the language. Finnish has a transparent orthography, meaning the relationship between letters and the sounds they represent is straightforward. So, when you say Finland is performing better than, say, Australia, you are not comparing like with like. One way, for instance, you could boost Australian PISA scores is to use only the results from Canberra – that’s similar to China submitting Shanghai’s results.
In contrast, comparing Finland now to Finland a few years ago is a much better comparison. Whatever they have done since 2006 is associated with a significant decline so it’s probably not a country to copy.
I do agree with you when it comes to following another country’s lead. I think Finland is certainly doing something right, though obviously not perfect. But when it comes to learning from them, that is a very complex process. What works in one country is not guaranteed to work in another if copied directly, so whether we decide it is performing well or not, we should still be cautious about what we take from it.
“Looking at its rank compared to other countries is meaningless. Countries vary on a wide range of factors that affect educational outcomes, such as wealth, demography and even the orthography of the language.”
Uh, then why are the PISA results a rank, if it is meaningless? And why have you used it as one of the measures to show decline, then?
Wouldn’t all those factors be controlled for, before the results are ranked in the way they are?
And can you please point to the research that shows that Finnish orthography gives them the advantage that you claim? Or is that just speculation?
Debating about how “Finland is doing something right” clearly misses the target by not acknowledging that the decline is, in fact, much more significant. Why? Because ed policy globally is determined on outcomes of participating nations. Researchers and ed wonks don’t give two hoots about where a nation stands in comparison to others…they examine the OUTCOMES much more rigorously. That is, ahem, why Greg continues to bang on about where the failures of Finland need to be examined much more thoroughly, rather than continuously report on where Finland is at in comparison to the rest of the world. And he’s right to do so.
Other nations continue to improve, such as Portugal as well, and this, along with Mexico “should be” held up as to what happens when nations pay attention to improving their education systems. Perhaps read this as well https://researched.org.uk/2019/02/27/everything-starts-with-the-curriculum/.
Disregarding how high performing nations have turned their backs on similar systems which are proving to be successful, needs to be acknowledged, rather than simply pointing out their ranking, like the media constantly does. Get past the headlines.
I find myself in the unenviable position here of defending an opinion I don’t really hold.
For one, I’m not a fan of PISA in the first place. Beyond that, I don’t think we should all be copying Finland, not necessarily because I think they’re doing anything wrong but because I don’t think copying another country’s education model is a very reliable solution, given the vast differences in context from one country to the other.
All of that said…
First of all, I have absolutely acknowledged the decline. I have put both factors side-by-side. The original article here failed to do that. Presenting the decline as though it were the whole story is disingenuous. I will not budge from that position.
“Pointing to Finland as an example of educational excellence is flawed.”
What else can we call excellence if not Finland’s performance? Even if it continues to decline, it will still be above average for the next decade or more. How can that be? First, because it is currently so far above average, and secondly because the average is also in decline.
That point is the most important, by the way. I’m not just pointing out that Finland is above average. I’m also pointing out that while Finland is in decline, so is the average, and so are a great many of the other countries on the list. So even the decline cannot be taken in isolation, because Finland is far from alone in its decline.
So. Finland is above average in its performance today and has been for well over a decade. Finland is in decline, but so are other countries, and so is the global average. In some areas, Finland’s decline is in fact slower than the decline of the global average.
I am not a fan of PISA. But the whole point of having a system that collates so much data is so that that data can be considered in context. The OECD PISA data is not a case study on Finland. It presents each country in comparison to the rest of the world. So that’s how I’m reading it.
And this brings us back to Mexico. Yes. It should be applauded for its improvement, and it can perhaps provide a model for improving an underperforming system. But it’s still an underperforming system now. As Mitch Hedberg might say, “it used to be bad; it still is, but it used to be too.” About a third of students in Mexico perform below minimum. So we can look at the work they have done to improve, but if we want to see a country that’s turning out high performing students, then Mexico is not that country.
In Finland, 86% of students performed at Level 2 or higher across the subjects, while in Mexico that number is 50%. The OECD average is 77%. In Mexico, less that 1% of students were top performers (Level 5 & 6). In Finland, 12%. The OECD average is 9%.
At the end of the day, if you have a kid, would you put her in a Mexican school or a Finnish school? Again, I don’t propose that any one country copy any other country. It’s not that simple. But *if* we want to use PISA to identify the country’s that are doing Education right, then Finland is going to be on that list.
If you want to point to Finland as a country that is offering a poor example for how to do education, then PISA data is not your friend.
You should be a fan of PISA, unless you can point out another global standard which clearly illustrates the pitfalls of edu policies as well as what works. PISA is just one tool in the toolkit to determine what systems work. If we do not have standards to go by, how do we know if something is working or not?
You’re arguing with the wind because you want to be right, and would prefer to be obstinate to the point of ridiculous in defending your view. It has been pointed out to you now several times why the league standing of Finland is pointless. No serious ed researcher is even remotely interested in its placement. They’re only interested in the outcomes and how it compares to previous years. Because THAT is what shapes future ed policy…not simply its ranking. And no…one cannot claim to suggest Finland is in a good position with its education, because it’s a sinking ship. So why suggest the world stay on board if we’re all going to drown?
As for Mexican schools, their basic knowledge at primary level is far superior to most countries in the Western world. At least it used to be. They learn basic maths, english, and writing at a much higher level than most schools in the US or in Canada. They learn several languages and a much more rigorous curriculum than most of their richer jurisdictions north of them and perhaps that needs to be examined why. Oh and wait…we already know WHY. Because their school system used to be based on the knowledge rich pedagogy that France and other European nations used to follow. The Europeans brought it with them when they came to the New World and they produced an excellent group of students even in the public system. That is, until the edu wonks and consultants from America started infecting ed systems globally with their inferior, failed teaching methods, such as Finland, and a catastrophic decline in academic performance followed.
You’re entitled to your opinion, but be prepared to support it. Because in this instance, your opinion is all you’ve got, which doesn’t bear much weight in a sinking ship.
That is absolutely the right way to read the PISA data. And, like you, I am not a big fan of the enterprise.
How foolish to suggest that nobody cares about Finland’s placement in the ranking.
So it’s pure coincidence then, that the country all of these “serious edu researchers” happen to be so interested in just happens to be the one at the top of the list? Why aren’t they all spending their time talking about the most average countries? The only reason we are even having this very conversation is precisely because Finland ranks so high. If it didn’t, nobody would care about it!
And you say I’m arguing with the wind!?
No, I’m presenting concrete data as evidence from the very source that you are endorsing. Data that you have not responded to, by the way.
And I am the one arguing with the wind…
TL;DR
I don’t object to the notion that Finland’s decline should be addressed.
I object to presenting that decline void of any and all context, and then saying that because of that decline, we should reject Finland as an example, when the contextualised data tells quite a different story.
And once more, I am no proponent of PISA nor of the “Let’s all be Finland” train. I just like to see data presented clearly and honestly.
Finland is doing something right, alright — it has little poverty and has a tiny immigrant population.
There’s a whole bunch of countries out there that score the same as Finland once the immigrant population is taken out of the scores. There’s nothing like not speaking the native language fluently and having parents that were undereducated (relative to their new country) to really hamstring a child’s education.
I personally would not be consoled as a Finn by being told that the new progressive educational policies were going to take 25 years to properly trash my education system.
Wealth is also important. Mexico pays teachers poorly and struggles to get them and their pupils into school consistently. Their schools themselves often lack vital equipment. At that end of the spectrum the greatest improvements are very unlikely to be curriculum at all. Looking at what they are doing is unlikely to yield very many answers for countries with much greater wealth.
Yes. All very good points.
Of course, we have the beginnings of a chicken and egg scenario when we bring in generational data. Are kids performing well now because they had educated parents? Yes. Were their parents educated because they received a good education? Presumably. Are wealthier kids likely to perform better? Yes. Is wealth a result of good education? At least partly.
I don’t make any of those points to argue with you, though. I just think they’re part of the whole, complex picture.
And of course, you’re right about the prognosis. I made that point more as an indication that there is plenty of time for them to fix the trend. Though, history suggests that trend is not showing any signs of recovery. Still, it only takes one effective intervention.
One assumption I make, given what is known about the Finnish approach, is that what looks like a decline in educational standards is actually more of shift in focus. Perhaps they’re spending less time and energy on the things that OECD monitors, and more time on things that are simply not captured by OECD. And if that is the case, then they can afford to keep that trend going for quite a fair way further and still be above average by OECD standards while also developing in other areas.
Also, “completely trashed” is a bit of a stretch. Excellent to average is disappointing, sure, but it’s not a tragedy on quite the scale you suggest. Especially *if* it comes with other benefits.
Again, my only real point is this: if you want to say that Finland is not doing very well, you don’t really get very much support from PISA data when viewed as a whole.
I disagree there are lots of cases where the context of the trajectory is more interesting than the absolute
GDP for countries would be immediately referred to as not doing well with a 12 year downtrend for the top country.
League standings of a top sports team would immediately be referred to as not doing well if they were falling for 12 years.
Salaries or share of wealth of those at the median income are referred to as not doing well if they are declining even in countries where the median is very well off globaly.
There is nothing abnormal as describing decline relative to your own past results as not doing well.
Some immigrants (eg from India and China) in countries where they are well integrated (eg UK) raise educational achievement averages.
Immigrants from some other countries and/or who are not well integrated, lower educational averages. I wonder whether something along these lines is happening here?
In Canada it’s already been proven that immigrants have helped our educational standards, although teacher unions would beg to differ, and love to suggest that immigrant children are the reason why our performance has been on a downward trajectory for 20 years. When, in fact, if it wasn’t for these kids coming to Canada, our performance would be a lot worse.
It is quite complicated, but integration isn’t always the issue.
In New Zealand the Pacific community tend to do OK educationally if wealth is taken into account and are well integrated socially. But because most are quite poor, and the parents sparsely educated, they drag down the PISA average. Take out the Pacific community and NZ’s rankings rise considerably — despite them working quite hard. (They also leave school early on average, because they need to work to help their families, but PISA tests younger than that.)
And while Canada might get an advantage, there are few European countries like France where is you take out the immigrant community there is a massive jump in PISA scores.
From the Saarinen dissertation linked to by Greg:
“pedagogical practices such as self-directed practices or use of digital learning
materials explain only a relatively small part of the variance in learning outcomes. Hence, the explanatory power of pedagogical practices on learning outcomes should not be overestimated.
Moreover, the findings obtained in PISA datasets cannot be directly interpreted as indicators of various school curricula.”