Better than science

Embed from Getty Images

It is often argued that science is no good for analysing educational practices. The use of science is dismissed as ‘positivism’ or perhaps ‘scientism’. The claim is that human relationships are really complicated and so cannot be subjected to the same analytical techniques as atoms and molecules. We cannot possibly know how any given individual will react to a particular approach and so the determinism of science is profoundly flawed. Some have even argued that such complexities mean that there is no such thing as a teaching method.

Although I accept the limitations of science – we cannot use it to decide what is moral – I am sceptical about the idea that science has little to offer education. It is similar to claims that people used to make about medicine. The whole point of using a statistical approach is to tease out underlying mechanisms. Statistics take account of the fact that students are not identical. True, I can never claim that if I use a specific technique with a particular student then I will obtain a certain result. However, I can makes claims about the likely effect based upon a large number of students and I can make generalised claims about the relative effectiveness of technique A compared to technique B.

Yet I would accept that there are problems that arise: There is the replication crisis in the social sciences more generally where different groups of researchers cannot reproduce the findings of an original study. This may relate to ‘p-hacking’ – the tendency to analyse and reanalyse a data set until something turns-up that appears to be statistically significant. Some things will appear to be significant just by chance so perform enough analyses and you can manufacture ‘significant’ results out of nothing. We could point to these issues and argue for better quality research or we could use them to bolster a claim against science.

So let us conduct a thought-experiment and assume the anti-science case. Let’s accept the argument that science is simply the wrong tool for examining education and think about what this would imply. I think there are two logically consistent positions that someone could take if this is what they believe.

We cannot make any causal claims about different educational approaches

We might suggest that, essentially, nothing can be known. We may be happy to toss out the scientific evidence for explicit forms of instruction but we would have to do the same with the evidence for collaborative learning. We would not even be able to make claims that there is ‘no best way’ to teach because that would imply knowledge of the relative effectiveness of different educational approaches.

This would be a hard path to follow because people make causal claims about teaching all of the time, including those who eschew science. It is almost impossible to talk about education without doing so. A mentor might suggest that a student teacher should ask more ‘higher order questions’. Why? On what basis?

Adopting this stance would require us to accept that anything goes apart, perhaps, from methods barred on ethical grounds. Even then, some techniques will cause obvious harm such as the use of physical punishment yet we could argue that approaches that lead to less learning or that waste time and resources are unethical and yet we would have no way of judging this.

We have something better than science

The other logical stance is to assert that there is some process superior to science that we can use to assess causal claims in education.

There are two main candidates that appear in the literature and are often intertwined. The first we might paraphrase as, “A great philosopher once wrote…” This is the practice of taking the writings of (usually French) philosophers and borrowing from them. We read phrases such as, “Using Bourdieu’s concept of habitus…” and so on.

I am not sure why I or anyone else should accept an argument from authority. How do we know that these guys are right? What should we do when different researchers’ exegesis of their works leads to different conclusions? It all seems a bit scriptural to me.

The second approach is the use of qualitative studies. Such studies have descriptive value in fleshing-out what different practices may look like in the classroom but they are weaker than science in teasing out causes because they are subjective and therefore prone to the myriad biases that plague human thought. For instance, if we are favourably disposed to something then we are likely to accentuate the positives and deemphasise the negatives. In contrast to science, which has the potential to determine whether effects transfer to different contexts, a subjective description of a particular classroom is simply a subjective description of a particular classroom.

Such descriptions represent a sophisticated version of the personal testimonial. Many teachers are utterly convinced of the effectiveness of a method through their own experience of employing it. Yet testimonials are the hallmark of quack science and for good reason. They are not systematic and are effected by various biases such as the sunk cost fallacy – you don’t want to think that something in which you’ve invested 20 years of your career is a load of old rubbish – and regression to the mean.

Sticking with science

For my part, exploring these alternatives makes me want to stick with science. I don’t mind being called names for doing so because it seems like the best bet. If someone can develop a process that is better than science for establishing causal relationships in the social sciences then I am happy to change my position. Simply highlighting science’s flaws is not enough. I want to know that you have something better.

It also strikes me as a dodgy argument to highlight the flaws in science and suggest that nothing can really be known, only to then turn around and claim that adoption of a particular pedagogy will lead to greater motivation or deeper learning.

How do you know that?

Advertisements

25 Comments on “Better than science”

  1. Some really good points here. Completely agree that science can be applied to education; its complexity is a challenge, but not an insurmountable one. Agree too that there’s far too much argument from authority; assumption that because someone famous came up with a concept it must be a valid one.

    One comment about qualitative studies: Over the last couple of decades, qualitative methods have morphed into Qualitative Methods – essentially a discipline of its own based on a particular constructivist worldview. It doesn’t follow that (lower case) qualitative methods are not scientific e.g. structured observation, repertory grids, laddering etc.

    • teachwell says:

      Indeed – the use of qualitative methods by constructivists is to avoid the scrutiny and the reality presented by quantitative ones. A shame because I think qualitative can lead to better understanding and framing of quantitative questions and unearth possible reasons but these still need to be explored systematically.

  2. I think one could claim that there’s ‘no best way’ while still throwing out science. In that I mean there are other ways knowing than through science alone. For instance, someone with vast experience could claim on their intuition that they know explicit teaching is more effective than PBL. Do I believe the individual vs a study? That would very much depend on the rigour the study and how much value I place in the individual.

    Now I wouldn’t say these other ways knowing are better than science as you imply to do so would give great comfort to purveyors of pseudoscience. I think science has a place front and centre in the education debate. That said, it would be scientism to say than only science can inform us of the best way to teach.

    • Stan says:

      I think this argument comes down to what we mean by “I know” and “We know why we know”. An individual can know something with certainty and be wrong. But if we are going to agree on why we can say we know something we need something that looks exactly like science.

      Even if we decide to base what we know on the judgment of those with lots of experience we would want to show scientifically that this is reliable. See https://www.gjopen.com/ where The Good Judgment Project has a wealth of information on how to judge the forecasters.

      Instead of the question, do we know this, it is useful to use the phrase, how much can we rely on this.

  3. Brian says:

    I cannot see any way in which it could be true that scientific methods could not be applied to education or any other area social science. The extent to which results are generalisable was always an issue in my day, I assume this is still the case.

    By the same token, for me the issue is mainly one of “treating learners as populations” or “treating learners as individuals”. Maybe my perspective is too simplistic, but for me I tend towards treating all kids as different and individualising my instruction whereas others see learners as a homogeneous group and maybe research findings are generalisable across homogeneous groups.

    Isn’t the issue that we are making statistical inferences in different circumstances rather than applying what works in science to social sciences.

    Doesn’t it boil down to how we look at learners, as individuals or as members of populations. This by necessity is perhaps an ethical issue.

    • Greg Ashman says:

      Interesting attempt to set up a conflict. Scientists analyse populations so that they can make decisions that are better suited to individuals. On what basis would you make decisions that are best suited to individuals?

    • Chester Draws says:

      but for me I tend towards treating all kids as different and individualising my instruction whereas others see learners as a homogeneous group

      Now I know you are not proposing this argument in it’s strong version, but let’s try applying this to medicine and see where it leads.

      A doctor says “but for me I tend towards treating all kids as different and individualising my treatment whereas others see patients as a homogeneous group”.

      You know what, medically we are all individuals. But there are also treatments that are shown to be effective on practically everybody. If I have an abscessed tooth I ask for a root canal — I don’t ask my dentist to try a variety of other things first, because I might be different. And there are treatments that are found to be ineffective on everybody.

      Because while we are all different, we have the same underlying structures.

      The woo brigade love the “we are all individuals, so we need individual treatments” because it gives them free rein to plug their nonsense. Any study that shows that homeopathy is mere placebo effect is discarded because that assumes we can be studied as a group, and we are all individuals.

      I think it is the same with education. Our individual differences are not half as important as people think they are. There are methods that are effective and methods that are not effective for all of us. Sure, there might be some marginal cases that work for some and not for others — as there are in medicine — but that doesn’t negate the fact that some things just do not work and some things do work.

      And the education woo-meisters love to plug their versions of learning styles, say, based on the “obvious” fact that individuals are different. Yet they have no evidence, and resist any attempt by others to provide any. They want our differences to over-ride any science.

      He’s an interesting parallel. Sports trainers do not have the progressive vs traditional problem. Despite the fact that they are all training different people, they have no issue with deciding that some techniques are effective and some are not. And they are happy to use science to distinguish between the two. Now that isn’t to say that everyone trains the same, of course not, but that there are still techniques that should be used and techniques that definitely shouldn’t be used.

      Given that we are all physically different, how is it that sports trainers don’t fall into the trap of having to justify every technique for every person?

  4. Pedro says:

    Reblogged this on From experience to meaning… and commented:
    Interesting post by Greg!

  5. patrickamon says:

    No one has ever made direct observations of a mind. Yet, I hope we can agree that minds are crucial to education. One approach is to argue that we no one has ever made any direct observations of quarks, or exoplanets either, but we are able to make inferences about them from indirect evidence, and that this represents the most promising line of research into the mind. It might be argued, though, that this rather misses a trick. Whilst no one has ever made any direct observations of a mind we all enjoy the peculiar priviledge of having (or, perhaps better, ‘being’) one. Phenomenological descriptions of mental experience are not subject to scientific verification, but our understanding of mind would be very much poorer without them. Dewey’s Analysis of a Complete Act of Thought in How We Think is not subject to scientific verification but almost anyone who reads it recognises that that is indeed what thinking is like.

    • Michael Pye says:

      By Phenomenological descriptions I assume you mean the definition of personal experience. If so this can very much be open to scientific verification. No scientist has every lacked person experience even of indirect phenomena, an example would be Hoyle’s refusal to accept the bing bang theory as it refuted a life time of knowledge and experience.
      The condition is so well known it is nick named the Nobel disease as so many world class scientists have fallen prey do it.

      It is hard to recognize that in another life my beliefs and ideas would be radically different simply due to the different interactions I would have, and therefore the different ideas I would develop. Science is an attempt however flawed to move us past that and allow us to both check, verify and if necessary change our core beliefs.

      The example you gave of quarks and exo-planets is interesting because in those fields many theories and variations have been proposed and we are constantly refining our knowledge and therefore concluding that many of those ideas are therefore wrong or highly unlikely.

      • patrickamon says:

        Yes, I do not mean that reflecting on personal experience gives any special insight into the nature of physical reality, only that it may be a source of insight into psychological reality. We may of course seek to understand other humans in the same way that we understand physical phenomena; by observing correlations etc etc. A richer understanding, though, I think, may be arrived at by a sympathetic appreciation of another person’s experience, an appreciation that probably ultimately will have to have its origin in introspection. I’m suggesting that Shakespeare or George Eliot may have at least as much to tell us about humans and our behaviour as does any amount of statistical psychology.

  6. David F says:

    Sadly, trying to get some of the major ed stakeholders to pay more attention to the science end of thing is tough, espeically when the non-science stuff supports their pre-existing beliefs.

    For example, “personalized learning”–something being promoted by the Gates Foundation in the US. One would think that the Gates Fdn would be pro-science, yet their justification for personalized learning is: “In personalized learning settings—which happen inside and outside the classroom—teachers assess students’ strengths and needs in order to better align their teaching with each student’s learning style and interests while maintaining high standards.” Of course, they claim that “digital tools” will help achieve this. http://k12education.gatesfoundation.org/college-readiness/personalized-learning/

    Calling Paul Kirschner…

    • Michael Pye says:

      Personalized learning is a valid avenue to pursue. Like most ideas it has been to readily accepted without critical review. Using computer programs to provide feedback is already a useful if limited tool. (I.e Khan Academy) and it may well improve in the future though we should not yet count our chickens. You are quite correct that individual learning styles and flawed ideas of differentiation are inefficient and unhelpful.

      The Gates foundation is heavily effected by the prevailing winds of educational ideas though it has resulted in data that other researches have been able to access and analyze, in that sense it is pro-science..

  7. eanelson2014 says:

    Science is in part about measurements that can be replicated. The brain where we think can be measured in some respects, and thus studied scientifically. Science has measured and confirmed that when the brain solves well-structured problems, the working memory where we think can hold and manipulate 3 to 5 non-memorized elements of knowledge at any one time, plus all elements that an individual has well-memorized and can recall in response to problem cues. That puts a premium on thorough memorization in math and science. See http://arxiv.org/abs/1608.05006 .

    Don’t educators have a moral and legal obligation, just a doctors do, to align our practice in aiding those who come to us for help with what science says are best practices? When science identifies best practices, at what point does doing something different become malpractice?

    • I agree, well put. Unfortunately education has proceeded on ideological grounds for so long it seems a seismic shift in thinking is required before scientific evidence will be widely accepted.

      • Michael Pye says:

        Debated some colleagues today about the importance of evidence yet again and how we do have some limited but useful insights. Pretty sure I had little success., it needs a organization wide approach. This is were the quacks win hands down, even when we get an idea out they manage to twist it as not enough people are interested in the details.

  8. […] Better than science → […]

  9. We can keep a healthy scepticism about applying science to education (or anything else for that matter) while at the same time respecting science’s immense value for informing day to day practice and helping slay the never ending scourge of the edu-quack if we simply use the Lindy heuristic when applied to research. This is a simple way education can avoid scientism while still getting all the positive benefits Greg mentions in the article. So in a nutshell, if a testable idea has been around for a while and not been falsified then it is probably worth a teacher’s time exploring.

    Nassim Taleb has written lots about the Lindy effect in anti fragile and this article:https://medium.com/incerto/an-expert-called-lindy-fdb30f146eaf

    • Michael Pye says:

      We can’t avoid being accused of scientisim as it is a pejorative used to dismiss an argument. We should of course be skeptical of all research and results(this is part of science) but this does not mean being skeptical of the process of science it self. It is perfectly reasonable to use a tool while being aware of its limitations, the only way we can shift peoples understanding is to teach them a basic understanding of the process and how it differs from there preconceptions. Be careful with Taleb he is an interesting read but his ideas are not exactly rigorously presented.

      • Here is a very short summary from Holton which you might find interesting Michael.

        “In politics and society at large, important decision are all too often based on deeply held presuppositions,ideology or dogma — or, on the other hand, on headlong pragmatism without study of long-range consequences.

        Therefore I suggest the adoption of Skeptical Empiricism, the kind that is exemplified by the carefully thought-out and tested research in science at its best. It differs from plain empiricism on the sort that characterized the writings of the scientist/philosopher Ernst Mach, who refused to believe in the existence of atoms because one could not “see” them.

        To be sure, in politics and daily life, on some topics decisions have to be made very rapidly, on few or conflicting data. Yet, precisely for that reason it will be wise also to launch a more considerate program of skeptical empiricism on the same topic, if only to be better prepared for the consequences, intended or not, that followed from the quick decision.”

        https://www.edge.org/response-detail/10123

        Regarding Taleb’s rigour, you are probably referring to his popular books which are written for a wide audience. If you are looking for all his technical notes he releases them all and they are extremely rigourous. I’ve put the direct links to some below.

        https://arxiv.org/a/taleb_n_1.html

        If you have a strong maths background (I don’t) he also has all the maths behind his main ideas here:
        https://drive.google.com/file/d/0B8nhAlfIk3QIR1o1dnk5ZmRaaGs/view

        Hope that is of interest


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s