FAQ: Models

Science proceeds by developing models to approximate reality and then testing the predictions of these models through observation or experiment.

Science does not deny the existence of an underlying reality and yet it does not claim to know or describe it. All it claims is the predictive and explanatory power of its various models.

While I have only recently come to explicitly draw this distinction, this is something I think I have implicitly known for a long time due to my physics training. In physics, we are constantly confronted by the incompleteness of our models. The best example of this is wave-particle duality where we have two conflicting models that both accurately predict different aspects of the behaviour of fundamental particles; the model of a wave and the model of a particle. These two models can be reconciled in a mathematical framework that is impossible to put into words by drawing on analogies from the everyday world.

Why is this important? Because there is a tendency for people to dismiss models in cognitive science by pointing out that they are incomplete or that they do not describe every aspect of the working of the mind. This is a misunderstanding.

For instance, in ‘Why don’t students like school’, Dan Willingham posits a simple model of the mind in order to help explain some constraints on learning. This model consists of the environment, the working memory and the long term memory. Some have criticised it on the basis that it is an oversimplification; that the working memory has sub-components – such as the phonological loop – and that the model excludes elements such as the sensory buffers. It may be an oversimplication, but it order to demonstrate this we would need to see how the addition of these elements would change the predictions Willingham makes on the basis of this model. If they don’t change these predictions then they are irrelevant to Willingham’s argument. If they do change these predictions to ones that are less aligned with experiment and observation then we should definitely leave them out. The only case where we should include them is if they change the predictions of the model in a way that is relevant to Willingham’s argument and that represents a superior description of reality. Otherwise, there is value in keeping it simple.

After all, it’s models all the way down.

Advertisements

8 Comments on “FAQ: Models”

  1. Dylan Wiliam says:

    As George E.P. Box said many years ago, “Since all models are wrong the scientist cannot obtain a “correct” one by excessive elaboration. On the contrary following William of Occam he should seek an economical description of natural phenomena. Just as the ability to devise simple but evocative models is the signature of the great scientist so overelaboration and overparameterization is often the mark of mediocrity.” (Box, 1976 p. 792)

  2. Rajiv says:

    “Science does not deny the existence of an underlying reality and yet it does not claim to know or describe it. All it claims is the predictive and explanatory power of its various models.”

    Thanks for this Greg. It would’ve taken me quite a bit of thought and consideration to arrive at this statement. An example of the power of the borrowing and reorganising principle!

  3. I really like this post. When I write about CLT, I often think (and write) about the difference between reality and models. Part of what makes CLT so interesting is that it is a model of a scientific model — simple enough to use to think through what it is that science is interested in, what issues arise with your choice of model, the inevitable tradeoffs between predictive power and usefulness. (This was a main purpose of mine in writing this essay.)

    As always, a minor quibble:

    Science does not deny the existence of an underlying reality and yet it does not claim to know or describe it. All it claims is the predictive and explanatory power of its various models.

    This is questionable. Electrons really do exist, many scientists would say. So do black holes and microbes. A great deal of science claims to do more than pragmatically make predictions.

    (You might disagree that science claims to know that microbes exist. But if we deny ourselves this, on what grounds do we ever claim to be able to describe or know our world?)

    The insistence that science just is in the business of finding useful models — not of discovering or describing an underlying reality — is probably more prevalent in the human sciences. For whatever reason that psychology has difficulty landing on a stable paradigm (I’m just beginning to read about this), there is no clear dominant model. This means that there are choices, and the choice of model leads to us being more reluctant to claim that any single one of them describes reality.

    Physics, with its clear succession of paradigms and models, has no such compunctions. They really do claim to know reality in physics — or at least a lot of them do.

    • Your minor quibble equals my thoughts too. A huge part of the physical sciences involves describing reality, collecting data on it – think of the classification of living things, or the classification or rocks – and this is in itself informative particularly when it involves hard to observe things like stars, microbes, DNA etc. Models often come later, after the data has been collected.

    • Nick says:

      I think you kinda missed the point of the blog… in fact I think you kinda proved it. CLT exists in the same way that electrons exist. We only know this because of the ability of the model to make predictions. The argument that every possible description isn’t accounted for doesn’t discount the model. Please provide a precise description of the colour of an electron before you demand to know what CLT smells like.

      BTW, your link is broken and I would love to know about the difference between predictability and usefulness as I believe they are one and the same.

      Meanwhile, my school district is spending money so that I can have Boaler’s book waved around at parent conferences and am told that practicing math is bad for learning math. But, to be fair, unless science can describe exactly what not knowing math facts tastes like, can we really know she is wrong? It’s not like the future opportunities of children was at stake here… it is just a fun semantic argument.

      • Nick: I agree with the point of the blog. I was criticizing a minor, philosophical point of the post. (Greg loves when I get minor.)

        CLT exists in the same way that electrons exist. We only know this because of the ability of the model to make predictions. The argument that every possible description isn’t accounted for doesn’t discount the model. Please provide a precise description of the colour of an electron before you demand to know what CLT smells like.

        Bla bla electrons can’t have color they’re too small. But, of course, you’re right. CLT is legit science. The model is supported by evidence. It makes good predictions.

        But, again: electrons and protons are really supposed to exist. Like really really. Here’s an excerpt from a favorite philosophy paper (“Ethics and Observation”):

        Consider a physicist making an observation to test a scientific theory. Seeing a vapor trail in a cloud chamber, he thinks, “There goes a proton.”… He can count his making the observation as confirming evidence for his theory only to the extent that it is reasonable to explain his making the observation by assuming that, not only is he in a certain psychological “set,” given the theory he accepts and his beliefs about the experimental apparatus, but furthermore, there really was a proton going through the cloud chamber, causing the vapor trail, which he saw as a proton.

        Greg, however, said that science is agnostic about describing reality. I don’t believe that this is generally true, though it may or may not be true about CLT. (I’m inclined to say that a prediction is only good if it in fact explains reality. An interesting test case is modern machine learning, which we might think of as making predictions without explanations.)

        Meanwhile, my school district is spending money so that I can have Boaler’s book waved around at parent conferences and am told that practicing math is bad for learning math.

        I’m sorry about this, but no one is forcing you to talk about philosophy of science. By all means, spend your time focusing on other matters. But Greg wrote a post about philosophy of science. Beats me why you’d read and comment on the post if you’re not interested in all that.

        BTW, your link is broken and I would love to know about the difference between predictability and usefulness as I believe they are one and the same.

        cognitiveloadtheory.wordpress.com

        Maybe predictability v. usefulness isn’t the right way to put it. But in the context of CLT there are debates about which factors to include in CLT models and experiments. So, for example, Sweller prefers to not include motivational factors in CLT experiments. Other CLT researchers (e.g. van Merrienboer) are interested in applying CLT to contexts in which motivation matters a great deal, and so are inclined to include motivation as a factor in their model.

        Which model is true? Which is correct? Like Greg says, there’s no way to include everything in your model of learning. You have to make a choice, and different people make different choices. When a model is simpler — includes fewer factors — it might allow for greater rigor and predictability. But it might be too simple, so simple that it wasn’t useful.

        The reason why I think this is important is because — again, as Greg says — a lot of people miss this point about learning science. (I think it’s because they expect learning science to be like physics and other “hard” sciences.) As a result, many dismiss the models for not reflecting all the nuance of reality. I also think that this leads to hard-liners who don’t understand the simplifying assumptions of the evidence we have for teaching. (I see this in feedback research especially.)

        So a better understanding of the nature of models of learning in science would be good for education, I think. Which, of course, is the point of Greg’s post and I agree with it. (Where we differ, maybe, is if we think that ALL science is like this or if this is a difference between different fields.)

  4. Unfortunately there is not just a tendency to dismiss a model like CLT, but a widespread unwillingness take a rational scientific approach to evaluating educational interventions. I continue to feel frustrated by the quite strongly held view among colleagues and senior managers that you can find research to support anything ergo it has no intrinsic value.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s