In yesterday’s post, I implied that research demonstrating that certain practices are associated with improvements in ‘conceptual knowledge’ should be read cautiously. I suggested:
“If you have the time, try reading some of the ‘productive failure’ literature. This often assesses ‘conceptual knowledge’ and yet the questions that are used to assess it will probably surprise you – they often look for all the world like recall of relatively simple declarative knowledge. This is why I am more impressed with evidence of transfer…”
Productive failure is on my radar as part of my PhD research and so I have been reading the relevant papers.
What I didn’t know at the time of writing is that this issue has already been investigated in a far more comprehensive way by Crooks and Alibali. Their article is not paywalled and is worth a read. Thanks to the helper elves who alerted me.
Crooks and Alibali note that the questions researchers use to assess ‘conceptual knowledge’ are not exactly what we might expect. For instance, they have noticed the prevalence of questions that ask students for a definition of the equals sign (i.e. what does “=” mean?).
I doubt anyone really thinks that conceptual understanding is equivalent to being able to trot out definitions. It might be a good proxy, provided test subjects have sufficient language skills and provided nobody has explicitly taught them the definition which they could, presumably, just parrot (see Willingham’s ‘menagerie lion‘). But that’s the problem: Why wouldn’t a teacher explain what it means to their students? Any student who then gives an accurate definition could either be showing conceptual understanding or could simply be remembering what their teacher said. The difference between these two possibilities rapidly becomes a philosophical question, particularly since it would be so difficult to detect.
Given the oddness of the research questions, I am more inclined than ever to think that conceptual understanding isn’t the thing we think it is. The more you stare at it, the more it disappears from view. Is it a subjective sense of realisation? Is it knowledge of certain underlying principles? Is it, as in the examples provided by Dan Meyer, just the difference between novices and experts, perhaps related to the development and connectedness of their schema? I think it may be all of these.
What I doubt is that there are ways of assessing it that are not highly intertwined with other skills and knowledge.