Worked examples or problems? Attempting to resolve an apparent contradictionPosted: July 14, 2016
I have written before about the notion of “element interactivity” and how it might be linked to the choice of optimal teaching methods. It is worth noting that the concept of element interactivity is controversial and has been questioned by respected researchers such as Jeffrey Karpicke. So please bear that in mind when reading this post.
One valid criticism is that no experiment has been conducted in which element interactivity (EI) has been manipulated. Instead, researchers are making inferences by comparing very different experiments. For instance, we might point to an experiment in which EI is supposedly low and compare its results to one in which it is supposedly high, but a better test would be one where EI was a independent variable within the one experiment. It is therefore interesting to consider a new study where an attempt was made to vary the amount of EI. The study was carried out by Chen, Kalyuga and Sweller. Sweller has argued strongly for the concept of element interactivity and Kalyuga and Sweller are my PhD supervisors.
What is element interactivity?
You may be aware of the finding that our working (or short-term) memories can only process a limited number of items at any one time. For instance, it is relatively easy to remember the string, BQXAY for a short period but it becomes increasingly difficult as the string length increases. Attempts to remember a string such as HSHKIUSGDJHAY are likely to fail unless a specific memory technique is used.
This gives us some indication of the limits of our working memory. The idea of element interactivity extends this. If, for instance, we wish to remember the chemical symbol for a given element then there are only two items – or elements – that we must process; the name and the symbol. Learning one chemical symbol will have no impact upon learning another. However, if we want to solve an algebraic equation such as, “ax=b, solve for x,” then simply counting the items in the equation, a, x, = and b, is not enough because they all relate to each other in some way. Therefore, when we consider the number of items that the working memory needs to process, we need to include additional items to represent these relationships. Moreover, these relationships generalise across different equations of the same type.
We might therefore suggest that learning chemical symbols is a task that is low in element interactivity whereas learning how to solve linear equations such as ax=b is high in element interactivity. I repeat that this is a controversial idea but I hope you might recognise that there is a difference in complexity between the two tasks and that EI is an attempt to try to describe it.
Chen et. al. make a further claim that tasks that have a high EI for novices will have a lower EI for experts. This is because many of the required rules and relationships are available to an expert through long-term memory. If we return to the earlier example of the string, BQXAY then we might consider this as being made-up of five items. However, let’s replace it with the string, PANDA. This is now only one item because most of us will have a concept of a panda in our long term memories. We can therefore draw upon this concept as a single item. As the authors put it:
The generation and testing effects
“In contrast to its limitations when dealing with novel information from the environment, working memory has no known limits when dealing with organized information held in long-term memory. Activated by external signals, large amounts of information can be retrieved rapidly from long-term memory to working memory allowing appropriate responses to those signals.”
The generation effect is well-known among educational psychologists. Imagine you take two groups of students and give one a set of opposites to study – e.g. ‘hot/cold’ and ‘up/down’ – while asking the other group to generate a response themselves – ‘hot/___’ and ‘up/____’. The group who generate the responses tend to remember more of the pairs in a delayed test. The exact reasons for this have been debated and the discussion centres around whether the generation effect depends upon one factor or a number of different factors. It is similar to the testing effect where effortful retrieval of information leads to better retention than restudying.
The generation and testing effects have led to the idea that there are ‘desirable difficulties’ that we should introduce into teaching that will help students to learn more.
The worked example effect
The worked example effect seems to contradict the generation effect. In worked example experiments, students are either given worked examples to study or they are given the same problems to solve for themselves. You might expect that the act of generating your own solutions would be superior to studying worked examples but the opposite is usually found. Although there is an important caveat – if you repeat the same experiment with relative experts then they will learn more from solving the problems than studying the examples.
How can the generation effect and and worked example effect be reconciled? Most worked example studies have used immediate post-tests whereas generation effect studies use delayed post-tests and so perhaps this is the cause of the difference. This explanation might suit those who argue that the level of performance immediately after instruction has little relationship to what will be learnt long-term.
An explanation based upon element interactivity
Chen et. al. put forward a different explanation. Perhaps the effectiveness of competing strategies depends on EI. Recalling pairs of opposites would be a low EI activity whereas solving algebraic equations would be high EI, unless the people doing the solving were experts, in which case solving equations would also be relatively low EI. Why might EI matter? If it is low then there might be working memory capacity available to take advantage of the generation effect but if it is high then the key thing is to try to reduce any unnecessary load. In short, we might predict the following:
To test these predictions, Chen et. al. conducted an experiment with Chinese High School students in Years 10 and 11. The Year 10 students had not yet studied trigonometry whereas the Year 11 students had studied it (quite extensively, it seems – it’s worth reading the paper and reflecting on the content of High School maths in China). This provided the variation in level of expertise. Students had to complete two types of learning – memorising trigonometric formulae or using these formula to simplify trigonometric expressions. The former is low in element interactivity whereas the latter is high (if you accept the construct). Finally, conditions of learning were varied to represent generation/problem solving versus the use of worked examples.
Chen et. al. found evidence to confirm their predictions, although we would all wish to see a number of replications by different researchers before drawing too many conclusions. Interestingly, post-tests that were delayed still showed a three-way interaction between guidance, element interactivity and level of expertise, suggesting that immediate versus delayed post-tests are not the explanation for the difference between the worked example effect and generation effect studies.