Studies that prove the existence of generic skills

Embed from Getty Images

The education zeitgeist assumes the existence of general skills and abilities. The influential Partnership for 21st Century Skills claims that there are ‘Learning and Innovation Skills’ such as critical thinking, communication, collaboration and creativity. The Australian Curriculum posits the existence of a ‘general capabilities’ such critical and creative thinking and personal and social capability which it requires schools to teach and assess, and has developed various rubrics to help do this. Which is not a trivial matter: The OECD has invested time and effort in developing its own methods for attempting to assess supposed skills such as collaborative problem solving.

If such skills did exist, then this would be huge for education. Typically, we find that knowledge and skills that students learn in one context do not transfer very well to similar contexts, and very poorly to quite different contexts. This is the problem of ‘transfer’.

If general purpose skills exist and, critically, we can improve them through the process of education – if they are trainable – then they should be relatively easy to demonstrate. All we would have to do is randomise participants into one of two groups. The first group would be given an active control i.e. some form of training that might conceivably lead to improvements in performance (this is necessary in order to rule out a placebo effect). The second group would be taught one of these general purpose skills. We would then measure the performance of both groups on a task from a very different context to the ones used in training. If the second group outperforms the first, and if this finding reliably replicates to other studies, we can claim to have identified a generic skill.

I don’t rule out this possibility completely. Some knowledge is applicable to a range of situations and so teaching it to students may well give us a transfer effect. Examples might include knowledge of logical fallacies or study skills. It is also probably useful to remember critical thinking heuristics such as ‘look at the issue from different perspectives’. The trouble is, as Dan Willingham points out, such heuristics are useless unless we know enough about the subject to be able to look at it from different perspectives, and that perhaps explains why we don’t find much evidence of transfer from teaching them. And the fact that we cannot point to a body of research showing transfer effects for broad concepts such as creativity, critical thinking and collaboration makes me sceptical that these really exist as trainable skills.

Ryan Campbell on Twitter has pointed me to a number of studies which he suggests demonstrate the existence of at least a trainable component to some of these abilities. I am not clear why this study supports the existence of generic skills and this study seems to be an exercise in the Hawthorne effect. However, a study carried out by Google, and reported in the New York Times, is a little more intriguing in that it seems to have relevance for the fashionable concept of collaborative skills.

Researchers at Google looked at different teams. They were somehow able to differentiate between effective and ineffective teams. Initially, they could find no patterns at all in what made teams more effective. However, after looking at group norms, they found that effective teams provided psychological safety.

Does this support a generalisable skill of collaboration? Not really. Firstly, the effective teams were effective without being taught anything at all, so this does not provide evidence for trainability. Secondly, this finding only applies to the kinds of teams that Google employees belong to. We don’t know whether other kinds of teams work in the same way. Setting this aside and assuming that it really is a generalisable finding, then it may be useful to know that effective teams provide psychological safety, but how could we train that as a skill in individuals? We could end up replicating form over deeper function in a cargo cult manner: Before too long, every employee knows the right things to say about teams but, underneath, they are still playing the same old Machiavellian games. Finally, what does this imply for school education? What if all we do is develop effective teams in a school context but do nothing to improve those team members’ ability to work successfully in teams in the future?

Generalisable skills have gained considerable momentum without much backing from research evidence and so I think these are reasonable questions to ask.


7 thoughts on “Studies that prove the existence of generic skills

  1. Greg,
    In a school setting wouldn’t the first items on a psychological safety improvement plan be the same ones found in any of the behavior improvement strategies you have mentioned?

  2. kesheck says:

    It seems just as likely to me that “psychological safety” made people feel safe enough to share their domain-specific knowledge with the group as “psychological safety” is some sort of generic skill.

  3. Michael Pye says:

    Read the article on the New York times and rather enjoyed it until the end when it finished with a blatant non-sequitur. Apparently (according to the New York Times) good groups are emotionally sensitive to each other and share speaking time. They conclude that one of their imaginary groups (A) does not do this because the meeting is kept on task and only one person speaks at a time. Earlier parts of the same article refute this and the idea that people could speak equally over large periods of time (big blocks when you have something to say) seems lost on them. Their is also no information on either groups emotional sensitivity to each other. Classic 2+2=5 journalism.

    Same issue in a TES article I read that told me the keys to teaching math is visual spatial and verbal reasoning (true for learning math) before admitting that they were only working on methods of improving these traits and there was a chance it was basically impossible.

      • Ryan says:

        Not sure if all the individual studies has random assigned groups (although that can be problematic with any research no matter how much rigor!)

        But did include studies with a control condition. See eligibility criteria below. I will check re allocation

        Eligibility Criteria
        To be included in the meta-analysis, a study needed to examine the effects of teamwork training by comparing teams in an experimental condition (i.e., those who received teamwork training) with those in a control condition (i.e., where teams did not receive teamwork training). Cross-sectional/non-experimental studies were excluded, as were intervention studies that did not include a control condition. As this review was only concerned with teamwork interventions, studies that focused on training taskwork—whether independent of, or in addition to, a teamwork intervention—were excluded. For example, as previously mentioned, simulation-based training (SBT) has been used as a means of training individuals to perform technical skills and also to enhance teamwork. In order for a SBT intervention to be included in this meta-analysis, it had to be clear that only teamwork (not technical skills) was being targeted during training. In order to address our primary research question, the study had to provide data on at least one teamwork dimension and/or team performance. The study also needed to provide sufficient statistics to compute an effect size. In cases of insufficient data, corresponding authors were contacted for this information. The articles were delimited to those published in the English language.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.