Before someone in your organisation goes blowing the budget on a set of virtual reality headsets, it might be worth having a look at the findings of a new study by Makransky, Terkildsen and Mayer. The research enthusiasts among you may recognise Mayer for his Cognitive Theory of Multimedia Learning (CTML) and this theory does play a part in the study.
The paper is a fascinating read and I can’t capture all of it but I will attempt to explain some of the main findings.
The test subjects were 52 university students who were learning about protein expression through a computer simulation of a laboratory task. In the low-immersion condition, the students viewed the simulation on a regular computer and manipulated the apparatus with a mouse. In the high-immersion condition, the students donned Virtual Reality (VR) headsets that made use of a mobile phone. These headsets provided a 3D version of the lab that the students could explore. They could select objects by clicking a button on the side of the headset.
In addition, the researchers manipulated the ‘redundancy effect’ by providing text in some simulations and text with narration in other simulations. This is an effect that is well known in Cognitive Load Theory (CLT) and CTML and it arises when subjects are asked to read information while listening to verbal narration at the same time. This causes interference because, as Mayer would argue, both sets of words need to be processed in the auditory channel of working memory. It is therefore better to have either the written words or the narration rather than both. This is the reason why you should never read-out or talk over text that you present on a PowerPoint slide.
The authors point out that there are two competing hypotheses about the effect of low immersion versus high immersion (actually, they comprehensively cover a range of ideas but I will stick to the two main ones). Firstly, arguments based on those of John Dewey would predict the high-immersion environment is more authentic, giving more of a practical experience and that this should therefore lead to more learning than low-immersion. Conversely, arguments based upon CTML or CLT would suggest that high-immersion VR would introduce more extraneous details to process, increasing overall cognitive load and degrading learning.
Students were randomly assigned to conditions, with every student attempting one low-immersion and one high-immersion activity. Interestingly, the simulations were in English but the surveys and post-tests were in Danish – I’m not sure how that would effect things although it may increase the cognitive load of the simulation, depending on how fluent the students were in English.
The researchers measured cognitive load directly using an Electroencephalograph (an EEG), a device that records electrical activity in the brain via electrodes attached to the head. The EEG was initially trained by making measures of people performing typical working memory tasks such as accurately reciting lists of digits (digit span tasks) or performing mental arithmetic, as well as looking for correlates with subjects’ own reports of cognitive load. A company has developed proprietary software to process all of this, as well as subtract effects due to people blinking or other muscle movements. I find this quite impressive.
After the simulation, students answered questions directly assessing their knowledge, as well as transfer questions that required them to apply their new knowledge in novel situations. The low-immersion condition led to higher scores on both tests but this was only statistically significant for the knowledge tests. The transfer test items had quite a low reliability and so, with a sample this size, it would have been hard to see an effect.
The main finding was that the low-immersion, computer version of the simulation was more effective than the high-immersion, VR headset version. However, the subjects did report a significantly greater ‘sense of presence’ when using the headsets, which is hardly surprising.
The EEG allowed the experimenters to calculate the proportion of time that students were cognitively overloaded. For the first simulation task, there was no significant difference but in the second task, the VR headset students spent a significantly greater amount of time overloaded. This is consistent with the predictions of CLT and CTML.
Interestingly, the redundancy effect was entirely absent. This is odd but it may be due to a combination of how it was tested and how subjects responded. Most evidence for the redundancy principle comes from comparisons of narration versus text and narration rather than text versus text and narration as used here. In addition, some subjects appeared to ignore the text and just listen to the narration. According to CLT and CTML, this should be effective because it makes use of the modality principle of mixing narration with visual images. I think this area is worth exploring further because it might hint at something we haven’t fully worked out.
So should your school go out and buy those VR headsets? The best we can say from this research is that they may be motivational. For instance, they could be used as a whole class or individual reward for effort or they might help generate excitement about a topic (although this will not necessarily persist once the headsets are taken off). However, if you are buying them because you think your students will learn more then this study suggests you are wrong.
Filling the pail 
So wild, I love the word rubbish and have also been studying the expression of proteins. What does it all mean? 😉
If any school I was working at ever decided to spend stupid amounts of money on these silly things, I’d leave.
They are cheap as chips if you use the students mobiles. They are also naff. Motion sicknesses and technical issues galore if you stream. Now if you go oculus rift then you are talking high spec pc and an expansive headset. Well over a grand each.
Well, the study is basically is irrelevant for one simple reason: the technology they used. Mobile VR-devices with smartphones basically just show 360-images and -videos of content. The true potential of VR learning however lies in the interaction with virutal objects as you would do in real live, so your motoric systems kick in while learning. That’s a whole new level.
Haptic feedback isn’t commercial viable yet. We also have no real idea how realistic it will be. The more real the scenario looks:The more sensitive we are to inconsistances.
While this research will be superceded it is not intrinsically irelavent.
It is not about haptic feedback, Michael. It’s about being able to move in a virtual world and interact with it in a natural way. That IS available right now with HTC Vive, Oculus Rift or Playstation VR.
Smartphone-VR is such a loose interpretation of that highend experience with just a tiny fraction of the effects, that a study based on Smartphone-VR is indeed completly irrelevant. It’s like saying that computers are irrelevant for class without providing a mouse, a keyboard and a internet connection. How would that help?
Most education experiences involve cheap smart-phones not expensive high end systems, the experience is disorientating and certainly not realistic. Even the good stuff is limited in its performance. I think the technology is cool but you are counting your chickens before they have hatched, the classic problem with tech in education. Wait for the systems to improve, get cheaper, and be appropriately tested. This study opens up the possibility that the outcome may not be what we expect, and the burden of prove rests with the technology.
I know you need a catchy headline but it seems VR helmets are a great way to do CLT experiments. Useful in a research lab but in a classroom there would be cheaper ways to play with technology if someone finds a good reason to do that.
I’d agree Stan. I’d imaging that extraneous CL would have to be high (too high?) in order to provide the immersive feel. Or maybe we need to redefine CLT for VR and all the various design principles that were developed years ago. Maybe the ‘redundancy principle’ and others are no longer fit for purpose in a VR or AR setting. Like you say, a great way to do loads of experiments that are needing to be done.
You miss the main reason for such things, Greg.
Schools buy nonsense like that to show how modern and hip they are. That they perform no better than the old tech is irrelevant.
A “smart” board has a few features my white board doesn’t have — they’re too low for those at the back to see, they break down, they are hard for novices to use, and they are expensive. I’m assured they have some pluses, but I struggle to see them. Nonetheless schools will spend many thousands buying them. They make the senior staff, who don’t have to use them, feel they are cutting edge.
Too true. Smart boards and all the rest of the technology wizz stuff also generate extra work that is not necessarily productive.
Smart boards are a great example of a technology you would have been better off skipping. The main feature of being able to save the image is available with any smartphone or an old ipod- where the incremental cost for the ability to take a photo is zero.
Technologies have two types of feature – doing something you already do more efficiently or doing something you can’t currently do.
In the first are word processing, preparing presentations, email and spreadsheets. For these it is relatively easy to evaluate the cost benefit tradeoff.
Anytime someone wants to buy a technology product to do something you don’t currently do – such as saving or printing the image on a black/white board it should be obvious that you want to see whether people will actually do this enough to justify the cost. Ideally you want to see someone else spend their money and try it before you do.
Pingback: L&D: Reality Overload with VR? | EPPIC - Pursuing Performance
Can we stop sounding like luddites. The main issue with tech is opportunity cost (it’s expensive), it’s proprietary nature ( making it both expensive and unfamiliar), and finally the lack of evidence on it’s comparative effectiveness (which is where this post is relavent).
As an example if you permit phones you can take pictures easily, search definitions and preform calculations. If you have to get a set of devices out to do this the opportunity cost is rather high. If the students can’t be trusted, or if they lack WiFi it is also going to be a faff.
We spent three grand a pop on Promethean whiteboards when a tablet and a projector would have cost a third of that and offered more functionality ( the ability to write without blocking the board and the ability to use marker pens while changing the overlay). We gave this up just to copy everyone else, and gain a wide suite of rarely used functions. This isn’t a tech problem though, but an administrative issue of large organisations. The same reason my new build left large pillars in the middle of classrooms.
Technology will change teaching but it will be slower, more limited and more utilising of everyday technology.
Forgot to say I am agreeing with Stan.
I don’t think anyone here is remotely Luddite.
No-one is railing against all new technology. I would hate a smart board, but use my data display regularly and have written code in order to do so more effectively. I would be in the upper end of technology using teachers.
I do however really appreciate informed criticism of new technologies, so we don’t get railroaded with junk.
BTW, the Luddites opposed technology precisely because they knew it was *more* effective, which is precisely the opposite of this discussion.
It said sounding like luddites, and I was using it in its more generic meaning, a perceived unreasonable aversion to technology or progress. Your earlier reply, as well as jamesisaylestonebulldogs, fit this description.
Stans reply and my own, was an attempt to add more precision, so that someone reading this, who did not already agree with us, would have the information necessary to understand this generally unexpected outcome.
It was a plea to refocus on more reasonable arguments, nothing more. We were in danger of falling into our own traditional teaching reality bubble, and wallowing in the curse of knowledge.
So, who of these commentators including the author actually used a highend device (HTC Vive / Oculus Rift) with an advanced learning app like for example universe sandbox 2 to learn about space?
I’m not saying VR is the next big thing for the class room. But I sense two things in this article and in the comments: You seem to a) do not know exactly what VR is and how it comes in different forms (I’m not blaming you for that, it’s still nascent) and b) you also see somewhat close minded.
I think we are open minded. We accept the possibility that tech could be helpful and we also accept the possibility that it might not be. Tech enthusiasts only see the first possibility.
Your headline is: “Virtual reality headsets are rubbish”, based on one study that only looked at the most simple form of VR which is so far away from what can be done with that technology that it shouldn’t share the same name.
That doesn’t sound very open minded to me.
Hmmm.
Your video to back up your point is the non-VR version of the app.
Here is where he tries the VR version: https://www.youtube.com/watch?v=ctCkuW4yBqg
He doesn’t really sell it well.
The desktop version he shows in your link appears to have greater functionality and is easier to control than the VR version.
As you say VR is in it’s infancy. It may result in some fantastic learning opportunities but we’re not there yet.
My worry would be that even when the price of the technology comes down, the cost of developing a quality immersive 3D environment with intelligent and thoughtful design given over to the learning experience would be prohibitive.
I’ve used the Vive. It was surprisingly enjoyable, but a year later the person who owned the kit has sold it due to the lack of decent software. If the game houses are struggling to develop for it I wonder how successful education software developers will be.
I’m more optimistic about Augmented Reality, however the costs vs effectiveness vs inspiration triangle will still need to be evaluated. Articles like this help in that process.
I used the desktop version because it easier to understand what is going on compared to watching VR experiences on a flat 2d screen. The game houses aren’t struggling to develop for VR, they are doing fine regarding their content quality. The issue is that gamers have a conservative attitude towards new forms of gaming. I don’t believe VR gaming will be a success because it doesn’t fulfill the needs of hardcore gamers. It’s not what they want.
My point is: You can’t call VR headsets rubbish for education just based on one study that only looked at the easiest, non-interactive version of this very nascent technology. Let’s see what the next generation headsets bring to the table before being so prejudgemental.
When people start blaming the users for the lack of make-up I know something’s wrong. Modern geeks aren’t conservative if the product is good.
Sorry, lack of take-up …
The headline was designed to catch peoples attention. it is unfortunately normal for media texts. Greg is openly Polemic, but the argument and the presented evidence still stands.
The point of your last post is deeply flawed. We need to decide whether to invest large sums into the technology now, or to wait until it matures. Traditionally education is an early, if incompetent, investor in technology. This study provides evidence that this could be a flawed decision.
Many of us see the potential of the technology, but the onus is on the technology to prove itself.
Your conclusion is: Computers without mouse, keyboard and internet are rubbish. So computers are rubbish. If that isn’t a deeply flawed logic, I don’t know what it is.
Here is a VR app that would be worthwhile if it cost $1 for the equipment plus app. You don the VR helmet in a noisy classroom and are transported to a virtual library where people around you study in silence, you can see your own desk and work.
Depending on their situation some might place a higher value on it.
Confused! Your analogy is inappropriate. The arguments seem to be the technology lacks evidence of efficacy. Yours appears to be that we shouldn’t judge it yet as it will develop.
The idea of Bayesian reasoning applies here, i judge the situation based on my current knowledge, and I update that decision later on when I have more information including, in this case, the likelihood of cheaper and superior technology.
The title is meant to be a bit of fun. See earlier works such as ‘concept maps are rubbish’
“We think that this behavioral task will push translational research in psychiatry. Psychiatric disorders manifest in two relevant domains: people’s minds and people’s behaviors. Nevertheless, the study of relevant behaviors in the laboratory has hardly been performed until now. For example, in anxiety disorders, avoidance of potential threats is a hallmark symptom. With the human EPM we can now quantify avoidance in the laboratory and link it to neurobiological findings.
We hope that the human EPM and other tasks that will be developed will help to better characterize relevant behavioral phenotypes in the future. Instead of questionnaires or substitutes, researchers will rely on actual human behavior. Clinical studies may include behavioral outcomes as primary endpoints in addition or instead of subjective reporting about symptoms. This said, we think with virtual reality we are approaching a new era of behavioral research in humans and the elevated plus-maze is just a beginning.”
http://blogs.biomedcentral.com/on-biology/2018/01/02/virtual-reality-brings-human-behavior-to-the-laboratory/
Same study would’ve been impossible with Smartphone-VR-Headsets.
Mattias this is a different area. Likely a much better funded one then the standard secondary school.
Reblogged this on The Echo Chamber.
Greg, you say that you should never read out or talk over text you present on a powerpoint. However, (slightly to my surprise) I don’t find this backed up by the research. Moreno and Mayer (2002) found that, where there was no animation, combining audio narration with visible text improved recall and comprehension at a significant level over just narration. Quote from their conclusion:
“when words are both presented visually and aurally, learners are able to select both pieces of information with no cognitive overload: Because visual working memory and auditory working memory work as independent processors, additional processing capacity is made available to the student when two modalities are used (Moreno & Mayer, 1999a).”
This is indeed very interesting and I’d be interested to read the paper; one I haven’t come across before. However, to refute my point would require a test of reading vs reading + narration.