Why we should not rely on testimonials 

If I were setting up a website to sell alternative medicines then I would make sure that I packed it full of testimonials. As the doctors over at Quackwatch explain: 

“Most people who think they have been helped by an unorthodox method enjoy sharing their success stories with their friends. People who give such testimonials are usually motivated by a sincere wish to help their fellow humans. Rarely do they realize how difficult it is to evaluate a “health” product on the basis of personal experience… the average person who feels better after taking a product will not be able to rule out coincidence (spontaneous remission)—or the placebo effect (feeling better because he thinks he has taken a positive step). Since we tend to believe what others tell us of personal experiences, testimonials can be powerful persuaders. Despite their unreliability, they are the cornerstone of the quack’s success.”

So testimonials are highly persuasive. But they are also potentially misleading and anecdotal. They do not substitute for proper empirical evidence.

We should be particularly wary of the use of testimonials to promote teaching methods. Teaching is applied psychology and our own subjective experiences of it are likely to be highly biased. The empirical evidence in education is flawed and incomplete but we should not make claims that ignore or dismiss it in favour of personal testimony.

This duty is particularly beholden on those who seek to change the way that we teach. Grand claims that teaching needs a revolution – the kinds of claims made by the 21st century skills movement – need to be grounded in evidence not rationalisation. We need to know that ‘entrepreneurship’ is a thing that can be taught and that teaching it leads to positive, measurable outcomes. Otherwise it’s just like homeopathy.

Dan Meyer is well-known in the field of maths education. He has made strong claims – notably in a popular TED talk – that we need to change the way that we teach maths. I have criticised these claims as lacking evidence and Meyer has disputed this

Meyer recently wrote an interesting blog post in which he took a few swipes at his critics whilst valorising personal testimonials. He described his own blog:

“But I had to testify. That’s what this has always been – a testimonial – where by “this” I mean this blog, these tasks, and my career in math education to date.”

I am afraid that those who seek to influence educators need to do much better than this. Education is starting to change. If you can’t point to the evidence – if you just want to supply a testimonial – then why should we listen?

Advertisements

4 Comments on “Why we should not rely on testimonials ”

  1. jennifer stephenson says:

    Too true – for a medical take on the power of stories see this “the power of stories over statistics” doi: 10.1136/bmj.327.7429.1424

  2. Tara Houle says:

    Products that have the best staying power are promoted by those who see the benefits firsthand. Well that, and having the evidence to support its findings. How many people died of the popular and trendy fad of bloodletting not so long ago?

    All professions require high standards backed by strict quantitative empirical data before any changes are made to their system. Quackery and faddish trends should not be tolerated, especially when children are involved.

  3. Chester Draws says:

    I don’t really think Dan is intending to emphasize the power of anecdote. I found his recent blog post quite confusing, but what I think he is stressing is passion.

    He is passionate about one aspect of teaching, whereas others are passionate about other aspects.

    Personally I find passion to be quite as dangerous as anecdote in finding what works.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s