Bem’s psychic study cast into doubt as it fails replication attempts

Controversial Psychic Ability Claim Doesn’t Hold up in New Experiments | LiveScience.

A new study has failed to find evidence that psychic ability is real.

Skeptics may scoff at the finding as obvious, but the research is important because it refutes a study published in a psychological journal last year that claimed to find evidence of extrasensory perception. That research, conducted by Daryl Bem of Cornell University,triggered outrage in the psychological community when the Journal of Personality and Social Psychology announced in 2010 that the paper had been accepted for publication. Psychologists immediately leapt on Bem’s statistics and methods, finding reasons how he may have come up with the unbelievable results.

The researchers chose the strongest of the eight positive findings that Bem originally published.

[Stuart] Ritchie, University of Hertfordshire psychologist Richard Wiseman and University of London psychologist Christopher French all conducted the experiment separately at their respective universities with 50 participants each.

The results were clear.

“We found nothing,” Ritchie said.

Tip: @RichardWiseman on Twitter

Bem suggested that because the researchers are skeptical, they “influenced their participants not to display any clairvoyance”. That’s an example of “special pleading” and it’s a cop-out. It also suggests that ESP is non-scientifically testable. And, that just throws cold water on the whole idea, making Bem’s study also meaningless.

But, as noted by the researchers, Bem’s study prompted a closer look at more than just ESP. Journals were apt to public the initial studies but NOT the replications, even those that failed casting doubts on the original. That’s a critical aspect of science. Thankfully, with the help of the internet, a registry for replication was set up for this study which provides a documented source to supplement the original works. This has changed psychology, hopefully, all of science, for the better.

UPDATE: A piece in The Guardian from Chris French explains more, particularly with regards to the problems of publishing replications:

Given that the Journal of Personality and Social Psychology was responsible for publishing the controversial claims in the first and that Bem’s paper included an explicit appeal to other psychologists to attempt replications, we figured that this journal was the obvious choice to target in terms of publishing our own findings. The editor of the journal, however, did not agree and rejected our paper without even sending it out for peer review on the grounds that his journal “does not publish replications”.

We then submitted it to Science Brevia and received the same response. The same thing happened when we submitted it to Psychological Science. Our failure to even get our paper sent for peer review became something of a story in itself.

It is highly suspected that Bem himself was selected as a peer reviewer for one journal, causing a conflict of interest and ultimately resulting in rejection. What a mess. But, it IS finally published and it was an illustrative example of a problem that needs to be addressed.

Update (16-Mar-2012): Here is another thoughtful response to this larger issue. Long but has many points in it.

Also, I will agree that some interpretations have been too optimistic in declaring that this disproves Bem’s research or psi in general. It doesn’t do that; as is the title of this post, it casts doubt. Psi should be in doubt. The results are still arguable.

  1 comment for “Bem’s psychic study cast into doubt as it fails replication attempts

  1. John
    March 20, 2012 at 11:32 PM

    By my reckoning there are now 8/9 failed replication, excluding the online ones that failed, and only 1 or maybe 2 successful replications:

    Failures (8/9):

    Moulton (2004), experiment 6.
    Galak and Nelson (2010), experiment 8.
    Snodgrass (2010), experiment 9.
    Robinson (2011), experiment 8 or 9.
    Galak et al (2012), experiments 8 and 9 (8 separate experiments, n=820).
    Ritchie (2012), experiment 9.
    Wiseman (2012), experiment 9.
    French (2012), experiment 9.
    Wagenmakers et al (in press?), experiment 1 (rumor says it failed).

    Successes (1/2):

    Batthyany (2010), experiments 5 and 7 (possibly not an exact replication as done in 2006).
    Franklin (2012), similar experiment but not a replication of any of Bem’s experiments.

    There is also the analysis in Francis (2012) showing statistical evidence of publication bias in Bem’s experiments.

Leave a Reply

Your email address will not be published. Required fields are marked *