The slums of scientific peer review

Take a gander at this. An investigator for Science mag, John Bohannon, pulls a sting operation to investigate predatory open-access journals. What he finds probably will not surprise you but it reveals a zone of rot in scientific peer review and publishing.

Who’s Afraid of Peer Review?.

On 4 July, good news arrived in the inbox of Ocorrafoo Cobange, a biologist at the Wassee Institute of Medicine in Asmara. It was the official letter of acceptance for a paper he had submitted 2 months earlier to the Journal of Natural Pharmaceuticals, describing the anticancer properties of a chemical that Cobange had extracted from a lichen.

In fact, it should have been promptly rejected. Any reviewer with more than a high-school knowledge of chemistry and the ability to understand a basic data plot should have spotted the paper’s short-comings immediately. Its experiments are so hopelessly flawed that the results are meaningless.

I know because I wrote the paper. Ocorrafoo Cobange does not exist, nor does the Wassee Institute of Medicine. Over the past 10 months, I have submitted 304 versions of the wonder drug paper to open-access journals. More than half of the journals accepted the paper, failing to notice its fatal flaws. Beyond that headline result, the data from this sting operation reveal the contours of an emerging Wild West in academic publishing.

This is the opening of a story posted and freely acceptable on Science mag site. It makes for fascinating reading and PLEASE click the link above and read the whole piece before commenting on “peer review” and scientific publishing. Part of the problem is people are OK with being spoon fed nonsense and for just taking everything given at face value, assuming it is what it says it is. There is ALWAYS a background story. There is always more to it.

Bohannon created papers just slightly different from each other with fake names and fictional institutional affliation and sent them to 304 journals for publication. The result was: 157 of the journals accepted the paper, 98 had rejected it. 29 journals never responded and 20 notified the author that the paper was still under review.

This from Retraction Watch: Science reporter spoofs hundreds of open access journals with fake papers | Retraction Watch.

[Bohannon] demonstrates an appalling lack of peer review and quality control at the journals he spoofed.

Some journals were clearly in business to make money. They did not do business at their stated location (“American” Journal X) was out of China. And he even received replies from people who should CLEARLY not be editors of anything because they had no concept of grammar or the English language. But, many are happy to take a publishing fee while chastising him for pulling this prank (See this piece on deliberate hoaxing). The article also notes that open access journals, freely available to the public, are important. Researchers who publish in open access journals are cited more and more people see their work – that’s a good thing. But are their standards too lax? The argument may be that the same may happen if you did this experiment with traditional style journals.

Oddly, the study, done by Science, showed that the most prestigious open access journal, PLOS One passed the test, by timely rejecting the paper noting it’s serious flaws. PLOS One can be considered a rival to Science. You can access PLOS One pieces (and I often do and am thankful for that). A subscription or fee to purchase (usually around $35 for a single article) is required for most Science articles.

While the results were not ALL bad, it’s clear that peer review is broken and must be fixed. How? Not sure. No one is. I do hope this scandal prompts change. There are too many journals and too much confusion. There is greed and corruption. We need restructuring, with integrity.

Scientific "peer review" may be done cheaply and sub-standard, not with integrity and ethics.

Scientific “peer review” may be done cheaply and sub-standard, not with integrity and ethics.

After reading this piece, one wonders why Dr. Melba Ketchum didn’t go this route. Since there are poor quality journals out there who would publish absolutely anything, putting her Bigfoot DNA paper in one of these “peer reviewed” journals would have looked far better than publishing her own piece and then claiming it was peer reviewed. Her process shows another flaw – that horrendous science is accepted by the public as just as good as solid research. No one checked her credentials and how her paper was published. She got away with circumventing the system. She rejected reality and substituted her own. And the press SWALLOWED it.

Those of us interesting in research integrity must recognize that peer review is subjective and questionable. We must consider the source of a paper and, we can’t trust that just because it’s in print, that it’s any good. NOTHING can be taken at face value. Trust in results should only come after review and commentary by the COMMUNITY of scientists who must now be ever more diligent in picking apart others’ work. Trust no one to be on the level or to use accepted procedures. Scientific publishing is a business and whenever money is involved, a certain amount of bias and corruption can be expected.

Thanks to the many who sent me this piece. It took me a while to cover it because I wanted to read it carefully. And, I was busy yesterday trying to explain to the public that Ketchum and Erickson’s nonsense was not science and should not be taken seriously. Man, this is an uphill battle that can only make progress if actual appreciation of science is taught early in schools and widespread in society. Will that day come? I have no idea.


  5 comments for “The slums of scientific peer review

  1. spookyparadigm
    October 4, 2013 at 9:51 AM

    Here’s a question I’d like to know the answer to: how many of the people placing articles in these journals work in industry vs. academia (and particularly academia in poorer vs. wealthier countries)? I suspect the answer is obvious, but I’d like to know for sure.

  2. Brandon
    October 4, 2013 at 5:37 PM

    This reminds me of the diploma mills some cranks use to trick others into thinking they are legit experts. I bet that if Ketchum (and other pseudoscientists) knew about this flaw in the system, they would take advantage of it. Maybe they will now that the cat is out of the bag.

  3. Andrew
    October 5, 2013 at 3:45 AM

    Interesting to note, the comments on the article at the ‘Science’ site seemed to have totally missed the point of said article. It’s not so much the fact that scientists would take many of those dubious sites publications on face value and without a critical eye, but that less informed Newspaper and News Outlets can and do use those sites as a source, as well as other scientists whom may be pressed for time, needing to rely on those publications for accurately peer reviewed studies, or pseudo science using those site to give an appearance of credibility to their publications.

    Yes, it talks about the ‘peer review’ publications that are not and should be, or be better policed, but, but it does point to more of not a trust issue, that some of the publications claim is at fault, but one of lack of filters, the peer review process, being there in the first place. Not to mention the dubius rags that are nothing more than a scam, or financially driven at best.

  4. Brian
    October 5, 2013 at 9:10 AM

    “She rejected reality and substituted her own.”

    Someone is a Mythbusters fan…:-D

    I can tell you how a LOT of this can be stopped, but the small detail is the fact that greed (and hopefully, corruption) need to be stopped first. That, is an order too large to fill until various economic things need to be dealt with. So simple a concept…. but the details… *cringe*

Comments are closed.