How to quickly spot dodgy science

Cross posted from The Conversation

by Michael J. I. Brown

I haven’t got time for science, or at least not all of it. I cannot read 9,000 astrophysics papers every year. No way.

And I have little patience for bad science, which gets more media attention than it deserves. Even the bad science is overwhelming. 700 papers are retracted annually, and that’s a gross underestimate of the bad science in circulation.

I, like most scientists, filter what I read using a few tricks for quickly rejecting bad science. Each trick isn’t foolproof, but in combination they’re rather useful. They can help identify bad science in just minutes rather than hours.

Okay, this looks bad

Good science is often meticulous and somewhat anxious. You discover something new or find something unexpected, and frankly you worry a lot about screwing up. Identifying and addressing what could plausibly go wrong, and then writing that up succinctly, takes time. Lots of time. Months. Even years.

If you’re taking the time to do meticulous science, why not take the time to prepare a good manuscript? Make nice-looking figures, proofread it a couple of times, and the like. It seems obvious enough, which is why a sloppy manuscript or poor grammar can be a warning sign of bad science.

Recently, Ermanno Borra and Eric Trottier claimed to have detected “signals probably from extraterrestrial intelligence”. I thought this was far-fetched, but still worth looking at the paper preprint. An immediate red flag for me was some blurry graphs, and figures with captions that weren’t on the same page.