A guide to spotting bad science. The guide presents twelve points to consider when evaluating scientific studies, particularly those pertaining to medical or health claims. 
1. Sensationalised headlines
2. Misinterpreted results
3. Conflicts of interest
4. Correlation and causation
5. Unsupported conclusions
6. Problems with sample size
7. Unrepresentative samples used
8. No control group used
9. No blind testing used
10. Selective reporting of data
11. Unreplicable results
12. Non-peer reviewed material
Click to enlarge

A brief detour from chemistry, branching out into science in general today. This graphic looks at the different factors that can contribute towards ‘bad’ science – it was inspired by the research I carried out for the recent aluminium chlorohydrate graphic, where many articles linked the compound to causing breast cancer, referencing scientific research which drew questionable conclusions from their results.

The vast majority of people will get their science news from online news site articles, and rarely delve into the research that the article is based on. Personally, I think it’s therefore important that people are capable of spotting bad scientific methods, or realising when articles are being economical with the conclusions drawn from research, and that’s what this graphic aims to do. Note that this is not a comprehensive overview, nor is it implied that the presence of one of the points noted automatically means that the research should be disregarded. This is merely intended to provide a rough guide to things to be alert to when either reading science articles or evaluating research.

EDIT: Updated to version 2!

EDIT 2 (April 2015): Update to version 3, taking into account a range of feedback and also sprucing up the design a little.

EDIT 3 (December 2023): Long time no update! I’ve updated this graphic to bring it in line with my current graphical style and also to make some light text edits. I’ve also adapted the final box on peer review slightly – preprints weren’t a thing when I made this graphic, but they are now, and I felt the language here needed to change slightly to reflect this.

Support Compound Interest on Patreon!

The graphic in this article is licensed under a  Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License. Want to share it elsewhere? See the site’s content usage guidelines.

Purchase in A2, A1 & A0 poster sizes here.

The second version of the graphic is also available in: Portuguese (courtesy of Marco Filipe); Russian (courtesy of orgchem.by); and Spanish, (courtesy of Carolina Jiménez at nuakas.com).

222 CommentsClose Comments


Comments are closed.