Question: what type of bad science do you have to deal with?

David Nunan answered on 12 Nov 2015:

Hey FizzyIzzy
You’ve asked a great question! There are lots of areas of bad science that I deal with but they basically fall into a number of categories:

1. Bad study design – this is when a study doesn’t use a fair method for testing if something works or not. An example is where a drug company might want to test a new drug. They will usually do this against a placebo (so a sugar pill) and not against the best available treatment (which is what they should test against). This is because testing against the placebo is more likely to show a much bigger difference in favour of their new drug.

2. Bad statistics – so this is basically playing loose and fast with the numbers to make things look better than they really are. An example: let’s say you have a disease that kills 2 out of 100 people who get it, so the risk of dying is 2/100 = 2%. A new drug is shown to decrease the number who die to 1 out of 100, so now only 1% die and 99% survive. This can be reported as a 50% reduction in the number of deaths due to the new drug. But the actual number of people saved is 1, and this is therefore a 1% reduction in the number of people who die. Not so impressive as 50% is it! And this is why drug companies often report the bigger number and often don’t provide the information to allow you to calculate the smaller, more important number.

3. Bad reporting (from the scientists): this is where the scientists doing a study decide to not to report findings because they don’t show what they want. Or where they don’t tell you enough about what they did for you to be able to do the study yourself (so you can replicate the findings which is one of the principles of science).

4. Bad reporting (from the media) – this is when the media misrepresent the findings from a study or present something that doesn’t actually have much evidence to support it. Examples of these include a new weight loss method which involves injecting the urine of pregnant women into your body (I’m not joking – look you can see here: http://www.theglobeandmail.com/life/the-hot-button/people-injecting-urine-to-lose-weight/article4081081/)
Even today there was a story in the news that reported on a study (poorly designed) in rats and monkeys but made the headline look like it was related to humans! (I tweeted about why it was a bad headline)