{"id":2448,"date":"2014-04-27T19:09:10","date_gmt":"2014-04-27T19:09:10","guid":{"rendered":"http:\/\/occamstypewriter.org\/sylviamclain\/?p=2448"},"modified":"2014-04-27T20:19:46","modified_gmt":"2014-04-27T20:19:46","slug":"spotting-bad-science-is-it-really-so-easy","status":"publish","type":"post","link":"https:\/\/occamstypewriter.org\/sylviamclain\/2014\/04\/27\/spotting-bad-science-is-it-really-so-easy\/","title":{"rendered":"Spotting bad science – is it really so easy?"},"content":{"rendered":"

On the Interwebs – I have seen several links to this helpful PDF on How to spot bad science – a rough guide<\/a>. <\/p>\n

\"\"<\/p>\n

Learning how to sniff out bad science – or really bad science reporting which is what this guide seems to be aimed at – is important. As most folks know, after cold fusion<\/a> and arsenic in DNA<\/a>, spotting when there is something that smells funny, should be part of any scientist’s or science aficionado’s tool kit. <\/p>\n

To be abundantly clear, I think detecting bad science (and there is much of it out there) is a good thing. I think, for the most part, this quick guide is also a good thing – especially with advice on being wary of ‘sensationalized headlines’<\/em> and being cynical about everything you read and believe. <\/p>\n

However, sadly, this rough guide is too simplistic in some of its advice, advice which is a bit too vanilla to be of much use when reading a scientific report. <\/p>\n

Take for example ‘unreplicable results’<\/em>. This is a big warning for bad science, clearly, but how exactly can you detect this in an NEW article? If you are reading a scientific press release it is likely a fresh set of exciting results, which no one has had the chance to try and replicate yet. Also you have to be a bit cautious between ‘unreplicable results’<\/em> and ‘results which haven’t been replicated yet’<\/em>. The former is bad science, the later may be opening up an new area of science that is somewhat outside the paradigm. Or in other words beware of the Physics-is-dead<\/a> syndrome. <\/p>\n

What about ‘misinterpreted results’<\/em>? Another big warning sign for bad science but incredibly difficult to detect, especially when reading a scientific study outside your area of interest. Misinterpretations abound in the literature, often because at the time they don’t seem like misinterpretations – it only becomes clear when some new data pops up, often well into the future. <\/p>\n

Then there is ‘conflict of interest’<\/em>. We all have conflicts of interest. It is impossible to be a human and not have a conflict of interest somewhere. The warning in the guide is probably to flag up tobacco-isn’t-that-bad-for-you arguments by the major cigarette companies but, importantly, it’s worth remembering that much good research takes place in industry. R & D laboratories which are privately funded are not all hell-bent on misleading the public. <\/p>\n

The rough guide advice I found the worst, hands down, was ‘speculative language’<\/em> where the guide states:<\/p>\n

Speculations from research are just that – speculation. Be on the look out for words such as ‘may’ ‘could’ ‘might’ and others, as it is unlikely the research provides hard evidence for any conclusions they precede.\n<\/p><\/blockquote>\n

The first tautological statement notwithstanding, Really?!? Science rarely presents exact conclusions – usually research opens up more questions than it answers. Saying ‘we see this’<\/em> and ‘we think this means that’<\/em> is perfectly acceptable in scientific literature. In fact it is the bread and butter of many research publications. I am far more dubious of a study which says ‘we see this, therefore it absolutely must mean that’<\/em>. Most scientific studies are a process of slowing marching forward (and back again and then sideways a bit, oops and then we all fell in the lake) to make new, even if they are small, discoveries. It’s important to be speculative, otherwise the whole shebang would get rather boring and dogmatic. <\/p>\n","protected":false},"excerpt":{"rendered":"

On the Interwebs – I have seen several links to this helpful PDF on How to spot bad science – a rough guide. Learning how to sniff out bad science – or really bad science reporting which is what this … Continue reading →<\/span><\/a><\/p>\n","protected":false},"author":16,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[8,44],"tags":[285,258],"class_list":["post-2448","post","type-post","status-publish","format-standard","hentry","category-bad-science","category-science-communication","tag-bad-science","tag-spotting-bad-science"],"_links":{"self":[{"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/posts\/2448","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/users\/16"}],"replies":[{"embeddable":true,"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/comments?post=2448"}],"version-history":[{"count":0,"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/posts\/2448\/revisions"}],"wp:attachment":[{"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/media?parent=2448"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/categories?post=2448"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/occamstypewriter.org\/sylviamclain\/wp-json\/wp\/v2\/tags?post=2448"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}