![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
On medical research and not being an idiot
Nov. 7th, 2013 05:28 pmI've posted a lot of versions of this. This is one of the more exhaustive, so I'm posting it here.
***
I feel like a broken record, but when it comes to getting quality information - pubmed. Your friend and mine, and US tax dollars at work. http://www.ncbi.nlm.nih.gov/pubmed/ And seriously, yes, a lot of this stuff is tougher reading that "What you don't know about vaccines can hurt you!" at naturalorganicsoitsgottobegood.com, but anyone with a decent education can at least get the general idea from the abstracts.
Other things to keep in mind: One study is pretty much never the last word on anything. If you don't want to be an idiot (or a popular science reporter*) keep in mind that science is a conversation, and you need more than a single sound bit. I recommend adding "review" to your search terms to see if you can pull up a few overviews of what's been done - unless you've specifically being following the area in the primary literature, that's going to be far more helpful than looking at individual studies at first. Once you've read a couple of reviews (or maybe hit google books to get reference book background) then the individual studies will make more sense.
Google and Google Scholar are also useful, though in my experience more so for tracking down copies of articles if you don't have institutional access. (More and more is available, much of it through pubmed because anyone getting NIH funding is legally obligated to make their findings publically available within a year of publication. Also, people are noticing that publically available articles get cited more, which makes it an easier sell.)
Keep in mind who paid for the research. Also look at where it was published and get an idea of the journal's impact factor. There are insta-journals out there. People actually in the field know to avoid them, mostly, but they can be confusing if you aren't. Some of these are sock puppets of big pharma. Some of them merely exist to get dubious research published - but it's not that hard to sort this stuff out, really. Put the name of the journal into wikipedia FFS.
In terms of non-academic sources... Look. It's becoming more and more obvious that peer review, while really important, isn't enough. It's possible to get a lot of crap published. The coralary is that if someone can't get their crap published it is probably really, serious crap. (The whole narrative of the maverick scientist who has their finger on The Truth but the rest of the community wants to keep them down is... well, better movie fodder than reality. Yeah, there is the usual backbiting and personality conflicts, but for the most part there are always scientists who really want to help overturn the established order, because it's just so much more fun. At worst, you submit to a different journal and ask that so and so not be one of your reviewers.)
When you see claims made in random non-peer reviewed sources (say naturalorganicsoitsgottobegood.com, but also your local newpaper, or whatever) even if they say they are citing scientific studies, look up the actual studies before you make any health decisions based on them. Because there's really an embarrassing amount of crap out there in the popular press. Sometimes there are studies, and that's not what they said. Sometimes there aren't studies. Sometimes someone put up a web page claiming that they did a study, but it couldn't get through peer review and if it annoyed enough people was broadly panned but not before it got loose on the internets. Most often, there's one study, that reporting one tiny thing that was maybe surprising or maybe not, and it was reported in the popular press as if it overturned everything else in the field... because that sells papers, and gets eyeballs, even if that's really, obviously not what's going on.
* Yes there are some good ones - I follow some great ones, even - but OMFG. If I hear one more person saying "the chemical imbalance model of mental illness isn't correct therefore there's no evidence that antidepressants work" I will... well, probably right another long post about the neurobiological literature. I really need to just put together a few blog posts on the most common ones.
***
I feel like a broken record, but when it comes to getting quality information - pubmed. Your friend and mine, and US tax dollars at work. http://www.ncbi.nlm.nih.gov/pubmed/ And seriously, yes, a lot of this stuff is tougher reading that "What you don't know about vaccines
Other things to keep in mind: One study is pretty much never the last word on anything. If you don't want to be an idiot (or a popular science reporter*) keep in mind that science is a conversation, and you need more than a single sound bit. I recommend adding "review" to your search terms to see if you can pull up a few overviews of what's been done - unless you've specifically being following the area in the primary literature, that's going to be far more helpful than looking at individual studies at first. Once you've read a couple of reviews (or maybe hit google books to get reference book background) then the individual studies will make more sense.
Google and Google Scholar are also useful, though in my experience more so for tracking down copies of articles if you don't have institutional access. (More and more is available, much of it through pubmed because anyone getting NIH funding is legally obligated to make their findings publically available within a year of publication. Also, people are noticing that publically available articles get cited more, which makes it an easier sell.)
Keep in mind who paid for the research. Also look at where it was published and get an idea of the journal's impact factor. There are insta-journals out there. People actually in the field know to avoid them, mostly, but they can be confusing if you aren't. Some of these are sock puppets of big pharma. Some of them merely exist to get dubious research published - but it's not that hard to sort this stuff out, really. Put the name of the journal into wikipedia FFS.
In terms of non-academic sources... Look. It's becoming more and more obvious that peer review, while really important, isn't enough. It's possible to get a lot of crap published. The coralary is that if someone can't get their crap published it is probably really, serious crap. (The whole narrative of the maverick scientist who has their finger on The Truth but the rest of the community wants to keep them down is... well, better movie fodder than reality. Yeah, there is the usual backbiting and personality conflicts, but for the most part there are always scientists who really want to help overturn the established order, because it's just so much more fun. At worst, you submit to a different journal and ask that so and so not be one of your reviewers.)
When you see claims made in random non-peer reviewed sources (say naturalorganicsoitsgottobegood.com, but also your local newpaper, or whatever) even if they say they are citing scientific studies, look up the actual studies before you make any health decisions based on them. Because there's really an embarrassing amount of crap out there in the popular press. Sometimes there are studies, and that's not what they said. Sometimes there aren't studies. Sometimes someone put up a web page claiming that they did a study, but it couldn't get through peer review and if it annoyed enough people was broadly panned but not before it got loose on the internets. Most often, there's one study, that reporting one tiny thing that was maybe surprising or maybe not, and it was reported in the popular press as if it overturned everything else in the field... because that sells papers, and gets eyeballs, even if that's really, obviously not what's going on.
* Yes there are some good ones - I follow some great ones, even - but OMFG. If I hear one more person saying "the chemical imbalance model of mental illness isn't correct therefore there's no evidence that antidepressants work" I will... well, probably right another long post about the neurobiological literature. I really need to just put together a few blog posts on the most common ones.
