Scientists are trained to read the scientific literature with skepticism. Forever question, we are told, the truth of various assertions put to us no matter how eloquent or famous the writer or prestigious the journal in which that writing appears. In thinking about it more closely, though, I was surprised to recall that my first lesson in skeptical reading did not actually derive from my scientific training, but from an undergraduate course in Cultural Anthropology.
I got mixed up with Cultural Anthropology because Oberlin College bestows a liberal arts degree, so its humanities students must take a number of science courses and its aspiring scientists, classes in the arts and humanities. Although my classmates groaned about this, I was in heaven, gobbling up whatever I could – Ancient Greek, ethnomusicology, Spanish poetry – I even got credits for joining the women’s Ultimate Frisbee team and learning how to play the steel drum, the Gambian kora and several teeth-grittingly atonal gong-like instruments from the Javanese gamelon tradition. But that’s another story.
The formative lesson in question was conducted by a wily professor who assigned Margaret Mead’s Coming of Age in Samoa (1928). For those of you not familiar with this work, it’s chock full of teenage sex and its central theory was that Samoan adolescence wasn’t stormy precisely because nothing, including sex, was taboo. After giving us a chance to marvel at Mead’s achievements, he then set the text Margaret Mead and Samoa: The Making and Unmaking of an Anthropological Myth by Derek Freeman, which basically claimed that Mead’s findings were biased and artifactual. So which anthropologist, we asked the Prof anxiously, was right? He shrugged and said that the truth would probably never be known, as numerous scholars had found problems in both Mead’s and Freeman’s methods, and the culture in question was long gone.
As a callow girl of nineteen, I found this whole affair rather shocking. How could something in a book be wrong?
Of course I soon learned that many printed things are wrong – and this was years before the internet taught everyone to take everything written publicly with a grain of salt. Still, I had a healthy respect for what I read in scientific papers, taking everything at face value unless I had a valid reason to believe otherwise.
But that’s all over now. I can’t decide if it’s a blessing or a curse, but suddenly I’m having problems believing a single word I read in the scientific literature. It all started a few months ago. With only a year and a half left of my fellowship, I’ve been thinking about the long game, prioritizing my activities, clearing the decks to embark on a tangential line of research that – with good luck and a following wind – I might be able to export from this lab to start my own. With a new project entails reading up in a new field – always a painful prospect when you don’t know the background or any of the lingo.
And there I was, lost in a dense review article paragraph about a pathway with dozens of proteins, with laboratory A saying protein W does P, and laboratory B claiming no, protein W does process Q, and laboratory C begging to differ, that W might bind the main effector of process P but under different conditions, W probably got co-opted to perform process R, while simultaneously initiating a positive feedback loop bolstering process P and Q. And I was clobbered: clobbered by the incredible complexity of this seemingly simple biological phenomenon; clobbered by the number of papers, many contradictory, chipping away at the problem, each revealing yet more complexity; clobbered by how bitty and insignificant each finding appeared against the larger messy advancing-and-receding scrum of raw observation, supposition, conjecture, and dissent. OK, laboratory A might have published it in Nature or Cell, but suddenly, it just seemed – in the grand scheme of things – unbelievable.
A little bit of skepticism is a good thing, but I feel like I’ve been infected with some terrible virus.
Is it terminal?
The diversity of opinion on the function of protein W is probably an indicator of the youth of the field (making it a good choice). I get suspicious when I see too many functions assigned to a particular protein – it’s just telling you that no-one really knows what is going on.
There is probably a wider problem of the degree of phenomenology that still gets reported in the literature. “We did X and Y happened. Isn’t that interesting?” With the advent of RNAi, it’s become way too easy to do this sort of thing.
But I guess you have to start somewhere. The good ‘uns are those who will stick to a problem and worry away at it. For years.
Ah, biology. So messy, so confusing! I think this is why I was always drawn to Mendelian genetics, with its lovely logical phenotypic ratios providing some kind of signal above all that noise!
I find I’m infected with scepticism too – I guess it makes me a good reviewer (as long as the editor knows my biases!).
A few years ago one (mathematical) statistician had in his sig file “It’s not the playing, it’s the taking apart that counts”.
I get suspicious when I see too many functions assigned to a particular protein – it’s just telling you that no-one really knows what is going on.
Stephen – you’ve nailed exactly the nagging worry I couldn’t quite articulate last night when I wrote the post. This is my problem: it doesn’t seem it could be that complicated. But I hadn’t appreciated the correlation between “big fat mess” and “youthful field” – and indeed I’m looking into this problem precisely because it seems understudied. So maybe I should take comfort in the fact that it seems a dog’s breakfast. And yes, RNAi may be to blame. But before RNAi, we had over-expression and dominant negatives, so it’s just got easier to make a hash of things — and more expensive.
bq. “And yes, RNAi may be to blame. But before RNAi, we had over-expression and dominant negatives, so it’s just got easier to make a hash of things — and more expensive.”
Jenny, you’ve nicely articulated why we old-fashioned physiologist/ pharmacologist types roll our eyes when we see another abbreviation- and pathway-rich mol cell biol talk…!
Stephen wrote:
Agreed, Stephen. But you’ve also reminded me why I’m not a very good scientist – never had the perseverance. Or perhaps I will call it a “low boredom threshold”, as that somehow doesn’t sound quite as bad.
Austin, that’s it exactly. I am suddenly incredibly bored with all this bitty stuff – perhaps that’s why I’m starting to lean more towards the idea of translational work; the abstract just isn’t fulfilling me any more.
I’m finding this all the time – no one really knows. A lot of things are like best guesswork. We need to go down to the details of biology i.e. the dna to figure out the code of life.
Computer scientists may be able to help with this after all they were the ones to create a language in base 2 that computers can understand – now we just to crack, hack the code that is in base 4. Although making up a language is easier than cracking one as biologists i think are finding.
I don’t think the answer is in the DNA code at all, to be honest. To much epigenetic and post-translational modification afoot to make things that simple.
Jenny, sounds to me like you might be ready for a move into Evul Big Pharma..! They obviously have their up- and their down-side but at least what they do is intrinsically “translational”, and on the whole they value sheer lab experience rather more than the academic set-up does. At least until the whole programme gets cut…
I moved into bionanotechnology for three years and as a result of seeing countless people try, and fail, to repeat the work published numerous papers, people would be hard-pressed to convince me that anything in the area is actually repeatable.
The field is also remarkable well endowed with papers on a random topic, such as fabrication of a particualr nanowire, followed up by, well, absolutely nothing. This I take to suggest that these experiments couldn’t even be repeated in the same lab, or the nanowire was a glib, fanciful exercise with little direction.
It’s scepticism meets cynicism.
(I do however exclude the likes of Belcher, Ellington, Breaker, Szostak, Liu, Lu and Yu from this, as much of their stuff was either fantastically audacious, or actually repeatable.)
Jim, there are loads of orphan papers in our field too – big lab x shows one tiny thing and then drops the topic. From this I intuit that a postdoc did a project a bit off-piste and then left the lab – or did a screen and got a not-too-interesting hit but had to publish it to show something for his time, but it wasn’t something the lab head wanted to follow up. You can’t really blame us for that. But it does give a slip-shod feel to everything.
Austin, I worked in industry for 4 years and was deliriously happy til the company went bust and laid me off. I would definitely consider returning because drug discovery truly excites me, but I am a little worried that a company might not be as pleased with me doing other stuff – being a novelist, blogging, all the other things that enrich my life currently that academia not only doesn’t mind, but positively encourages. What do you think?
Hard to say, Jenny. It certainly used to be with the companies where my friends worked that the company wanted you to do the company’s work on the company’s time, but didn’t expect you to be doing their work on your time. Certainly at a couple of places I knew people they were actively discouraged from taking work home, and also discouraged from being on the company premises outside 8 am – 6 pm. But it may have changed, of course, in these recession-pressured times.
I have come across bloggers who work for scientific companies – an example is Ian Hopkinson of the Somebeans blog, though he doesn’t write about his work particularly. I guess that some places might balk at the idea of you turning their workplace politics into grist for your next novel, though…!
Interested to hear you say that academic positively encourages the kind of “non-core” stuff. My experience as a not terribly efficient PI has been that academia encourages (or perhaps more accurately tolerates) this kind of stuff as an add-on. However, if you are perceived to be slipping on the main stuff (i.e. research grants+papers+grad students) the attitude tends to get, er, progressively less positive.
I wasn’t referring to taking work home – industry is usually 9-5 M-F for the foot soldiers as you note. It was the “what you do reflects on us and you can’t be seen as frivolous” thing I was worried about. Maybe it’s not such a problem, though. I did write my first novel when I was industry, but I didn’t tell anyone I was doing it, either.
I think academia is happy when its employees do things that are traditionally literary – like write books, and get the university’s name in the papers. UCL has been very positive about all my endeavors, even the silly stuff – but that’s UCL the big organization. I don’t know how my department feels about it – I suspect it could be more like it is for you. But you have to keep in mind that I’m “just a postdoc”.