Do Scientists Believe in Luck?

I wrote previously about the results of the national ASSET survey into attitudes of men and women working in SET departments. Recently I have been looking at the results specifically for my own university. It is harder to make sense of these, since numbers in any given part of the university are relatively small and statistical significance is therefore much reduced. Nevertheless it is tempting to tease out whether biologists really do have a different culture from physical scientists or perhaps medics. I won’t spill any beans about my own university (not least because I haven’t checked the stats), but the wealth of data we have available to us makes for intriguing reading. I hope all universities which had high numbers of returns will be similarly analysing what the data shows for them locally.

One particular question asked respondents to say which activities and factors they perceived to have contributed to past career success from the list:

  • Hard work
  • Research publications
  • Work on high profile/successful research project
  • Ability to attract PhD students
  • Luck
  • Size of grant income

Everyone – strictly speaking about 98% -thought hard work was instrumental; one wonders what the other 2% thought or get up to. Of the other factors, academic men uniformly attributed a higher weight to each of them than academic women, which is slightly curious in itself (the difference was less marked for postdocs, with female postdocs actually weighting work on high profile projects higher than males); research publications were the second highest scoring for both sexes. But here I want to consider the role of luck in career trajectories. Amongst the academic staff 56% of women and 67% of men felt it had been a significant factor in career success, with corresponding figures of 65% and 66% for female and male postdocs.

Self-help books would have it we make our own luck, so it is interesting that so many scientists are willing to admit that maybe they had a bit of random help on their way, and men even more so than women (at least amongst the established academics).  I find this quite heartening, that as a group we are not too hard-boiled or arrogant to admit that our successes are not simply down to innate brilliance and hard work. One could categorise different kinds of luck. My immediate list would include:

  • Being in the right place at the right time;
  • Not throwing away the experiment that didn’t do what one expected or accidentally making a mistake in an experiment which then proved illuminating;
  • Random networking turning into future job openings or collaborations or other opportunities.

I expect readers can expand that list, and these three items probably reflect items of ‘luck’ that I think have happened to me.  The third one is the one that I suspect the self-help books would say was part of the hard work of making one’s own luck. If you don’t do the legwork of networking, of making contacts at each and every opportunity – be it at a conference, in the tea-room or waiting on a station platform – then you might be deemed to be wasting opportunities for success. Nevertheless, the frustrated might feel they’ve done all the hard graft they can as postdocs at conference poster sessions without anything transpiring to their advantage, and thus an element of luck remains.  The second item on my list you could also say was not luck pure and simple, but indicated good judgement, a thoroughness in experimental procedure or innate intuition in dealing with the unexpected.

However the first one surely is really where luck comes into play. In my own career I feel this has happened at least twice. Firstly when as a postdoc I needed to stay in the US for a third year so that my husband could finish his PhD after I’d had 2 disastrous years on a very unsuccessful project concerned with X-ray and electron diffraction of grain boundaries in metals.  Being offered a project on polymers (again using electron microscopy) was a lifeline, but hardly one at the time I expected to transform me into a career academic from a very disenchanted postdoc just wanting to leave science and become a stay-at-home mum. But that’s what happened. Later on, when back in Cambridge on a research fellowship, a job opening arose because a colleague decided to return to his home country of Israel, thereby freeing up a position. There I was, on the spot, working in the right field so that I was able to secure a lectureship at a time when they were like gold dust – and here I have been ever since. Both those turning points in my life I would say were down to luck; particularly in the first case, not a carefully worked through decision so much as a choice of necessity due to the need to find a salary and a job to give me a visa. However, it wasn’t luck that turned both opportunities into something that enables me now to say they were turning points, and that again is no doubt what those fast-selling self-help books would say. Luck is what you make of it.

Nevertheless I think it is rather wonderful that so many of us are willing to admit to luck playing a significant part in our lives and our successes. If the ASSET survey is to be taken at face value we believe luck is more important than the size of our grant income or the ability to attract PhD students. Would you have expected that?

The data for the ASSET survey can be found at here.

This entry was posted in Research, Science Culture and tagged , , , . Bookmark the permalink.

19 Responses to Do Scientists Believe in Luck?

  1. It’s a really interesting topic. Hodgkin said chance and good fortune played a part in his breakthrough – even though it all seemed to slot logically together in retrospect. But I’m a big fan of Peter Medawar on this. He said scientists always overestimate the amount of luck involved in discovery because they forget to account for the effect of bad luck in impeding progress!

  2. Pam says:

    Oh Athene, thank you for this. I could have strangled the odd Athena Lecture presenter who has simpered “anyone could have done it, all it takes is hard work”. Thousands of women (and men) scientists and engineers have worked extremely hard for no recognition, no promotion, no chance to lead things where they want to go.

  3. Steve Caplan says:

    Athene-

    Another very interesting and thought provoking commentary! I would argue that there is absolutely no question that “fate”, “luck”, being in the right place at the right time, etc., all have a bearing on the degree of academic success that each of us has achieved. My non-statistical intuition tells me, though, that some will always be successful and land on their feet no matter what cards they are dealt, while others will never be able to capitalize on the luckiest breaks ever. In summary, I would say that “luck” is more likely to play a role in the degree to which one succeeds (ie., quantitatively), but have less bearing on the actual question of success or failure (ie., qualitatively).

    Steve

  4. Mike says:

    I guess it’s extremely hard to quantify exactly how much of an impact luck has. Most appear to feel luck has been important (in a binary sense – it required a lucky event to get a desired outcome), but as has been alluded to above, this is because we find it easy to associate a successful result with some (generally positive) ‘lucky’ moment.

    I think this really downplays the (quantitatively larger) role that hard work has played to allow us to arrive at the lucky moment. Years (and years) or slogging through expensive education and low paid jobs are generally required for a single lucky event to turn into a great scientific/academic opportunity. Or, as has been put more eloquently, it’s the result of

    1% inspiration, 99% perspiration.

  5. I would agree that luck is a major factor. Of course, in Athene’s example no 2 (not throwing away the anomalous result), there is also the kind of “chance favours the prepared mind” idea. I would say most people I know in science are clear that luck is important, but also that it “interacts” with stuff like hard work (to exploit the bits of luck that do come your way) and also the judgement to pick the right project areas.

    Like Athene, I had some big bits of luck early in my career, ending up with a Faculty job before I was 26. I’d love to claim that the fact that I am still in the same job (at the same rank!) nigh-on 24 years later reflects bad luck thereafter, but I think I might have more problems selling that idea….

  6. Everyone seems to agree it’s luck PLUS hardwork that matters – I do think hardwork alone may often be insufficient as Pam says. I wonder how the 2% who don’t think hard work matters are surviving.

    Anyone like to comment on why women are less likely than men to attribute success to luck (unless this is merely is a self-confidence thing and the women are less likely to feel successful anyhow)?

  7. Steve Caplan says:

    I would venture that perhaps women, who have typically had to work harder than their male counterparts to attain successful careers, would be more inclined (and justifiably so!) to attribute their success to hard work rather than luck.

  8. Steve Caplan says:

    Athene- I did a quick survey of some of my own lab members, and came up with a few numbers:
    Gender:—————Hard work————Luck
    M————————-50%——————-50%
    F————————-95%%—————–5%
    F%———————-70%%—————–30%
    F%———————-90%%—————–10%
    F%———————-75%%—————–25%
    F%———————-70%%—————–30%
    F%———————-60%%—————–40%
    ME%——————–98%%—————–2%

  9. Cath@VWXYNot? says:

    Yep – you have to be lucky to be good, and you have to be good to be lucky!

    The scientists I’ve shared labs with have been, in general, some of the most superstitious people I know. Unlike some other fields of endeavour, success and progress in lab-based projects are not necessarily proportional to the effort expended – sometimes experiments just don’t work, for apparently no reason, and I’ve known people to wear lucky socks, perform little rituals before and during procedures, pass a “good experiment karma” object around the lab depending on who needed the luck the most that day… I guess it’s just another way to make sense of patterns of success that sometimes seem random!

    This is a large part of why I decided (during the thesis writing stage of my PhD) that I’d rather make a career out of writing about science, rather than actually doing it myself. When I started writing up, taking breaks from looking after some notoriously fiddly long-term cell culture assays to start my methods and materials section, the progress I made was suddenly proportional to the effort I expended, for the first time since undergrad… I did a postdoc anyway, just for the fun of it (and for the Canadian work permit!), but I knew pretty much in that first week of writing up that I didn’t want to have to depend on luck so much any more!

    • Cath, I guess this ties in with your recent OT post on astrology. It seems to me that the luck I was describing was the outwith-your-control sort of stuff, and wearing ‘lucky socks’ is presumably someone’s attempt to gain control over that. And I would infer it is attemping to take control that attracts scientists to this sort of thing, rather than the more passive ‘making sense of patterns’ you refer to. At least the rituals you describe here are probably a little less serious than using astrology to judge people, as your post described. That is superstition of a totally different kind.

      Steve, I can’t help noticing the gender balance of your lab is very different to anything I am familiar with!

      • Steve Caplan says:

        Athene,

        I never really thought too much about it, but being on out dept. graduate admissions committee, I would say it’s probably close to 1:1 (F:M) overall for accepted students. For some reason, our lab is considered a desirable one by a lot of students, so we are able to be highly selective, and (as I like to view myself as “gender-blind” in these matters) we simply accept those we feel are the best for our lab. In my 7 years here, I think we’ve always been at least 2/3 female. I hope that will do its part to eventually balance things out at the faculty level. I hope to post a blog over the next few weeks about my own perspective on women (and minorities) in science.

  10. Catherine says:

    Athene, it’s so encouraging that a “right place right time” moment can help even a very disenchanted postdoc turn into a career academic! There’s hope for us all…(as long as we keep working hard I suppose)..

    Cath, good point about progress in proportion to effort. Nearly everyone I know who has left the bench has done it for this reason. I guess we have to take luck into account when the outcome of each new experiment can’t be known, and a perfectly plausible hypothesis could either fly or be trashed at any moment…

  11. Kirsty Flower says:

    Really interesting blog post, made me
    remember a talk I saw given by Prof Richard
    Wiseman, a psychcologist, writer and magician – you probably know him!
    Forgive me if I don ’t get this exactly right, but
    as memory serves he talked about a study in
    which self professed “lucky” or “unlucky” people
    were asked to count the number of
    advertisments in a newspaper. There was a
    large notice in the paper saying
    “ congratulations! Tell the investigator you have
    noticed this and you will receive £100″ (or
    something to that effect). It was found that the
    “ lucky” people noticed and attempted to claim
    the money more often than the “unlucky”
    people. So perhaps successful scientists notice
    the opportunities more often and this leads
    them to believe luck has played a part in their
    career progression?
    Having said that, I believe I have been incredibly
    lucky so far (I am just reaching the end of my
    phd) and live in fear of the day my luck runs out
    and I get found out!!!

    • Kirsty, I haven’t heard that story before, but it sounds like an interesting study. I am sure luck ties into having confidence – confidence to seize the opportunities and not just assume (as maybe an ‘unlucky’ person would) that an offer of £100 was a scam and not worth following up on. I’m in the process of working up a posting on confidence, so watch this space…..

      In the meantime, your last statement would indicate you suffer from the impostor syndrome. Waiting to be ‘found out’ is a sure sign of that. I discussed this in the article (attached to my blog) Science and Gender – Obstacles and Interventions on p10. I suspect almost every scientist, successful or otherwise, suffers from this (probably every academic whatever their discipline) but not everyone is willing to say so! Anyhow, hope the luck holds up for you.

      • Stephen says:

        If it’s any consolation Kirsty (though I fear it may have the opposite effect), I was speaking to a Nobel laureate last year who expressed precisely the same fear of being found out… So, I’m afraid it looks like that — for some — that horror may never go away. ;-)

        • The more I learn about impostor syndrome, the more I wonder if it’s like stage fright for actors – if you don’t suffer from it then maybe you aren’t driven/ aren’t going to succeed.Your anecdote about the Nobel Prize winner, Stephen, would point in that direction.

  12. Stephen says:

    I don’t believe in the kind of luck that drives people to use a particular pipettor or to assign one pair of underwear as especially fortunate.

    But I recognise your three main sources of good fortune, which are probably a necessary ingredient of any successful career. As someone who has worked in crystallography for a few years, where the outcome of many months of effort can turn on a crystallisation experiment over which you have little control, I have realised that — not through conscious choice — I have become a professional gambler. I know chance plays a part in the outcome of many experiments (as others above have noted), but we feel it acutely in crystallography.

    There is no remedy for the problem of fortune and of course hard work and a ‘prepared mind’ are necessary ingredients for making it in science, but it is salutary to be reminded that luck has a role to play too. Although it seems to be a pretty common experience among scientists, I think the recognition that we are obliged to be risk-takers should be more openly acknowledged in funding and reward systems. Perhaps if more scientists were more open about the lucky breaks littering their back-stories, there would be a fewer students and postdocs ground down by a sense of inadequacy.

  13. Steve Caplan says:

    In following the interesting conversation here, there is a little story that I feel compelled to relate about luck vs hard work (preparedness) in science. Some years ago in the midst of my postdoctoral studies, we were interested in a specific protein (that I continue to work with today!), and hypothesized that it might interact with another protein. It was a reasonable hypothesis for various reasons, and I set up an assay using “my” protein as bait to “pull-down” the other protein (or so I hoped) from bovine brain cytosol.

    We purchased a known antibody that recognized the second protein, and were ready to start. Every time I did the expt., the antibody recognized a beautifully clean band of about 120 kD in size. However, protein B was only 75 kD in size. There were no known splice variants and no way to explain this anomaly. And if tested just on lysates, the antibody would beautifully recognize the 75 kD band as expected.

    I could have easily “given up” and said “non-specific” and gone on to do other things. But I was able to purify enough of this large 120 kD band to eventually identify it–and it turned out to be a far more interesting and fitting “hit” than our originally hypothesized “protein B”. This turned out to be my first publication when I moved to Nebraska and was a wonderful start for me at my new position. Eventually, I realized that there was a very small overlap in the sequence that allowed the antibody to recognize the new protein, but only when it was highly concentrated.

    So, without trying to boast–is this luck, making your own luck or hard work and perseverance?
    I don’t know. I once heard the great Judah Volkman give a seminar and tell how a postdoc worked for 7 years in his lab to isolate an angiogenic factor. He himself noted: “There is a very fine line between perseverance/determination and pigheadedness”. Had the postdoc not succeeded, it would have been pigheadedness.

  14. Michael says:

    Great read! Wonderful insight into the scientific community and the consensus on “luck.” I just had to use this material as reference into my paper (yes, properly cited). I was…astonished when looking at the numbers. Thanks for everyone who replied too!