In which I ponder the strengths and limitations of my own humanity

There’s a new kid on the block in our institute: a shiny high-content screening apparatus. I don’t want to engage in any gratuitous product placement, so let’s just refer to it as the Swanky Apparatus. A market leader, this sleek number costs about as much as a house and is the showcase piece of our just-built state-of-the-art screening lab. Not only will its confocal front-end incubate and photograph your billions of cells in stacks of multiwell plates, but its brain will analyze them for you in real-time. Understandably charmed, the entire population of our building is now busy cooking up proposals for how best to entice this machine to peek into the darkest corners of our lines of research and throw up an enticing cache of biological signals out of a veritable abyss of noise.

I poked my head into the half-unpacked lab recently to admire the Swanky Apparatus, along with the massive liquid handling robot that had been brought in as its live-in valet. The head of the lab – let’s call him the Master – was only too happy to show it off.

Although it’s all very impressive, I must admit to feeling a little depressed at the thought that this machine could probably repeat my entire two and a half year stint in the lab in about ten seconds, and not even raise a silicon sweat in the process. Knowing how fed up I am with screens at the moment, my boss mercifully suggested I forego the intensive two-day training course. But my lab mates returned full of wondrous tales about the machine’s prowess – with one odd footnote.

“You can’t actually see what the cells look like,” my benchmate said. “We asked the Master how to access the photos, and he looked and us and said, _why would you want to do that?_”

And therein lies the rub. The Swanky Apparatus’s raison dêtre is all about reducing biological complexity into numbers. Its built-in software packages are all tailored to translate pixels to math, comparing the control cells to the experimental samples and working out, numerically, how they differ. And it calculates the so-called zed score – the line under which you decide that the numbers are not above background – that the genes or conditions giving rise to them are not hits. This clashes a little bit with the raison dêtre of our lab, which is interested in why cells and tissues take up their particular shapes in order to perform their functions. Of course some aspects of the appearance of cells can, to a certain extent, be represented numerically: the software can segment individual cells, and count them, and measure their sizes, and tell if they’re dead or alive, and no doubt do a lot of other common tasks very well indeed. But how well could it say that this particular phenotype or texture is a bit funny looking, subtly but reproducibly? Much less tell you what that means?

I have to admit that, at this point, I would kiss anything that came along and decided on my zed score for me – even the Swanky Apparatus. I’ve gone through my visual screen exhaustively, and our talented research assistant has gone through a replicate screen even more exhaustively, and we are currently fighting it out trying to agree on a Venn diagram consensus based on what our eyes and brains can see. It’s all quite messy, and a bit subjective, and very, very exhausting, but from this murk of human observations, refined over a long period of time, patterns are slowly starting to crystallize. And to me they feel a lot more believable than a column of numbers.

The scary thing is, though, that I don’t know if this new upstart robot, given the same task, would give substantially different answers. Even more scarily, if it did, I don’t know how you could tell which one of us was “right”.

About Jennifer Rohn

Scientist, novelist, rock chick
This entry was posted in Uncategorized. Bookmark the permalink.

64 Responses to In which I ponder the strengths and limitations of my own humanity

  1. Richard P. Grant says:

    bq. “You can’t actually see what the cells look like,”
    No photos? This cynical old cell biologist is nonplussed. Here, have this to make up for it:

  2. Richard P. Grant says:
  3. Jennifer Rohn says:

    Just had a disturbing image of the Swanky Apparatus making a pass at the liquid handling robot: “Would you like to inspect my zed scores?”
    Because I stare at my images so much, sometimes I dream cells, and in my dreams there are all sorts of meanings that don’t make any sense when I wake up.
    Also, the floor of the ladies’ loo in our institute has a distinctly cell-like pattern which is really hypnotic.

  4. Richard P. Grant says:

    And then the L-H-R squirts its liquids all over—no, sorry. Not working for me.
    Isn’t that kind of the definition of a dream, though? Like being chased through a field of spaghetti hoops by a giant crocs-wearing rabbit?

  5. Jennifer Rohn says:

    I was just wondering if you put the same plate through the Swanky Apparatus, would it produce exactly the same numbers every time? Is there anything stochastic about what it ‘sees’ and analyzes?

  6. Alejandro Correa says:

    Jenny – you have that prepare several times the sample of assay of validity and error, before enter confidence with your robot friend.
    The scary thing is, though, that I don’t know if this new upstart robot, given the same task, would give substantially different answers. Even more scarily, if it did, I don’t know how you could tell which one of us was “right”
    I hope you have fortune.

  7. Jennifer Rohn says:

    Yes, actually I still have all my old plates. I wonder if I could run them through the Apparatus and see if it gave the same zed – but then, I’m almost positive there is no built-in programme for the parameters I would like to measure.

  8. Åsa Karlström says:

    Jenny, that is interesting. I mean, I can understand why you can’t access the photos. Although, I would like to be able to see said photos “just to check”. It might not sound to clever but at least it would be easier to confirm there isn’t any funny going on with the cells. Or maybe would the Wonderful Apparatus pick up potential contamination in the plates?
    (I think my feelings after the screen I did with phage display where now you can fish around in the genome databank would maybe similar to what you are experiencing? In any event, it hurt my little heart when the response to one of the abstracts I had written came back with “why would anyone do this obsolete method of actually checking for binding when there is genome sequenceing done and present online” [answer: it wasn't before?! ;) ]
    Richard> that is one nice looking heart :)

  9. Jennifer Rohn says:

    Asa, you can actually access the images with a few manipulations (and I wrote an ImageJ macro to help unpack and name them intelligently). It’s just sort of odd that it’s not a standard feature – and yes, it does make me wonder what people might miss. The bacterial contamination is a great example.

  10. Richard P. Grant says:

    Oh, another one. Places like J Cell Biol pretty much make it a condition of publishing that you have pretty pictures.
    Maybe that’s not quite true. I make it a condition of reading that an article has pretty pictures. But seriously, a picture shows what’s happening in a more intuitive way than a column of numbers. It’s easier to grok pictures; it’s how we evolved.

  11. Jennifer Rohn says:

    I do think there will come a time when everything about a cell can be reduced to numbers. I just don’t think we’re quite there yet.

  12. Richard P. Grant says:

    Oh sure; but my point is that we won’t understand it. Not intuitively. Not, if you like, artistically (despite what the Marquis of Satay might have said).

  13. Åsa Karlström says:

    Jenny> I would think that many things can be changed into “numbers” and parameters are easily comparable. Although, I always end up as the skeptic wondering if it will really show “all those nuances” that messes up the “visual reading with no numbers” ;) I guess that is why I am still counting my bacteria by hand and not on a “computerized counter”?

  14. Austin Elliott says:

    Speaking as a grouchy old live cell calcium imaging type from the Ark (well, the early 90s, anyway) I am always rather wary of techniques and machines where people don’t necessarily have to understand how the raw images generate the processed outputs, or ever look at the raw images. Perhaps that’s just my age and curmudgeonliness, but I’m with Jenny that one would hope people eyeballed the raw images at least some of time to make sure what they were seeing in the processed o/puts was really there.
    Let me risk calling down a storm of scorn, given that this is _NN_and thus awash with highly go-faster striped cell/mol biol Ninjas. I am tempted to opine that people brought up in the Age of Swanky Apparatuses (rather than the days of cobbling the bits of the Apparatus together yourself) and with the training of Mol Biol (where various kinds of Swanky Apparatuses tend to stand around on benches generating print-outs or on-screen data in the manner of CSI)) are perhaps more prone to accept such things from Swanky Apparatuses without wondering what is going on under the hood.
    PS Quite like “The Swanky Apparatus” as a name. We christened the first calcium imaging system we built The Orgasmatron, and I’m sure there are others names out there too.
    PPS I’m now quite curious to know it’s real identity so that I can ask my high-content biology friend about it.

  15. Benoit Bruneau says:

    One of my colleagues published a great S2 RNAi screen on lipid droplets a year and a half ago. They let the machine find the odd-patterned droplets, then they did a visual screen. The visual screen picked out most of the good mutants. Perhaps screen-specific, but for patterns, the eye/brain is best.

  16. Jennifer Rohn says:

    Austin, you can tell your high-content biology friend that the Swanky Apparatus and its suite of software have a cheesy musical theme. S/he’ll know exactly what it is if you say that.
    Of automated image analysis, I think if the algorithms are developed in parallel in collaboration between biologists and computational types, they can be really fruitful. I am more dubious about using off-the-shelf programmes to try to analyze something that’s not gone through the proper training. I have collaborators who even say that once you’ve performed your screen, it’s too late to develop algorithms – the screen and the algorithms have to be developed in parallel (which doesn’t bode well for me, I’m afraid.)
    Benoit, I’m pleased to hear that the eye is still winning on some fronts. I’d love to count lipid droplets – sounds a lot more concrete than my current parameters!

  17. Alejandro Correa says:

    Jenny – Yes, actually I still have all my old plates……
    I would like to know the machine well and would have my free humble assistance. I like your pseudonym Jenny is sweet, is more you’re sweet, my sweet baby Jenny.

  18. Richard P. Grant says:

    I have collaborators who even say that once you’ve performed your screen, it’s too late to develop algorithms – the screen and the algorithms have to be developed in parallel
    that’s interesting—what exactly do they mean by that? You have to train as you screen? I’d have thought you’d want to show the algorithm the data and ask ‘what’s the answer?’ rather than train it on the data set you’re analysing.

  19. Henry Gee says:

    @ Jenny: The Swanky Apparatus’s raison dêtre … hmmm. Life imitates art.
    @ Richard: Isn’t that kind of the definition of a dream, though? Like being chased through a field of spaghetti hoops by a giant crocs-wearing rabbit?
    There are some places where that’s not a dream. It’s reality.

  20. Jennifer Rohn says:

    It makes sense to plan your experiment around what you’ll eventually be analyzing. The computer guys might ask you to have your cells at a certain density, or make sure your exposures are set to a certain brightness, or might advise you that a particular time resolution might be more optimal, because only they know what problems needs to be overcome computationally. Apparently.

  21. Richard P. Grant says:

    Ah, right, gotcha. That makes a lot more sense.

  22. Alejandro Correa says:

    Jenny – left me worried about the weekend, was able to operate that machine?

  23. Jennifer Rohn says:

    Alejandro, I’m not using the machine at all at the moment. I am only thinking about it. For the paper we are soon to publish, the main point will be describing the visual screen, possibly compared to a standard Cell Profiler automated analysis of the preexisting tiff files (if that gives us anything remotely useful). Because there isn’t much time left, I would be reluctant to do any more screens at this stage. What I want to do is focus on a few hits and really do some good biology on them.

  24. Åsa Karlström says:

    Jenny> Although I would be interested in knowing the difference between the output from the Swanky machine and you’re own screens, I’d be too scared of doing the check before sending in the manuscript. Why? Because I am perfectly sure that the output will be different just because normally they’d be that with machine vs person
    afterwards though, it might be a fun thing to do? ;)

  25. Jennifer Rohn says:

    Definitely. Actually, I’m sort of off the hook because the plates I used have a weird border that means that Swanky Machine’s stage couldn’t access a few of the rows!

  26. Åsa Karlström says:

    Jenny> that’s the best. “not possible due to techincal matter”

  27. Jennifer Rohn says:

    Of course it wouldn’t stop certain referees from insisting that I fork out £10,000 more to buy more siRNAs and replate them all.
    Complete digression: were referees like that abused in childhood? Are they just misunderstood and wanting to be loved?

  28. Alejandro Correa says:

    Jenny – I would like to know the machine well and would have my free humble assistance….
    Sorry I was wrong. Was my sweet Jenny. I hope you don’t get angry. Is just a taste of affectionately.

  29. Alejandro Correa says:

    Friends!

  30. Richard Wintle says:

    The other irritating thing about Swanky Bits of Kit(TM) is that they tend to (a) come with annual service contracts that are in excess of a reasonable lab-bod’s salary, and (b) cost masses of money to fix if said service contract isn’t renewed.
    You, on the other hand, can probably be fixed on the NHS, no? Advantage: Jenny.

  31. Jennifer Rohn says:

    I suspect that some of my parts are not user-serviceable.
    You’ve got me curious now, Richard. I’m going to ask what the service contract comes to – it never occurred to me that after buying something that costs as much as a house, your financial troubles were only beginning.

  32. Henry Gee says:

    Alejandro...
    <a href="http://www.flickr.com/photos/27848370
    N04/4256421257/” rel=”nofollow”>

  33. Jennifer Rohn says:

    Were any superglue or staples involved in the making of that photo?

  34. Henry Gee says:

    No. But cats’ claws have similar adhesive effects.

  35. Jennifer Rohn says:

    This just in: one of my labmates just got back from his first experiment with the Swanky Apparatus – and nothing happened. The machine is apparently refusing to recognize that we have a license agreement with it and so all experiments are canceled until further notice, when the software engineer can come to fix it.

  36. Richard P. Grant says:

    howls of derisive laughter.
    Back to the confocal, then?

  37. Nicolas Fanget says:

    @Jenny: interesting that your lab bought a house, but the estate agent didn’t leave you a key?

  38. Jennifer Rohn says:

    Ha ha. Maybe it has a diseased dongle!
    Actually, he’s gone back to our trusty Nikon – it’s about 8 years old, three different systems cobbled together and running MetaMorph for automated acquisition. I just helped him set it up and it’s well underway, purring like a kitten. And yes: you can see the pictures as they’re taken.

  39. Eva Amsen says:

    The main problem with servicing swanky apparati is that after X years (where 0

  40. Eva Amsen says:

    And you can’t upgrade to Swankier Apparatus, because all your ongoing work is done on Swanky 1.0, obviously.
    But otherwise it should all work.

  41. Jennifer Rohn says:

    I feel a bit bad for poor Swanky now.

  42. Nicolas Fanget says:

    Don’t Jenny, Swanky and its diseased dongle will take the first opportunity to block your access to/destroy some data you need at the most critical moment. I do not trust any machine that comes with defective dongles!

  43. Richard P. Grant says:

    I blame the affair with the Liquid Handling Robot.
    Anyway, I’m unfeasibly pleased that you can see the images coming off. So much more satisfying.

  44. Åsa Karlström says:

    I really needed this laugh. Eva, that is so on target what will happen. It’s fun as long as you are not in the middle of it….. “what, this machine? oh, no… we stopped servicing that a long time ago. You need to upgrade. what? You have a contract on the old amchine still… well… they shouldn’t have done that since no one here knows how it opereates any longer. New machine is the thing now”

  45. Alejandro Correa says:

    Thanks Henry: No. But cats’ claws have similar adhesive effects.
    I will have take this into account. Obviously the cat is my great friend sweet Jenny.

  46. Cath Ennis says:

    Have you tried switching it off then switching it on again?
    This post brought back memories of standing next to the luminometer trying to work out if the raw numbers supported or disproved my hypothesis… a step up from waiting impatiently next to the film developer for the first glimpse of my CAT assays (renamed CATH assays in my PhD lab due to the number I managed to plough through).

  47. Austin Elliott says:

    Heh. “…diseased dongle” struck a chord. Though perhaps no-one who has owned one will be laughing. I suspect Jenny, like other users of Dongle-ed software, will have had at least one “Dongle fail” over the years, usually at extremely inconvenient times.
    I once decided to cease dealing with a particular supplier of Metamorph because, when a Dongle that was only a couple of yrs old mysteriously expired, the supplier insisted we would have to pay 100+ quid for a new one, even though we had, of course, already paid through the nose for the software. They are joined in my personal “Dongle Hell” by the competitor imaging software company whose install CD package didn’t include the Dongle verification programme for the Dongle with which they protected their software. So… when the system crashed and required major Windows surgery, after a full day’s worth of system restore and registry repairs, software re-install and fiddling, we couldn’t get back in and run experiments because the #*!#ing programme wouldn’t recognise the Dongle. At least this time the company coughed up the programme gratis, but the delay was still a major pain.
    Over the years I have defaulted to using equipment companies mainly on the basis that you actually get after-sales service – and in particular that, when something goes wrong with something they sold you, they view it as their professional responsibility to get you up and running again, rather than seeing it mostly as a chance to make some more money out of you. This has typically not been the cheapest seller/re-seller, but the saving in aggravation, cursing and lost days has been more than worth the little bit extra upfront.
    Had to laugh when Jenny said the “cobbled together” Frankenstein-system was bazzing away doing the business while the Mighty Wurlitzer, sorry, the Swanky Apparatus, was having its Prima Donna moment. I reckon that bears out exactly the kind of grumpy old git things I was saying earlier.

  48. Henry Gee says:

    Have you tried switching it off then switching it on again?
    LOL

  49. Jennifer Rohn says:

    Austin, we had a “dongle moment” with MetaMorph too…three years old and stopped working, and they charged us a lot of money for a replacement. We were at their mercy because after paying thousands of pounds for MetaMorph Offline, we couldn’t very well not fix it. I thought they were bang out of order, but being a stranger in a strange land with notoriously crap customer service, who was I to say? It’s not like in America, where you can say “I can have you fired” (or sued) and they fall all over themselves to help you.

  50. Richard P. Grant says:

    being a stranger in a strange land with notoriously crap customer service, who was I to say?
    Jenny does herself a disservice. She helped me draft a letter to a certain company last week that was going to charge me for something I was not responsible for, and effectively called me a liar in the process. The email went to the company address, to their PR agency, and contained the very real possibility of also going to their founders and a journalist at the Guardian. I had also started tweeting and facebooking about it.
    I got a call back in about ten minutes; they waived the charge and credited my account with a goodwill gesture. And it’s all down to Jenny, the stranger in the strange land.

  51. Austin Elliott says:

    Heh. Nice work Jenny and Richard.
    In the non-science world, I always advise people who are being shafted by some company or agency to say they are copying the polite but offended letter to their local Councillor and MP, and that the next port of call is the local newspaper. That has sometimes done the trick.
    The most nervous I ever managed to get an imaging equipment mfr was many years ago (1990s) when we were having endless problems with a basically “development” system… after many months of frustration I wrote round all the other UK folk (not that many at the time) who I knew had the same system, suggesting we form a kind of “User Community”, compare our experiences of what the company had done/said, send the company a list of what the software didn’t do that they had promised it would etc etc. Once the company found out they initially got rather huffy, but their attitude did change noticeably… though it still didn’t fix all the bugs in their software.

  52. Jennifer Rohn says:

    Good work, Austin. I don’t understand you Brits. I’ve been in restaurants where the waiter has brought the wrong food to someone, and not only was it wrong but it was something impossible, like meat to a vegetarian or shrimp to someone who’s allergic, and these someones just mumble, Oh well, mustn’t kick up a fuss and don’t do anything about it.
    Send it back! It’s not what you ordered!

  53. Henry Gee says:

    Jenny is right. We Brits ought to complain more. Another effective strategy is the one routinely offered by Mrs Gee in such situations, and that is to overwhelm people with waves, tsunamis even, of niceness. Such that on one occasion when a bank misplaced a cheque for a five-figure sum I happened to be laundering depositing, I wrote to them very calmly laying out all the various transactions, processes and so on, each with its own customer reference number, and then saying something like ‘in an uncertain world, it is reassuring to know that one can always rely on XXX Bank to screw things up royally’. I was credited within the day, and got a rather nice bottle of champagne by way of compensation.

  54. Jennifer Rohn says:

    Just received an invitation to the official opening of the room containing the Swanky Apparatus. Apparently this room is called “The Translational Research Resource Centre”. Very cool!

  55. Austin Elliott says:

    How about a cut down version? You could call it the “Transl-ARRC” (pronounced “Tran-sill-ark”). Has a certain ring.
    Though personally I still prefer “”Wurlitzer Central.””:http://en.wikipedia.org/wiki/Wurlitzer#Theater_organs

  56. Richard P. Grant says:

    Transillark sounds vaguely Tolkienesque, and meshes well with the curse of the Rohnian Headache.

  57. Henry Gee says:

    I think Transillark sounds more space-opera than swords-and-sorcery, as in ‘Set the Coordinates for the Transillark Sector’.

  58. Alejandro Correa says:

    {Sarcastic humor of the British}
    Ha, ha, ha, ha ha ha you’re good joker, Austin. I did not expect from you. Really humor is you that you have the British

  59. Richard P. Grant says:

    Henry, I’m setting the controls for the heart of the sun.

  60. Richard Wintle says:

    Nature Network has misleadingly told me that 9 people posted comments since my last one. What it didn’t say is that they each posted five.
    Getting back to the service contract question, Jenny – it’s reasonably common that the first year of warranty would be included, and that subsequent years would cost between 7 and 10 percent of the purchase cost per year. Most robotics I’ve used were in the 7-8 percent range, but that was in Canada and they were rather tremendously less fancy than yours.
    There’s something a bit daunting about using an instrument with a $50,000 (or whatever) annual warranty on it, isn’t there? Of course, I once had a sliding robotic arm crash into the housing of a Multimek liquid handler and frying three controller circuit boards. The parts alone for that one were $12,000. Fortunately, it was under warranty…

  61. Jennifer Rohn says:

    Oooh…I heart robotic arms. Sounds like whatever was controlling the arm you mention was getting frisky with the liquid handler. Which is sort of becoming a theme in this thread.

  62. Richard Wintle says:

    Heh. It turned out to be a seized elbow joint. The arm didn’t raise up high enough to clear the Multimek housing and whacked into it, stopping the arm and burning out the three controller boards (all tucked away inside the arm) at the same time. I think the bulk of the 12 grand in parts was actually the joint.
    I was very fond of that system – Beckman/Sagian core robotics, three metre rail, Multimek, 4 plate washers, six 384-well multidrops, barcode scanner, laser box, plate reader… a real homonculous of things, but a beauty when it worked well. I really wanted to use it to serve drinks at investor functions, but never quite got around to it.
    I’ve a photo of it on my Flickr account somewhere but can’t see it from here, unfortunately (blankety-blank institutional firewall).

  63. Richard Wintle says:

    Ah. The Magic of Google(TM) reveals the photo in question, even though I can’t navigate to it at present:
    http://www.flickr.com/photos/ricardipus/3843796617/

  64. Jennifer Rohn says:

    Heh. I love the idea of your robot serving drinks at an investor function – the multidrop dispensing precise 384-well arrays of champagne. I love even more the image of them trying to sip it out of the plates.