This is the year that AI in science first began to scare me.
(Disclaimer: I’m not talking about machine learning, which is a useful and powerful scientific tool. I’m using the term AI here as shorthand for Large Language Models.)
It’s been building for a while, of course. Starting as a quirky curiosity, AI soon ramped up to a minor annoyance: a cringeworthy ChatGPT approach letter here, a too-polished student essay there. On the one hand it made marking student work less frustrating – the preternaturally perfect grammar being something I’ve never witnessed in all my decades of undergraduate evaluation. But the novelty soon wore off, to the point that, these days, when I stumble over a rare essay with a comma splice or the lack of subject/verb agreement, it now approximates the pleasure of listening to an authentic vinyl album with its faint crackle of dusty audio blemishes.
This is all just cosmetic stuff, nothing existential. Recently, however, it’s begun to snowball: a search engine result summary containing misleading or false scientific information. An author response to peer review clearly generated by LLM prompts, resulting in nonsense replies. Entire submitted manuscripts so heavily steeped in the queasy tone of AI – half used-car salesman, half sycophantic cheerleader, alongside its entourage of conspicuous Oxford commas and M-dashes – that you no longer trust the data behind it.
You can almost imagine a day when the grants will be written and reviewed by AI; the papers will be written, peer reviewed and rebutted by AI, and selected for publication by AI editors – at which point, science will eat its own tail. Maybe they’ll still need actual people to pipette the small amounts of clear fluids from one tube to another in a physical lab, but liquid handling robots already exist and could soon render the entire process human-free.
For me, it is grant writing that’s the most endangered: the burning crucible where research questions and testable hypothesis are born in the fire of the mind. I have now started to witness the ultimate betrayal: a few ride-or-die wordsmith colleagues who previously revered the process of writing as strongly as I did, and executed it so well, now turning relentlessly to Claude and CoPilot to “refine their thoughts”. It’s a lonely feeling, thinking I might be the last scientist on earth who composes her own sentences, and finds it enjoyable. Of course I might be left behind in the process, dying on a hill that’s already been conceptually bulldozed and turned into an motorway – destroying a string of charming hamlets in the process.
You could argue that the keyboard did the same thing to the pen, but I disagree: the keyboard didn’t replace your thoughts; it just made it easier to get them down. (Mind you, I also still use a pen to scribble both my personal and scientific thoughts, as this process seems to mine different parts of my brain.) The use of LLM is something else entirely: the potential death of independent and original human endeavour, until everything is a regurgitated remake of last season’s Netflix hit, the Chinese whisper errors rippling until the result is a hopeless muddle of slop.
We are already seeing the first lines of defence being hastily erected. The UK Medical Research Council is now requiring live interviews before making the final decision on who should receive major grant money. Can you look the panel in the eye and persuade them that the ideas in the grant are yours, and can you extrapolate further into the future without AI prompts? It’s like a Turing test, except now we poor humans are the subjects. A friend of mine thinks that even this doesn’t go far enough, as any smart scientist could memorise and swat up on an AI-composed grant. No, they say, we will need live “grant camps” where you show up, get locked into a Faraday cage with ten key papers, and have to bash out your entire application with no internet in a limited amount of time. In theory, only the true originals and visionaries would rise to the top.
Surely you are wondering whether I ever use LLMs myself. The answer is, of course. It’s a useful tool for shortening longer text, although I always have to edit heavily to exorcise the techbro tone and fix the inevitable errors. I sometimes try to use it to find papers, but the choices are almost always wrong, whether that be because papers have been hallucinated, or just aren’t quite right. AI summaries on search engine results, while again often misleading, at least point towards useful links on occasion, sometimes in a more organised way than the traditional first page of results. But beyond that, I have not embraced the tool for anything meaningful. Maybe that is a personal failing; maybe I am actually missing a trick. I do not know, and I do not want to find out. Not yet, anyway.
In the meantime, writing that is genuinely human becomes incredibly rare and thus, increasingly precious. I recently received a thank-you gift bag from my outgoing flock of medical students; tucked in among the gifts were a few cards containing multiple paragraphs of heartfelt sentiment, written by hand – with a pen. Sure, there were smudges and a few crossings-out, but the true feelings leapt off the paper with a brilliance and humanity that is as luminously unmistakeable as it is irreplaceable.
I found my eyes filling with tears, as if these scribbles were a missive from the past.
Or a warning about the future.
