Examining Season

Students, you may want to look away now as I’m going to give away some of the secrets of exam marking, as I’ve discovered them over more years than I care to remember.

Firstly, it is extremely boring. If you have 100 scripts to deal with in a compressed period of time, as so often, it is very hard to keep concentration going reliably. Sometimes the pile may be much larger than 100, which compounds the pain. But we all know that getting every question correctly marked is extremely important, so a wandering attention is simply not acceptable. Nevertheless by the end of a long day it is a big ask. Every examiner knows that, even with a minutely documented mark scheme to go by, no two students answer questions in exactly the same way in exactly the same order, and it is important to be sure you don’t give the same mark twice – because the correct answer appears in two places several pages apart – or not at all, because the relevant remark turns up out of context. How anyone marks an essay in the humanities I don’t pretend to understand. In physics setting a mark scheme is at least viable.

Of course there is then the problem of reading what has been written. Students don’t write much these days (unless they are penning billets doux to pass across the lecture theatre, and I suspect this too is a dying art in the days of social media). That means their handwriting is under-utilised. Writing under pressure is hard. My late-lamented mother, always so willing to take me down a peg or too, used to say I only did so well in my History O Level because no one could read what I had written. With the hard won experience of marking I am sure that is the last reason I did well (instead I’d attribute it to the excellent teaching of my school teacher, a certain Mrs Milliband, now probably better known as the mother of Ed and David); a tired examiner may just decide not to give the benefit of the doubt to the finer points of an argument (or proof) if it just so difficult to read.

For one exam, for which I was the external examiner, no one could read the candidate’s writing at all and so the student was called in to read out their answers and dictate them so they could be typed out. It turned out they couldn’t read their own writing either. So, the warning given on the front of many exam booklets to ‘write legibly’ (and I’ve just checked this year’s batch and yes, that is exactly what is said) means just what it says. Of course, with any luck, soon students will be able to use laptops for much of this, to everyone’s benefit.

Thirdly there is that moment when you realise that a significant number of students are consistently not giving the answer expected. One year, long ago, it transpired that a particular question was entirely ambiguous and so two completely different answers had to be accepted. In my department questions are checked and rechecked by those other than the setter, but still these things happen far more often than one would like. Even without ambiguity it can be disconcerting when you realise that – for instance for an algebraic answer in terms of fundamental constants and some parameters – there may be multiple ways of writing the same equation which look wildly different but are actually all equivalent. It can take me (and presumably others) a large number of scripts to realise that that answer you’ve been marking wrong all morning is simply a different manifestation of the right answer. Re-marking is an even worse penance than doing it the first time.

Fourthly, there is the checking. This always takes an extraordinary amount of time, and usually far longer than the time mentally allocated. Firstly one has to check that every page of every script has been marked: students sometimes sneak the last part of question 4 onto the tail end of question 2 in a different booklet, but it has to be ferreted out and marked and correctly totted up. The practice here is that some indication has to be put on every page to show that it has been looked at (I use a red scribble in the corner of each page, if no actual mark is entered). Then one has to check that all marks have been correctly totalled, a task that is surprisingly difficult to get right consistently. And that all the marks are correctly entered against the correct candidate. Yes I know this all sounds trivially easy but each year I find I’ve lost concentration and put things in the wrong place. And, as the ultimate check, a second examiner checks everything again. And as second examiner I know how often a handful of errors are still found in the markbook. It is mortifying when my second examiner points out my own idiocies, but it is better than candidate A getting candidate B’s marks.

Finally, there is the report to write. This needs to identify how the mark scheme was actually used in practice, since small deviations from the original plan may be required because no student picked up one aspect or many students commented on something that hadn’t originally appeared in the scheme (these things usually arise when the lecturer has shifted content around a bit from previous years). It also needs to spell out the sorts of traps that students fell into or the things they found particularly hard – or easy. All of this takes time at a moment that the exhausted examiner just wants to curl up in a chair with a nice hot cup of tea, if not anything stronger.

I don’t expect any student to feel sorry for me and my fellow examiners, but I would like them to appreciate we take our duties very seriously. I have not yet met an examiner who was careless by intent or who didn’t work flat out to complete the task in good order. I have met examiners who have failed to deliver because of ill health (vide my last post) or other unanticipated problems. On that score I can at least say the only time I ducked out of my examining duties I had given about six months’ notice. This was when the due date of my first child fell neatly around examination time. When my mother died a week before I had to mark three years ago, I firmly decided I would not opt out of my duties because the load on anyone else would be intolerable. Interestingly, that was the one year my second marker found no mistakes in the mark book, showing that my need to focus hard in distressing times absolutely had worked.

Students, if you did stick with reading this, we do try hard on your behalf to be fair, accurate and consistent!

Posted in Education | Tagged , | 1 Comment

A Crisis in Mental Health in Academia?

It will surprise no one in academia to know that it is an environment that is stressful, frequently precarious and there never seem to be enough hours in the day. The HEPI report on mental health issues in academia by Liz Morrish published last week (Pressure Vessels: The epidemic of poor mental health among higher education staff) highlighted the dismaying extent of the problem and considered some of the driving forces. It’s not just the student body that has seen a rapid rise in referrals for counselling, something that has already been much discussed, the increase in numbers amongst HE staff has likewise soared. The HEPI report highlights the difficulties that managerially-driven workload models impose; the ever-increasing use of targets and metrics of different kinds (despite all the warnings introduced in The Metric Tide report, and the recognition that many of these metrics are just proxies for the true value of what is being measured); and the ubiquity of short-term contracts which allow no real opportunity for either security or career development. As a recent article by Ellen Kirkpatrick in the THE highlighted from personal experience, exploitation is rife of those at the bottom of the ladder.

Need this be so? Liz Morrish thinks not, and proposes some solutions which, she claims (without costings) would be cost-neutral. Certainly it is easy to see many hidden costs in driving academics so hard: the cost of running an ever-expanding counselling service, or all those ‘well-being’ activities – yoga at lunchtime anyone? No time! Let alone the working days lost due to poor mental health brought on by overload and stress. There is a huge cost involved in lost days and they only add to the pressures on other folks who have to step in, often at short notice.

That the workplace needn’t be so stressful, that other sectors manage better (albeit their characteristics may be significantly different) was brought home to me, coincidentally also last week, by a visit from a professional manager in health, driving issues around improvements in their multi-national company by considering what makes a difference to mental well-being. By investigating factors that cause stress – potentially leading to serious accidents in their industry when a worker can’t perform at their best – the company was able to improve both health and productivity. It seems quite small adjustments can lead to substantial and measurable gains in anything from profit to retention. I was challenged to consider what metrics HE might use to determine equivalent improvements. For instance, if better support is offered formally to recent academic staff joiners, will their grant success rate improve? If an eye is kept on drop-out rates of PhD students, will it become obvious which academics are abusing their power and making the life of their group a misery? It isn’t difficult to think of metrics that may actually measure the ecosystem’s health and identify failings in structures and processes.

As indicated above Liz Morrish highlighted over-ambitious workloads being imposed on staff by ‘the centre’ as a major problem. Filling up every hour of the day at the start of the year with lectures, tutorials, labs, committees, where does that leave time for writing references, replying to enquiries from anxious students, writing grants let alone attending conferences or getting out there in the big bad world passing on hard-earned expertise? I am lucky my university does not operate like this, and we are largely judged on what we deliver not the timing of when we deliver it. That of course has all kinds of downsides too since the work may never stop, academics are all too often driven by perfectionism so why would I knock of at 6pm when I could work till 9 and get that referee’s report finished, that grant form completed….Nevertheless I am grateful neither presenteeism nor slavish filling in of time sheets are required of me.

The Athena Swan process is an opportunity to explore workload models too, which are one of the elements an action plan for an award might consider. But a model is only ever as good as its ingredients. If the workload model takes no account of external professional service, for instance, sitting on committees for research councis, conference organisation or those of  professional bodies, then it damages the whole community. The system would not run smoothly if people did not take on these roles, and many people may be better at these tasks than being patient in a first year practical class with the klutzy novice, or possibly even than in teaching a first year service lecture course to hundreds of unmotivated students who would rather be anywhere else than having basic thermodynamics thrust down their throat at 9am on a Monday morning. What about all that outreach work? Does that get factored into a workload model? Or giving time to the local LEP (Local Enterprise Partnership) to support and drive relevant innovation and build community bridges with a university? There are so many roles that almost certainly benefit the wider community, at least as much as the individual, yet which carry no weight in a typical departmental model. With the net effect that the staff member has to add in hours of extra work in each week, reducing quality time at home or time recharging batteries.

The system as is seems unsustainable. Mental health is suffering up and down the country’s universities and, with pronouncements about ‘scaremongering’ amongst universities about their financial futures from the Education Secretary Damien Hinds in the run-up to the (purported) release of the Augar Review Report, it is hard to see there being any slack in the system likely to reduce the burden on the individual in the near future. On the contrary, if Hinds gets his way it would seem likely that pressures will only increase to get more bums on seats, more ‘efficiency’, and less actual education (as opposed to training) of students and more staff with burn-out and poor mental health.

We need to learn from other sectors about looking after the health of our staff, not just with ergonomic workstations for physical well-being, but in support and thought about what is best for everyone’s mental health. The HEPI report highlights the two well-documented suicides consequent on overuse of targets and excessively heavy workloads. It reminds us of the human cost of an impersonal management that ignores the individual and their needs. As a sector, we need to do better and I hope senior management teams – at the institutional and departmental level – will take time to reflect on the messages and vow to do better in the future.

Posted in Education | Tagged , , , | 1 Comment

Letting It Go

To many people Steve Shirley is an early entrepreneur in software development who made a fortune; a woman who rebranded herself with a man’s name in order to avoid being ignored by the blue chip companies she wanted to use her services; and a woman who employed women working from home to create her business via this new flexible-working model. That is probably how I thought of her. A leader and a brave woman who made good in a world made difficult for her, first by her birth (she escaped the Nazis via the Kindertransport aged 5) and then by her gender. Having just read her autobiography Let It Go (recently republished) I realised how much more to her there was. Indeed, by the end I realised I’m not sure that the descriptions I’ve just given are how she would want to be remembered because there is something central to her life I haven’t mentioned: she had a severely autistic son. From his original diagnosis this blunt fact dominated her life, despite her managing to continue to excel in the business world.

Her book is stark. She is brave in what she writes. I wrote recently about how when giving talks about my own life – by comparison uneventful and easy – I can choose what I do and don’t relate. Shirley’s book would seem to hold nothing back, although even that may not be true. Her son Giles wasn’t just ‘on the spectrum’, in that revolting phrase cavalierly trotted out so often, he was classed as ‘undeducable’. For part of his life locked up in an institution for ‘mental defectives’ (this was the 1970’s), he died following a seizure aged just 35. Trying always to give a mother’s love to a violent son who couldn’t really speak whilst trying to ensure her nascent company thrived simultaneously, it isn’t surprising Shirley herself had a breakdown and had to be hospitalised, nor that her marriage suffered as a result. With today being the last day of Mental Health Awareness Week it seems fitting to celebrate her life, and what she went on to achieve in and for the field of autism.

Her child, as for many parents, was core to her being. Watching a child suffer yet need to be locked up, she wanted to find a more humane treatment. In her case, her successful business made it financially possible for her to find a solution, creating a home for Giles and, in due course, others that was much more of a domestic home than an institution; and then creating more such homes and ultimately a school. She became involved with research, funding research, funding synthesis of research to enable much more clarity to be achieved about what autism was and wasn’t and what treatments did or did not help alleviate suffering. As a multi-millionaire all of that was possible for her. More, it seems to have felt like a moral imperative that this was what she had to do, particularly once the stresses of being always there for her son ceased after his death.

It is a remarkable story and one for which many parents of autistic children will feel deeply grateful. Her own experience of the breakdown due to the multiple stresses she was attempting to operate under at full stretch was barely touched on, but no doubt many readers in the HE sector will recognize the challenges posed by multiple, conflicting demands on limited time and energy. As has been remarked so often, many academics work ridiculously long hours, trying to satisfy the requirements of heavy teaching loads, pastoral care and their research (not necessarily in that order). If simultaneously they are trying to care for a young family or elderly parents the circle simply can’t be squared. Burnout, breakdown and despair are not infrequent consequences. As a sector we frequently don’t have the balance right, nor do we often manage it as individuals. Most academics have fallen into the profession because of love of research and research, when it is going well, can be all-consuming. The ‘highs’ of discovery are, in my view, like no other. But the grunt work occupies the bulk of the time. Teaching, too, can be infinitely rewarding and infinitely time-consuming. Our careers and our universities too often overlook the need for space to breathe and recoup with family and friends. This is not healthy in any sense. Mental health awareness week should remind us of this.

But there are other demands on us too – emotional demands provoked by racism, sexism and just about any other kind of -ism. I have placed my order for Angela Saini’s new book Superior to help inform me about the insidious (and growing) forms of the first of these. The misogyny and inappropriate behaviour that so many women face is highlighted by a strong recent article by Charlotte Proudman – this time not in academia but in the legal profession where the behaviour of those responsible is particularly egregious as they are the ones who pass judgement on others guilty of similar crimes. She is a woman who has faced plenty of sexism herself so she knows of which she writes. The harassment, bullying and intimidation that remains endemic across our society all contribute to an individual’s shaky mental well-being. Letting these ills go takes time and energy that many of us simply can’t summon.

Steve Shirley’s autobiography is a stern reminder of the challenges many people face in work and at home. Her book is at times deeply moving and uplifting. She is a brave woman, and not just a multi-millionaire entrepreneur who defeated the sexist odds. Her book’s title comes from the wisdom she has gained in not holding on to those things you have in reality moved on from (but keeping the central tenets close). I recommend this book to you.

 

 

Posted in Equality | Tagged , , , , | Comments Off on Letting It Go

Friends with Benefits

A recent study shows – in Switzerland at least – that nominated referees judge grants more favourably than those unconnected with the applicant. I’m afraid I didn’t find the conclusion of the study a surprise. Additionally I suspect that having ‘friends’, nominated referees or simply people you know in the field, is a benefit that will inevitably work better for the well-connected. Well-connected perhaps because they work in a ‘top’ lab, or the applicant is already a giant in the field who knows everyone who is everyone. In this way there is every danger that the Matthew effect will come into play: those already successful will just be given more funding, reinforcing a conservative status quo, hindering the new and original.

Just like the impact factor of a journal not being a good proxy for quality, being already a giant does not necessarily mean that the current grant proposal is well thought through. It is perfectly possible that said PI (Principal Investigator) is operating a grant factory, churning out the equivalent of pot-boilers or ‘outsourcing’ the work to a junior colleague who has not yet mastered the art of grant writing even if there is the germ of a good idea lurking. An objective reviewer, neither swayed by the giant’s reputation, nor alternatively someone who has crossed swords in the past at conferences with this giant or knows their reputation for pot-boiling, should be able to form a more accurate assessment. That is, after all, what objective means. Biases may equally creep in if a long-running feud operates between applicant and referee. This is the reason why it is sometimes permitted by a grant agency for an applicant to name those, individuals or groups, whom they want the application not to be sent to.

I once sat long enough on a grant-giving body (in the days of standing committees) to see a particular applicant fail time after time because one particular referee – this was in the UK and the field was a small one, so this same referee always seemed to be used – raised the same criticism every time, a criticism which was never satisfactorily addressed by the applicant who clearly hoped that somehow they could dodge this referee the next time. In the end the panel got fed up. Feedback was provided that referee and applicant should get together and write a joint grant application to resolve once and for all which of the two was right about how the analysis should be corrected. I think they went on to have a long and productive collaboration.

On the other hand, on this same grant-giving body, I watched with astonishment as the Chair remarked of an application – possibly of a current panel member currently out of the room, but certainly of a friend who had previously sat on the panel (I forget which, but we all knew they were pals) – that of course all of us knew what they had meant to write. Clearly, the Chair agreed, we also appreciated the grant as written wasn’t very good but if funded excellent work would undoubtedly be done. I was gob-smacked, but still quite junior. Luckily another panel member had more courage, or possibly seniority or even both, and called the Chair out for improper behaviour. The grant, I am glad to say, was turned down. I hope the Chair was suitably abashed, although I never enquired.

However that story, which by now is many years old (probably at least twenty) indicates just why the study on using ‘friends’ as referees is important and why we should all be alert to such dangers. Equally, it highlights why I was so pleased to hear Melanie Welham, Excecutive Chair of the BBSRC, say – in our public conversation this week at Churchill College (a recording of which will soon to be on the web) – that ‘safeguarding’ training will soon, possibly already, be required for their panel members, alongside Unconscious Bias training. Time for reflection at the end of a panel, before the rank-order list is finalised, ought to offer an opportunity for any uncomfortable member to speak up; to say that they were worried by the way a grant had been handled, or they felt not enough attention had been focussed on a dodgy referee’s report (perhaps written by someone with an undeclared conflict of interest, i.e. a ‘friend’) perhaps. Such action seems to me to be an excellent idea.

Nevertheless selecting a ‘friend’ as a referee, doesn’t always work. I’ve sat on committees for fellowships where applicants have been able to nominate a referee alongside one chosen by the organisation. It was not that unusual to find the nominated referee expressing strong doubts about the applicant. I wanted to be able to write to the applicant to say ‘choose better next time’, but of course that wasn’t possible. Nominating your referees is the norm when it comes to job applications. Sometimes too, that can go horribly astray. The art, or is it science, of choosing referees probably needs much further thought.

The harsh reality is that, the UK is a small community when it comes to science. Researchers in a field tend to know each other and decisions will have been internalised about their good and bad points by those who’ve been knocking around for a bit. Sometimes it’s hard to remain objective when you know that Professor X treats his students as bench monkeys or that you never agreed with the way they interpreted a certain set of experiments – or alternatively that you think they’re the best thing since sliced bread and think anything they do will yield gold dust. With the best of intentions, it can be hard to forget prior knowledge.  Whereas newcomers may face all the challenges of being comparatively unknown, that sort of baggage will not apply. I know there are those who champion blinding applications as the way forward. But I fear in such a small community it would still be all too easy to guess whose application you were reading, even if personal details were removed.

Related to this point I’d like to highlight the Smith Review into what a domestic fund should look like if EU funding becomes inaccessible to scientists in the UK. As I, along with others, have frequently said, one of the big plus points of the ERC decision-making process is that its panels and referees are drawn very widely from well beyond Europe, and potential conflicts of interest are treated with great seriousness. In considering any new possible future domestic funding, I hope the evidence from the Nature study regarding ‘friends’ and all the benefits they bring, is taken very seriously by the Review group. If a new programme is going, in any faint way, to emulate the ERC we need to be sure refereeing and panels are not peopled by ‘friends’ but by objective experts.

 

Posted in Science Culture, Science Funding | Tagged , , , , | 1 Comment

Telling A Life Story

It has been a while since I last posted on my blog. In part that was down to the Easter holidays, but more than that I think it was a combination of the exhaustion of the previous term, coupled with horror over the unfolding Brexit debacle at the end of March (like many another in the UK, I think following the news slavishly was upsetting my mental equilibrium) followed by, as the new term started, a deluge of ‘stuff’ to get through. This lead to what I think of as overload paralysis.

Although there are times when having many tasks to do can lead to greater efficiency, since there is always something to do that fits into the mental and temporal space available, there comes a point when the overload is just too great for any kind of efficiency at all. Instead, one is constantly pulled in multiple directions, each neutralising the other so nothing gets done. If the time slot is simply the odd half hour between meetings (with, in my case quite often, receptions and dinners at the end of the day removing that time from the equation), then it is all too easy to find that there is no time for anything beyond deleting some spam emails and reading the committee papers.

It should be remarked I have yet to turn up at a committee meeting without having read the papers; it is more common I turn up simply without the papers (or device) on which I have scrawled my notes probably because I’ve grabbed the wrong batch. Anyhow, those are my excuses for my relative silence. Whether this term will ultimately turn out more conducive to regular posting remains to be seen.

I am sure long-term readers of this blog will have formed their own view of my character and life, which may of course be accurate to a greater or lesser extent. In the wake of the recent ruling regarding whether Caster Semenya is allowed to run in women’s races, there have been interesting articles written about identity and what defines it. I would refer you to Angela Saini or Gaby Hinsliff for some compassionate and thoughtful writing. Our identities are composite and complex. How much of it we choose to share is – for most of us for most of the time if not for Semenya – something we can decide. This is as true on my blog, as in my talks about my life or in my conversations with my friends. There may be parts that I choose to share when trying to encourage those setting out on their careers. You will never know about those parts of me that I am determined no one will ever discover (if of course any such there be; I’m giving nothing away). That is my choice.

When presenting a ‘careers’ talk’, as I did last week at UCL’s Institute for Healthcare Engineering as part of their ‘Adapt to Thrive’ series, it seems to me to be helpful to present the passage of my life in a variety of ways to show how different strands weave together to make a whole. There is the ‘standard CV’ format: the dates I moved from one role to another and where I did them. That is familiar territory to anyone who has ever applied for a job. I overlay on that the times when my research made some (relatively) abrupt change either deliberately or – as often in my case – driven by circumstances. Circumstances which varied from needing a new job to extend my visa in the USA after my first postdoc, to ‘inheriting’ a grant from my predecessor who’d left the country. I feel this is an important version both because it stresses the importance of luck and chance and because the belief that it is not necessary to go in a straight line with a siloed attitude to research to thrive is also something I feel strongly about. As I tend to put it, seizing opportunities is good, at least most of the time.

The third version of my life, equally important and an equally important message to give out, is the more personal: marriage, children and most recently elder care, clearing a house after death, becoming a grandmother. All of us need to recognize that ‘stuff happens’ and not always according to plan, but life beyond the bench matters hugely. Denying this, trying to squeeze everything into the standard CV format and forgetting our humanity, is destructive on all kinds of fronts. Yet listening to some eminent folk talking about their careers we are more likely to learn about the impact factors of the journals they publish in than some of this and never sense that their life might not have gone smoothly right from birth to the current day. I think some outburst like ‘pshaw’ needs to be introduced here to express my disbelief when I hear such accounts.

Of course these days I am rarely near a lab bench and my career took a turn I had not had in my sights when I moved to be Master of Churchill College. When people ask me what I enjoy most about this role I tend to say (though it does depend on the occasion and my mood) that it is the incredible variety of things I get to do and the incredible and (almost invariably) interesting people I meet, from students to Nobel Prize winners. Last weekend, in this role, I had yet another ‘new’ opportunity to seize – I went to a Rugby match. Indeed it was the final for the Cuppers Plate between Churchill and St Catherine’s*, which Churchill ultimately won fairly easily. A good match for me to start with. The rules have changed a lot since I used to watch what was then the Five Nations matches in black and white on the TV in my early teens, and I’m sure I did not follow the nuances, but it was still more than possible to get wrapped up in the game and cheer along with the best of the crowd. A crowd, I was interested to note, that contained a high proportion of women. I even got a photo opportunity to pose with the winning team. Well done Churchill!

* A Catz alumnus has pointed out that the correct spelling in Cambridge is St Catharine’s – the alternative spelling being the correct form in Oxford. I am embarrassed that after 40 odd years in Cambridge I had never appreciated that.

 

Posted in Science Culture | Tagged , , | 1 Comment