As a principal investigator, or PI, one of the tasks that I am consistently faced with is “interpretation.” How so? Well obviously, my job revolves around interpreting data and trying to understand if experiments done by my students and co-workers really do support or rule out hypotheses that we have proposed.
However, the role of a PI in interpreting extends far beyond data. It can be interpreting the mood of a student, or his/her motivation. It can be in interpreting the comment of a collaborator–on the phone, e-mail or in person–as to whether they are enthusiastic about a given project. And it can, and often is, about interpreting reviewers’ critiques regarding manuscripts or grant proposals submitted. I hope to expand on the latter two issues in some upcoming blogs.
Today, I would like to focus on another form of interpreting–one that needs to be honed to a fine art form, and can be rather tricky–interpreting reference (recommendation) letters. In the 8 years since I have been in this position, I have come across thousands of such letters: recommendation letters for students to gain acceptance to our Ph.D. graduate program, letters on behalf of potential post-doctoral fellows eager to gain employment, letters in support of candidates for faculty positions in our department. In my role as reviewer, I have also examined recommendation letters for students and post-doctoral fellows who have submitted fellowship applications. Okay. Enough, you say. I’ve convinced you that I’ve read (and written) my share of these letters. SO what?!
The point that I want to make is that interpreting recommendation letters is a fine art that needs to be honed. Moreover, there are huge cultural differences when reading letters from across the pond, or from other parts of the globe.
US recommendation letters, for the most part, are extremely diplomatic. They follow the old axiom “If you can’t say something nice about a person, then don’t say it.” What this means is that the reviewer needs to proceed with extreme caution. Why? Because it’s not what’s in the letter, but what’s missing that’s important!
An example: if a letter discusses an excellent student, who is extremely determined, very bright, and excellent speaker and writer–but ends with this short description–then an American PI will have the impression that perhaps this student doesn’t get on well with his/her colleagues. So a nuanced reviewer needs to constantly look for whatever is lacking. On the other hand, so-called minor issues such as “despite being in the country a short time, the student has done a good job catching up in language” will often mean the person can’t speak a word of English!
So PIs in the US become accustomed to looking at a checklist of qualities, and taking special note of anything that is either understated (ie., “the student is good at writing” will generally mean the student can’t string 3 words together in a row), or worse, missing altogether from the letter.
As it turns out, this is not a universally accepted way of writing and interpreting such letters: in Israel, and I believe in many European countries, the system is far more direct. PIs will give a more candid assessment of applicants, often highlighting out their good and not-so-good traits. An American PI reading a recommendation letter from Israel or Europe that notes “the student is outstanding in work at the bench, but needs some guidance conceptually” will undoubtedly conclude that this student can’t think and plan his/her own experiments. However, the referee writing this letter may only be noting that the student fares better with some guidance and discussion and isn’t yet fully independent.
Meanwhile, a European PI receiving a post-doctoral application from an American-based student may be surprised at how the student is so “overwhelmingly outstanding” in every category listed, not realizing that because communication skills are not addressed in the letter that the student might actually be essentially illiterate!
Expanding on these cultural differences, I suppose the worst danger for a student or post-doc would be to move with his/her PI from Europe to a lab in the US, and to look for a job in the US before the recently moved PI has had a chance to catch up culturally. The newly migrated PI might think he/she is writing a strong supporting letter, but eventually find out (when the student doesn’t receive any interviews or job offers) that “one man’s strong recommendation letter is another man’s weak letter.”
This takes “divided by a common language” to a whole new level!
Frank,
I’m still laughing at your comment about including the fiber content of the stick on the ice cream!
Yes–divided by a common language. Bryson hits on this, but not, of course, for reference letters:
http://www.amazon.com/Mother-Tongue-English-How-That/dp/0380715430
I’ve been involved in job searches here in Germany, and the prof. in charge told me we shouldn’t bother asking for references, because she can never work out what they’re saying or not saying anyway. I rather agree.
Yes, anything left unsaid is suspect. If someone in the US writes really positive things about a scientist, but does not explicitly state that he/she is pleasant–then the person might well have temper tantrums in the lab. Can’t sue the PI afterwards–there was no comment about that!
Yes indeed, a tricky issue. Related is the writing of a reference for a not-so-stellar former employee… what do you put in, and what do you leave out? The “reading between the lines” of reference letters is an art entirely unto itself.
I’ve had similar experiences in speaking with referees directly on the phone… some probing often sends up flags about aspects that would have been difficult to glean from the written letter.
I think the choice of referee is also sometimes telling – sometimes people avoid their most recent supervisor because that person is not aware they’re looking for a new job, or because of a personality clash. Once in a while I’ve found out that a referee is a friend, or was a peer rather than a supervisor.
And then, following on from Bob’s comment, is the whole international issue – occasionally one will read a letter of reference provided by a non-speaker of your language (English, in my case), that just doesn’t make any sense at all!
Yes, a reference letter from someone who is a peer (rather than a few levels higher on the ladder) is not good practice. I’ve seen a student who had 2 post-docs in his own lab write recommendation letters for a fellowship, and that is a no-no. You also don’t want a letter from your grandmother, uncle, niece, or third cousin twice removed.
“I think the choice of referee is also sometimes telling – sometimes people avoid their most recent supervisor because that person is not aware they’re looking for a new job”
Is that necessarily a red flag? I know that within academia it’s normal and expected for people to openly look for jobs at the next career stage, and for their supervisor to help them – but in the private sector (and even in the non-tenure track parts of academia) it’s much more common to have to conduct your job search behind your supervisor’s back. For example when I applied for my current job, I did so in an environment where if you were found to be looking elsewhere you’d better make damn sure you got the new job, because you’d almost certainly be “let go” from your current job soon thereafter if you didn’t. I therefore don’t think secrecy should be held against a candidate.
I had the senior member of my team as one of my references, btw, and that person was one of only two colleagues at my old job I told. I had “reference from current direct supervisor available upon request” underneath that person’s name, and was open in my covering letter about not being able to openly interview and ask for references. My postdoc supervisor, whose lab I’d left just over two years earlier, was my second reference. Luckily my current boss has industry experience and was very understanding about the whole situation.
The interview process for my current job was rather scary, as you can imagine… I had to meet with multiple PIs and clinicians, often at very short notice due to their packed schedules. Luckily the two buildings are very close together (although that did increase my risk of detection), so I could slip out for a (very) early/late “coffee”/”lunch” some days… if I had enough advance warning I would say I had a “doctor’s appointment” (hey, it was an appointment, with a doctor…) The process dragged on for many weeks, and I was planning my wedding at the same time… not one of the most relaxing periods of my life!
There are clearly times when it’s hard or impossible to get a letter from one’s mentor/supervisor for various reasons–including not seeing eye-to-eye with that person. At the same time, IF a student or post-doc wants or needs to go this route, it’s imperative for find a good substitute; not another student or post-doc in the lab, but someone of relatively senior stature to offset this problem. Otherwise, I think it will not look very good.
For example, in reviewing fellowship applications, today things are so competitive that if there are 3 good applications and it’s hard to distinguish between them, reviewers would most likely elect to choose the applicant who has a sterling letter from his/her mentor rather than the one who provided letters from sources either more removed or less senior.
As an American working in the UK, I have seen this working in both directions. In fact, I often read letters and filter: is this an “American” letter [superlatives about a few traits], or a “European” letter [the “truth” about everything]?
I always call the referee if I am serious about a candidate. This tells you much more than the actual letter. Of course this really doesn’t help those who don’t make the cut because their advisors are “European”.
Calling is usually better, but it’s surprising how cagey people can become on the phone!
Great post, Steve – we were just talking about this at work last week! I never really got to see any reference letters in the UK as I was too junior when I worked there, but I’ve seen some positively glowing references here that make the candidate seem like a cross between Marie Curie, Albert Einstein and the Dalai Lama, but when they show up to an interview they give a mediocre talk and can’t answer even the most basic questions about their project. Perhaps my European boss and I both need more experience in the interpretation of North American reference letters! I’m just glad that my PhD supervisor, who provided references when I moved to Canada, had done a postdoc in the US and knew about these cultural differences!
Yes, if one is looking for work in the UK, it’s good to have N. American references; the other way ’round might be tricky! Good that your former mentor “knew all the tricks!”
We’ve recently had a rule introduced that reference letters may not be taken up until after the interviews. I wonder if other institutions work this way? I must admit that I found it made more sense to have all paperwork, i.e., references, CV, personal statement etc., available during the interview in order that decisions could be made immediately afterwards while the panel is assembled and all thoughts are still fresh in the mind.
Very late comment for the old post! I have found your blog by one of your papers and I am happy for that!
I enjoyed reading this post. Besides what you mentioned as “Interpreting Reference letters”, I have another problem with reference letter: what if the supervisor is mentally sick? (e.g. your PhD supervisor) What should I do in this case? It is hard to believe that how a professor at the top universities can be mentally sick!
Presumably all the referees should tell the truth, regardless of how the truth is helpful or harmful for the applicant, but imagine the case that the referee tells lie. How awful is this! I am suffering a lot from this and it damaged my career and my life heavily. I lost so many good opportunities and I ended up in a very bad research group and wrong PI where I am in a serious conflict with him because, honestly, he does not know anything about biophysics.
Any solution or suggestion for this kind of problems? If I cannot find a proper solution I should quit science forever and it is impossible for me to do what I do not like to do. How should I save my scientific life?
Sorry, I cannot write my name, but you have it in my email address.
Regards,
A