What is a LIAR ?

Why, but a Lexicon of Inconspicuously Ambiguous Recommendations, of course !
(credit to Ewin.com for the acronym)

Confidential letters of recommendation have long been regarded as a precious tool in the evaluation of professionals in all walks of life. Perhaps nowhere as in academia, however, do they form an integral part of the portfolio of an individual. Virtually at all stages of a scholarly career, letters of recommendation are sought by departmental committees, principal investigators and administrators, in order to assess the aptitude and readiness of candidates for various academic positions (graduate assistant, postdoctoral associate, faculty at various levels); a letter of recommendation will also typically include a statement of the person’s relative standing among peers.
Confidentiality has been traditionally regarded as an important ingredient, for fairly obvious reasons. The reliability and usefulness of a letter of reference depend on its candor, openness and (if necessary brutal) honesty.

Writers of letters of recommendation (“referees”) ought to feel comfortable expressing their reservations about candidates, without fear of future retaliation [0]. Thus, it seems opportune that those soliciting a letter of reference take appropriate steps to prevent the person being evaluated from reading it, and to ensure the referee’s anonymity [1].
Naturally, since a harsh letter can have devastating, long-term consequences on a career, the fairness and legitimacy of the process crucially hinge on the integrity of the referees, who are handed a great deal of influence and power over a person’s future. The line is often blurry between frank but honest, accurate and impartial assessment, and gratuitous disparagement, the latter reflecting a referee’s idiosyncrasy and animosity toward the applicant (or, the applicant’s PhD supervisor…) rather than scholarly objectiveness. The only protection that a candidate is offered, is just the collective wisdom of the committee evaluating the application and reading the letters. This is not as weak a defense as one might think. A mean-spirited letter, full of of personal attacks, or criticisms not backed by some objective evidence, may in the end reflect more negatively on its writer than on the candidate, and simply be dismissed, especially if other letters should paint a drastically different picture of the applicant. In tenure cases, for example, when as many as ten different letters may be sought, it is certainly expected that not all of them will sound equally positive, and that an oddball, weaker than average one, may be received [2].

In recent years, however, it has been increasingly felt that candidates at different levels ought to be able to exercise greater scrutiny and control over what is written about them. For example, it is now typically the case that graduate school applicants are given the option of retaining the right to read letters of recommendation written on their behalf. Moreover, as a result of widely voiced concern over less-than-transparent tenure reviews, as well as some highly publicized litigations, some universities are now granting assistant professors applying for tenure significant rights, e.g.,
a) participating in the selection of external reviewers, possibly vetoing some names
b) examining all letters of recommendation written by external reviewers.
(I had both privileges when I applied for tenure at my former institution in 2000. I ended up waiving b) upon request from the Dean). It is generally a good “rule of thumb” for anyone writing a letter of recommendation these days, not to assume that the applicant will never read it.
Although the above measures, as well as others also having the effect of eroding confidentiality, may be well meaning, they are seriously misguided, and are likely doing more harm than good. In particular, it will serve the best interest of tenure-track assistant professors if the practice is retained of keeping letters of recommendation written in support of their tenure case strictly confidential
(the things for which probationary faculty should fight, are adequate research support and reduced teaching load, not the right to read letters…).

The feeling that one’s application may have been derailed by the negative assessment of a reviewer, may conceivably prompt someone to hold the grudge, seek to retaliate, and even file a lawsuit against said reviewer (see here, for instance). Because nobody likes to be taken to court, an easily predictable consequence of doing away with confidentiality, is that for the most part referees nowadays stay away from the type of blunt, candid and objective statements that evaluators seek in the first place, opting instead for letters that are “safer”, i.e., ambiguous, often misleading, and in many cases congratulatory to parody level. Obviously, all of this all but defeats the purpose of the entire exercise, as it renders letters untrustworthy, hence useless.
In particular, over the past few years I have consistently observed the following:
1) Seldom do letters mention explicitly a candidate’s shortcomings, even the most obvious ones (e.g., manifest difficulty with the English language). As a result, when the candidate shows up for an interview, on realizing that referees were less than forthcoming, interviewers turn suspicious, wondering what else may have been left out. I have witnessed job interviews gradually morphing into exams, a most unpleasant experience for a candidate.
2) In order to convey an opinion in the safest possible way, referees resort to the type of lingo ridiculed here or here, with the expectation that committee members will “read between the lines” [3]. Problem is, the exercise of reading between the lines is terribly subjective, and of dubious reliability. One will often “read” something that is actually not there, e.g., interpret as an implicit, “coded” warning about the candidate what may be no more than a minor omission, a misstatement, an oversight on the part of the referee. Any sentence that may come across as half-hearted, or tentative, is fraught with danger for the applicant.
3) Along the same lines, because explicitly unfavorable remarks on an applicant are purposefully avoided, evaluators (especially administrators) almost inevitably come to regard as negative anything that is less than hyperbolic. In other words, a system is set in place wherein inflated, over-the-top and almost caricatural praises are now regarded as the norm, lack thereof being conversely interpreted as a sign of weakness. This can be a serious problem, especially in tenure cases and especially if administrators should feel that a candidate has already succeeded at keeping out of the process the potentially most critical reviewers.

Thus, good, solid applicants with the misfortune of having referees with integrity, unwilling to go on record with nonsense such as, “this is the new [insert name of your favorite Nobel laureate here]”, “the best student I have ever supervised” [4], “one-in-a-million”, “a superstar“, etc., will be likely passed over, and branded as “middle-of-the-road”, “just average”. A reasonable, positive but measured assessment of the capabilities of a candidate, may well be the kiss of death for his/her career aspirations.
True excellence and uncommon talent of the few really gifted ones end up lost in the pile of superlatives used for everyone. In short, the very dishonesty that one was aiming at eliminating in the first place (likely to occur only rarely) is instead encouraged and promoted on a large scale.
And of course, because at some point we all run out of hyperbolae, another consequence is that the content itself of the letter becomes almost irrelevant, and only the names and fame of the referees and the prestige of their affiliations come into play… but let us not go there today.

Notes
[0] It is often suggested that the proper course of action for a referee unwilling or unable to write a glowingly positive letter of recommendation, is simply that of declining to write one. As those who are in the profession know, this is almost never a viable way out. First off, the refusal to write a letter has often the same effect of writing a negative one, especially if the referee is in no position to plead ignorance (e.g., a direct supervisor). In fact, the absence of a letter from a supervisor may damage a candidate even more than a slightly downbeat one (“How bad can this person be, for the supervisor to decline to write a letter ?”). Secondly, in some cases the letter is not sought by the candidates but by third parties, and complying with the request is simply expected, as part of the academic profession (e.g., in the case of someone going up for promotion or tenure, or competing for an award). Finally, if a student is dead set on applying to graduate school and needs a minimum number of letters, (s)he may well insist that a letter be sent out anyway, even after being warned about its lukewarm-at-best tone, for the simple reason that (s)he has no one else to ask.
[1] Incidentally, this is very much in the same spirit of student evaluation of instruction, whose anonymity is widely deemed necessary by its proponents, even though the scenario of a professor retaliating against a student for a bad evaluation seems exceedingly unlikely.
[2] Obviously, it is an entirely different ball game if the same, pointed and damning comment about a candidate, appears on two or more letters. At that point it becomes difficult to attribute it mostly to one referee’s caprice and mean-spiritedness.
[3] Having served on three tenure-track search committees over the past decade, I can speak to the fact that letters of recommendation can indeed sometimes be “read between the lines”. For example, a negative letter will often praise the applicant very superficially, but without mentioning any specific contribution made by the person. Or, it may simply list the projects on which the applicant has worked, without discussing their significance. Other times, however, it is far from clear whether any “subliminal” message is embedded, especially when the letter is written by a non-native speaker of English. In some cultures, for example, compliments are used sparingly. Thus, letters often come across as unenthusiastic, but in most cases nothing unfavorable about the candidate is really implied.
[4] Yes, I have seen this one written by the same referee about more than one person, in the same year.

Tags: ,

4 Responses to “What is a LIAR ?”

  1. Massimo (formerly known as Okham) Says:

    A postdoc I know was denied grant funding specifically because one of his recommendation letters was less sterling than the others. Perhaps an applicant ought to be given the opportunity to respond to, or explain, a negative letter?
    I doubt if that would change anything, frankly. Opinions are opinions, and there is not really much you can do, if someone else simply does not think that you are stellar. You can only hope that with time they’ll be proven wrong, or simply that they will never review another application of yours. I am afraid we have to live with a degree of imperfection.
    My worst experience by far has been with a couple tenure-track candidates who were actually hurt beyond repair by glowing letters of recommendation. Their referees simply went overboard, thereby setting very high expectations. Candidates showed up for the interviews, and it was clear in no time that they were nowhere near the level suggested by the letters. Committee members felt duped, and turned hostile. I think both candidates still have nightmares about those interviews.
    (By the way, when I called one of the referees to ask for an explanation, he would not talk to me about that — kept saying that all he had to say was in his letter… I tell ya, it’s getting crazy).
    And actually, calling referees is explicitly forbidden at many public universities — at my previous institution we were instructed not to do it.

    Massimo

    the above was in response to the following comment:
    Submitted on 2008/03/30 at 9:48pm
    I completely agree that there is a trend toward exaggerated praise in recommendation letters, and that the abolishment of anonymity will exacerbate this.

    But I still think there ought to be some recourse for an applicant who finds his career derailed by a negative (or not positive enough) recommendation, particularly for early-career scientists. Postdoc applicants in my field, for example, usually only furnish 3 letters such that two rave reviews may not be sufficient to counteract one negative one. A postdoc I know was denied grant funding specifically because one of his recommendation letters was less sterling than the others. Perhaps an applicant ought to be given the opportunity to respond to, or explain, a negative letter?

    As for having to read between the lines of hyperbolic recommendation letters, I’ve found it useful to disregard the written letters altogether and simply call the applicant’s references. In my experience, people are more forthright in a phone conversation, and the tone of their comments much easier to discern.

    • Anonymous Says:

      You’re probably right that in most cases, responding to a negative recommendation won’t change anything. However, this particular postdoc got his postdoctoral position despite the negative recommendation in part because he specifically addressed it during his interview. (I don’t think he knew what was in the letter, but suspected it might be negative because he and the writer had not gotten along.)

      I suppose I find the idea that someone could stab me in the back with impunity unnerving because I’m still junior enough to be quite vulnerable to a more established scientist’s bad opinion.

      We’ve interviewed several postdoctoral candidates who came highly recommended and then failed to live up to expectations. I think in those cases the writers of the recommendation did a fair amount of damage to their own credibility as well.

  2. Anonymous Says:

    famous boss

    Well, in light of what you wrote in this thread and based on my own experiences, it is the reputation and credibility of the letter writer that counts at the end of the day rather than the actual content. A friend of mine who got hired by MIT as assist. prof. last year had letters from one nobel lauarate, one prof. who had fritz london prize and another who is in national academies. Since almost everybody is saying these days his student walks on the water, it depends on how accurate his previous letters turned out to be about his previous students. Did they live up to the expectations?

    • Massimo (formerly known as Okham) Says:

      Re: famous boss

      it is the reputation and credibility of the letter writer that counts at the end of the day rather than the actual content

      Completely agree. And while I do not think that this is a consequence of erosion of confidentiality, I think it is significantly amplified by it.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: