“I like his hair”
(from one of my student evaluation forms)
Student evaluation of instruction (SEI) is a frequently discussed subject among academics, one of those that tend to elicit heated debates and strong emotions. This post at academics_anon and the ensuing comments, have rehashed some of the most common aspects of SEI that college teachers find objectionable.
Having taught at the college level for twelve years, I share some of the concerns that a good fraction of my colleagues voice; however, my main problem with student evaluations does not really have much to do with what students write about me. It is just that I have never found the type of feedback provided by students on anonymous evaluation forms all that useful, which in turn makes it difficult for me to take SEI seriously, as a way of assessing teaching effectiveness.
I am not going to discuss here the wealth of literature and research on SEI (for those who are interested, a good place to start is here); instead, I am going to describe my own experience, and make some personal remarks not based on any scientific study.
First off, a full disclosure of my own record is in order. My subject is physics (not exactly a popular one), and over the past twelve years I have been evaluated by students at all levels (and of all levels). Much to my embarrassment, my evaluation record is pretty boring. Aside from the numbers (I typically get averages around 3.5 out of 5.0 at the undergraduate [lower division], and 4.5 out of 5.0 at the undergraduate [upper division] and graduate level), which pretty much put me right there with the vast majority of my peers, countless conversations with colleagues have led me to believe that my evaluations are fairly ordinary.
Obviously, I have received my fair share of, er, “candid” comments (the best one to date remains Good bye – good luck – get high – get … OK, I’ll leave it up to those who are more poetically inclined to guess its ending). In twelve years, only two or three times have I been genuinely hurt by what I read, which I thought was gratuitous, unfair and untrue. On a few occasions, on the other hand, I received comments that were glowingly positive, and that made me feel good about myself, for a little while. For the most part, however, it was all pretty forgettable. These days, I quickly glimpse at my forms, looking for anything unusual and finding instead that they almost look identical to those from the previous year.
I do not mind being evaluated — after all, who the hell am I not to be. I think it is fair to let students voice their opinion, and even do some venting if necessary. I can see how SEI may be sometimes the only way to spot infrequent, but occurring, seriously deficient teaching performances . However, relying exclusively on SEI in order to assess teaching effectiveness is ill-advised.
SEI is often predicated as a way to help faculty improve their teaching, based on the comments made by the students. Personally, I can honestly state that I have not made a single change to my teaching style as a result of feedback that I received from students by way of anonymous evaluation forms. Suggestions from senior colleagues have proven much more useful. It is not that I am unwilling to take advice from students, it is just that I often find it difficult to decipher what evaluations are really telling me, much less act upon it. Student comments are typically ambiguous, inconsistent and whimsical (I suspect this to be generally true of any customer feedback — I would not be surprised if much of it were dismissed as unreliable).
For a physics professor, evaluations are normally important in connection with lower division service courses (i.e., first and second year undergraduate level). When dealing with physics majors or graduate students (i.e., fellow geeks), evaluations are almost like patting ourselves on the back.
Now, whenever I have taught a large (50+ students) introductory physics class (typically for engineers or life science majors) I have consistently observed the following:
1) An instructor is criticized for things that are beyond his/her control. Examples include textbook, course content, make-up exam and course withdrawal policies, as well as the fact that the course is required. Individual instructors have no responsibility in any of that, yet students will put the blame on them, as the persons with whom they are dealing on a daily basis.
2) Almost invariably the most vitriolic comments are directed at testing and grading. Naturally, these are two aspects on which the instructor exercises some (not full) control, but are only indirectly part of classroom teaching.
3) Teaching effectiveness, namely the area in which student feedback is presumably sought most avidly by universities, is one on which students will comment only rarely, and when they do it is very difficult to extract signal from noise. Very often, the same aspect of one’s teaching is praised by some and criticized by roughly as many others. Some students like in-class demonstrations, others think they are a waste of time; some want essay-type tests, others want multiple-choice; some like to see derivations of formulae, others just want more emphasis placed on problem solving, and so on, endlessly. These are all legitimate opinions, of course, but there is no reason for an instructor to act upon a recommendation which only a minority of students would endorse.
The simple truth is, instructors are damned if they do, and damned if they don’t . Obviously, students deserve no blame for doing what they are asked to do, namely express an opinion on the instructor. And none of this would be a problem, if SEI were taken for what they are, namely a way to exercise some quality control, and spot those few, indisputable instances of unprofessionalism and/or incompetence. Anything beyond that is unrealistic. No person or business can expect not to receive a single complaint ever; there is no such thing as “100% customer satisfaction”, and one person’s “best thing after sliced bread” will always be someone else’s “horrible experience”. Only when dissatisfied customers represent a substantial fraction, and/or consistently voice the same concern, is any action really required.
What worries me and many a colleague of mine, is not the notion of being evaluated by students, but rather how SEI will be used by college administrators. One of the most irritating and frustrating experiences consists of being told: As long as even a single student complains, you have work to do to improve, as it is your job to make as many as possible of your “customers” happy. What is that supposed to mean ? It is difficult for a college teacher not to feel, at some point in his/her career, that student evaluations are utilized in a way that is dishonest, or unfair, for the purpose of exercising power over him/her.
How else should a teacher be evaluated if not through SEI, is a question often asked by many a SEI proponent. I do not have a good answer to that, and I suspect that it is probably a good idea to make SEI at least part of the process of evaluation of teaching. But I am nervous about it being the whole thing. Teaching can not be equated to a mere delivering of a good, for which “customer satisfaction” is the only criterion to judge effectiveness. If anything, a more broadly encompassing definition of “customers” is needed, to include not just the student, but also parents, employers (both in the private and public sectors), society at large, whose expectation is that graduates of the institution will be competent.
In many respect, the comparison of a teacher with a physician seems appropriate. Patients certainly appreciate good bedside manners, a friendly demeanor, and good communication skills; ultimately, however, what they really expect of physicians is that they will cure them.
And, curing someone is not the same as making that person think that (s)he was cured.
That is really the only thing that matters in the end, and therefore a physician should be primarily evaluated through this lens. And while people skills are certainly part of the package, no sane hospital administrator would ever make the sole criterion to determine whether a doctor is performing adequately or not whether patients walk out of the hospital wishing to go on a date with that charming physician, or in any case feeling good about themselves regardless of any objective assessment of their health.
Now, teachers have an advantage over physicians, in that they can give an illusion of competence through grade inflation, whereas physicians cannot really give patients an illusion of well-being (not a long lasting one anyway). Eventually, however, even students will come to a crossroad where their preparedness will be dispassionately, brutally assessed. Those in charge of doing it will care very little about student evaluations. An employer who comes to the realization that a freshly hired engineer is not adequately versed in electricity and magnetism (E&M), is likely to come to the conclusion that the school where that person studied is mediocre, and that no more of its graduates should be hired, regardless of how popular with students its E&M teachers are. Likewise, to a biology major taking the MCAT, what (s)he thinks of his/her physics college teacher matters very little for her future — it’s the physics that (s)he knows the only thing that does. I am not sure that the two things are necessarily correlated, and in any case it seems to me that the degree of learning should be assessed independently of SEI , with an eye to standards that society has set, regardless of what each and everyone of us may think of so and so teacher.
Which is to say that, in the end, the “better” teacher is the one whose students, on average, learn more. Duh ?
 Increasingly I hear of universities that actively try to compel students to evaluate their instructors, to the point of withdrawing grades of those who fail to comply. I find this really bizarre. Why force someone who has ostensibly no opinion about something to express one anyway ? The argument put forth by those who spouse the notion of mandatory evaluation, is often that of “customer care”, but I cannot think of any business withdrawing the service from a paying customer in exchange for an evaluation of the service itself. How would you like it if one of these days, the nurse at the hospital should tell you that you won’t get your stitches removed until you have filled out a surgeon evaluation form ? …
 Anecdote time: at the end of a term, I received once quite a few comments from students lamenting the fact that other students would constantly walk in and out of the lecture room during class, distracting everyone else. Even though that was in no way my responsibility, and therefore I should simply have done nothing, because I was young and naive I decided to do something about it. So, I adopted in the following term a closed-door policy, i.e., nobody would be let in after ten minutes from the beginning of the class. Aside from bitter complaints that I received during the entire semester, at the end I got a lot of negative comments from students, calling such a policy unfair.
 Comparing the performance on a course final exam, or on a standardized subject test, of students who were taught by different teachers, might be a way to assess teacher effectiveness.