I want to know, what you are thinking….

Imagine the following scene: you are vacationing at a five-star resort. It is costing you a fortune, but it is what you saved money for, and hey, we only get to live once, right ?
Your stay is ending, it’s your last day. You want to use the fitness room and put yourself through a vigorous workout, before you check out, at 11 am. As you walk into the fitness room, however, you are stopped by resort personnel, and asked to step outside. Surprised and bewildered, you ask for explanations; you are politely but firmly told that you won’t get to use any of the amenities, in fact not even the TV in your room — the only thing that you will be permitted to do, from then until check-out time, is fill out a customer satisfaction form.

As you hear those words, at first think that it must be a prank. You cannot believe what they are saying, they cannot possibly be serious. When you realize that they are, you cannot help becoming angered. You demand that access be granted to what you are paying top dollars for. Filling out a customer survey is not one of the leisurely activities in which you had planned to engage during your vacation. You would be happy to fill out one of those forms once you are back home, and send it wherever appropriate — but you do not want to do it now, you really want to use the fitness room…
Those people don’t budge, though. They insist that customer feedback is of the outmost importance to the management, in order to make the experience of the resort guests always more enjoyable. Because most guests leave without bothering to fill out a form, and because the management simply cannot take that as evidence of customer satisfaction, the policy has been adopted of denying any kind of service during the last few hours, in order to motivate each customer to furnish the constructive criticism that the management values so much. So, you have no choice but sit down and fill out the form… or leave ahead of time.

That sounds crazy, doesn’t it ? No hotel, no museum, no supermarket, no business would ever operate in that way, I think. One thing is giving customers the opportunity of providing feedback, including making complaints about sub-par service; one might even consider offering incentives such as tickets to events, in order to entice patrons to express their opinions. But, to hold service for ransom, in exchange for essentially extorted comments, over aspects about which one may not feel strongly enough to express any views anyway ? That seems too much. One might also expect that, due to irritation and lack of interest, a customer pressed for feedback, required to sit down and just write something about her experience, may not be in the position to make the most useful remarks. I am no customer service expert, but I am an avid consumer and traveler, and can only recall one instance of such “forcible feedback” [0]. My impression is that business is far more sensitive and attentive to unsolicited comments, either positive or negative.

So, why is it that universities very much insist with doing just what described above ? Why is it that, in the digital era, when few things are easier than setting up a web site where anonymous opinions can be collected about anything or anyone, twenty, thirty, forty minutes of class time must be devoted at the end of each and every course, to the administration of in-class student evaluations ?
Last week, on my last class, forty-five minutes of lecture time were utilized to have the several hundred students registered in the course that I have been teaching this Fall fill out evaluations (yes, it does take that much time when there are 400 students). During those forty-five minutes, I could have covered parts of the syllabus which I did not manage to cover in the end (I teach twice a week, each class is 75 minutes), or I could have worked on preparation for the upcoming term exam, or gone again through topics that seemed unclear to some students.
But, noooo, the pressing need of my institution for student feedback took priority over delivering to the “customers” what they actually paid for, namely instruction time. Does this make sense ?

Whenever I bring up the subject, and mention the scenario of the university setting up a web site, where students could log in anonymously, and say at their own leisure whatever they want to say about a professor (much like it is done here, with the obvious advantage that the university could set things up so that only legitimately enrolled students could express an opinion), the response that I invariably receive is “oh, but then most students would not bother to do it”. Yes, I agree, they probably would not bother, in the vast majority of cases … and the problem with that would be ?

I remember back in the long gone days of me being a student, I rarely encountered professors about whom I would have wanted to say anything particularly good or bad. Don’t get me wrong, some of them were lousy, and should have been told to work harder and do a better job, while others were really superb. For the most part, however, they were just honest, competent, serious professionals, conscientiously doing their job. I did not feel the need of going out of my way to express an opinion, I believe for the very same reason I rarely fill out customer satisfaction forms at hotels (when I do my wife does it, it is usually because something has gone seriously wrong), or ask to speak to the manager to complain about an employee at the supermarket, or call the dealership to let them know how inspiring that last oil change was.

And all of that is a good thing. It means that most of us take our job seriously, and do whatever it takes to do it well. This is why, even though on occasion we are disappointed or upset by someone’s mediocre performance, or greatly impressed with the professionalism of a specific individual, on a normal day most of us do not spend any time congratulating or thanking anyone — not only because there is no time, but because there is no reason to congratulate workers for simply doing what they are supposed to do, and for which they receive a regular compensation. That is the reason why many businesses frame and hang on the wall letters from enthusiastically satisfied patrons — because they are rare. That is what makes them valuable. It is the same reason why exceptionally effective professionals in all walks of life (lawyers, physicians, football players, business owners) receive prizes, accolades, are held as examples, make more money — because most of us are not exceptional. But that does not mean that most of us are “bad”.

There is nothing intrinsically different, in my view, about teachers, especially if “customer satisfaction” is used as the main motivation for doing student evaluations. The notion that a good instructors must be perceived by students as “an inspiration”, her course a “life changing experience” and so on, is disingenuous and out of touch with reality. Of course, some of them are, and they should be properly rewarded and held as example for their colleagues. But the plain truth is, most of us will have a hard time even remembering all of our teachers a decade later, even though the benefit of having been taught by them will stay with us for a lifetime. If a sizeable fraction of the students do not bother to write comments on their student evaluation forms, or even walk out of the room without filling them out because they have something better to do, there is no reason to see that as an indictment of anything or anyone — they simply had nothing to say. Insisting for them to say something, when they have nothing to say, is not conducive to useful feedback. Attempting to frame their lack of interest as somehow indicative of scarce enthusiasm or appreciation for the teacher is downright dishonest. It is all too easy to suspect that there is in fact a different reason for university administration to insist with this policy of “forcible feedback”, and that is has little to do with customer satisfaction and more to do with establishing a hierarchy, building cases against teachers.

Notes

[0] It happened recently at a car dealership. As I was ready to drive away my new car, the salesperson with whom I had dealt came to me and asked me to fill out a customer satisfaction form, in which I would rate my experience with her (i.e., that particular salesperson). I said that it was a bit inconvenient to do it then, that I could do that at home and send the completed form to the dealership, but she (clearly a person new on the job) insisted that I do it there, and would not give me the car keys unless I would fill out the darn form. Even more irritatingly, she demanded that I do it before her very eyes, and yes, on each question she would suggest that I check “excellent” as a response. I did not want to argue with her, I wanted to drive away, and so I did as she told me. As soon as I returned home, I phoned the dealership, asked to speak to customer service, reported my experience with that salesperson, stating in no uncertain terms that I felt that such a way to obtain feedback was ridiculous, unprofessional, and disrespectful to the customer.

Tags: ,

15 Responses to “I want to know, what you are thinking….”

  1. KJHaxton Says:

    I have done just this for one of my courses this year – set up an anonymous survey on WebCT instead of a paper evaluation. So far the main problem seems to be getting a high enough response rate – I need 60 or 70% ideally because that’s roughly what I’d get if I did it in a teaching session. The students can fill it out at their leisure but I need to think a little more about motivating them to do so. I’ve also set up a couple of generic evaluation boxes as part of our student staff liaison which allows students to leave anonymous comments at any point of the course.

    I much prefer the idea of not wasting teaching time to do this but the powers that be require (and various other forms I have to fill out to evaluate the course require) a good response. Should I offer a prize draw for all responses or should I just accept that sometimes people don’t want to fill out 8 evaluations a year and don’t want to provide feedback.

    Like you, I think we need to change how we approach evaluation – if we have systems in place where students can pass comment if anything specific comes up, why do we need to force them all to comment on things that are just standard?

  2. Ian Says:

    I wonder if you could use the WebCT option (we use that in some courses at SFU) for feedback but also grant 1% of their grade as an incentive to fill it in.

    The only other main objection I saw to the evaluations was that they always came before I got my final exam and maybe more importantly, before I got my grade back. Of course some will complain worse if they got a bad grade, but it may be legitimate (my graduate quantum class was curved to a B- average – meaning people needed to do well in their other course or else they were on thin ice in the department).

  3. James Says:

    Without a measurement, how can one objectively judge the quality of their teaching? It is fair to argue that the measurement tool currently being used at your institution is insufficient (it probably is) but, unless I am misunderstanding the thrust of your post, you are arguing against making the measurement and not against the validity of the measuring instrument in use.

    Starting with the premise that the instrument suits the measurement, an assessment is required for instructors for the same reasons that an exam is required of the students (i.e., you can’t know for sure what is their level of mastery without measuring it).

    If an instructor changes, for example, how they present content (e.g., blackboard versus powerpoint slides; direct lectures versus clickers and peer discussion), s/he cannot know whether that change was an improvement upon the old style, perceived or otherwise, without asking the students. Presumably, these students are taking the class for the first time, and so simply don’t know any differently. I do not think one can conclude that because they don’t volunteer feedback on your use of, say, powerpoint slides that they don’t have feedback of value concerning powerpoint slides. They might not know that powerpoint slides are something you even care about. But if one compares survey results from consecutive years (and makes the reasonable assumption that the demographics are comparable), an objective measure can be made: the instructor can know if the changes were positive or not. It informs your teaching, which can very naturally result in greater learning gains for your students.

    Also, once you have data on what or how students think, it becomes infinitely easier to convince colleagues that something does/doesn’t work in the classroom. I can tell you I think that a certain tweak to my course was beneficial, but you probably won’t buy it until you see some cold, hard data supporting my claim.

    In short, our pedagogy should be evidence based. Shouldn’t it?

    • Massimo Says:

      James, my post is not about the need or the wisdom of having students evaluate teachers. It is about whether all students should be compelled to do so, even when they are not interested in doing it and even at the cost of hijacking lecture time for which they have paid.
      My belief is that the feedback that is useful is the one that concerned customers offer willingly and unsolicitedly, not extorted from the ones who are uninterested in providing any. I think any business outside academia abides by this principle.

      And I also question the argument according to which in-class evaluations are more effective in getting the feedback being sought. In partial support of my point, I respectfully direct you to this piece of research, showing how the ratings from RateMyProfessors appear to correlate with those obtained in class. So, my question is: why not simply set up a web site and let those students who are interested in providing feedback say whatever they please, and let me finish my syllabus ?

      • James Says:

        That RMP versus SET paper is great – thanks for bringing it to my attention! I am part of a reading group that I think would love to discuss this work. I am interested to know if they have replicated this correlation… If it turns out that a few data points on RMP can indeed tell a prof what a few hundred SETs could, then I guess my argument is significantly weakened (and this answers your query about instead using a web site to collect unsolicited feedback).

        That the feedback which is useful is the one that concerned customers offer willingly and unsolicitedly is, to my knowledge, an open question (and we seem to disagree about what we think the answer is). I also do not know whether your characterization of student time being hijacked or feedback being extorted is one you present to strengthen your case or if it is an accurate reflection of how the students typically feel.

        I’d like to say, though, that I usually agree with most (if not all) that you write on this blog – it’s refreshing to have my panties in a bunch every now and again.

      • Massimo Says:

        Well James, my question is — if feedback obtained “forcibly”, even at the cost of withdrawing the good being purchased (and I think you’d agree with me that students are paying to come to class and listen to a lecture, not to fill out a questionnaire) is so valuable and important, then why don’t hotels and supermarkets do the same ? Are they less concerned about customer satisfaction than universities ?
        And I think that an established, robust correlation between RMP and SEI does disprove the contention that feedback has to be actively sought out…
        Anyway, thanks for your comment and your kind words.

        PS I have to tell you, I dread the day the nurse tells me sorry, the surgeon told me not to administer any pain killer until you have filled out the hospital evaluation survey…

  4. transientreporter Says:

    It must be the end of the semester if Massimo is complaining about evaluations… AGAIN!

    I would take issue with the comparisons to private businesses. A university is not a shopping mall. Customers are usually not the ones paying (their parents are, or the state is), and not all that interested in the product. Many don’t seem to know what the product is.

    • Massimo Says:

      Awe, c’mon, man, you LOVE my student evaluation rants… and it is not about the evaluations, I am perfectly all right with them, it is how they are administered and used that bothers me (sometimes).

      I would take issue with the comparisons to private businesses.

      Me too. It is absurd and ridiculous. But, it is that upon which much of the SEI rhetoric is based, hence my insistence on it.

  5. Calvin Johnson Says:

    Now we’ve gone to online evaluations–essentially when students go online to get their grades, they can (?) must (?) (I’m not sure) rate the class. The opportunity to do so is after grades are posted but before they actually view the grades.

    While I share most instructor’s feeling that student evaluations are generally not terribly insightful as to just how effective an instructor actually is–I agree that the view that the student is the “customer” is misguided and even harmful– this seems the least bad of the options.

    • Massimo Says:

      Calvin, I’m pleased to hear that SDSU is breaking ground on this.
      Make no mistake, I am not advocating in the least doing away with SEI, and maybe i should have stated it upfront. I do think that SEI are necessary. Short of placing video cameras in the classroom, I don’t see how else blatant, egregious cases of unprofessional behavior or misconduct on the part of teachers can be spotted.
      The issue here is not whether or not to have students express their views, it is whether that is best done in class during lecture time.

  6. Hope Says:

    Awesome post! – and I liked the last one, too. Is it sad that I knew where this was headed by the end of the 1st paragraph?

    A friend of mine is at a school that has been doing online teaching evals for the past two years or so. Students can fill them out anytime during the last three weeks of the semester, and this includes exam week. The participation is a bit lower than the in-class compulsory version, but not significantly so. Students who submit evals online get access to their grades a week before everyone else, and apparently, profs make a point of encouraging students to fill out the online evals.

    To me, this sounds like a good system. I always hated having to come up with meaningful constructive criticism on the spot.

  7. Cath@VWXYNot? Says:

    You know that by inserting a link to Rate My Professor (which I’d never heard of before) in your post, you’re practically inviting people (i.e. me) to immediately go there and search for you by name, right?

    Also, are you kidding me about the car dealership???!!!

  8. Cath@VWXYNot? Says:

    So apparently you are “the man”, but you often forget your black marker.

    • Massimo Says:

      Actually, it was not my job to bring markers with me, they were supposed to be in the room — problem is, the !@#$%& who was teaching before me walked away with a pocket full of markers. Of course this year with the removal of the white board it all became a moot point. And no, I am not kidding in the least about the car dealership — but I am told that it happens more frequently than I thought. You’ll see next time you shop for a car…

  9. Silver Fox Says:

    With the car dealership, I’d recommend sending letters to that person’s boss at the dealership and to the head of the car company (after getting your keys, of course).

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 28 other followers

%d bloggers like this: