Wednesday, May 21, 2014

Harvard Stops Offering (Information About) Easy Courses

Today's Crimson has a story that combines several of my favorite subjects: information freedom and information control, student course evaluations, the incentive and reward structure for faculty, and how hard it is in higher education to attack a real problem rather than a symptom.

Harvard will no longer make the "difficulty" rating available to readers of the student course evaluation guide (known as the Q). Concerned that these ratings encouraged students to choose easy courses, Harvard will try to make it harder for students to figure out which the easy courses are.

At the micro level, it's hard to argue too strongly against this decision. Of course students shouldn't generally be choosing courses by difficulty. There are certainly circumstances where it's an entirely reasonable strategy -- for example, when a student has to take an extremely time-consuming course at the same time, or when her physical or mental health makes it hard to carry a heavy load. When advising students in such circumstances, I sometimes use this information myself. Still, the information is doubtless more often misused than used well.

But wait a minute. Since when does a university decide what information to provide students based on our institutional judgment of whether they are likely to use it well or badly? Our entire purpose is to teach students how to use information, not to curate it for them so they won't misuse it. We hate the way people use the US News rankings, but we don't refuse to disclose the information on which the rankings are based. Imagine if the librarians or the health service started to go down the road toward curating information we feared people might misuse.

If Harvard were going to stop collecting the difficulty information on the course evaluation questionnaires, that would be one thing. But the plan is to keep asking the questions of the students filling out the end of term surveys, but not to return the aggregated answers back to the students.

Which raises another issue. What is the purpose of this guide anyway? It bills itself to students as "Your Voice, Your Guide," but it seems the difficulty data will actually be collected from students for use by someone else. Indeed, the student evaluation data are routinely included in dossiers for faculty promotion and tenure, even though the use of these numbers as indicators of educational quality is scientifically more than a little suspect. More than twenty years ago, Ambady and Rosenthal demonstrated that Q evaluations correlate highly with student evaluations of half a minute of video of the instructor--with the sound off! (PDF of the Ambady-Rosenthal paper. Both were at Harvard at the time; Ambady later moved to Stanford, and, sadly, died last year.) The dubious use of Q data in promotion decisions is the source of the fear that faculty will make their courses easier in the hope of getting higher ratings from students. But a better solution to the problem of perverse faculty incentives would be to develop a more respectable way of evaluating faculty. It can be done; HBS does it.

All of which gets us back to first principles. If the worry is that too many students are opting to take easy courses, why don't we try teaching fewer easy courses? That would seem to be educationally more constructive than managing the information about them.

Remember, this is not the first time that easy courses have been an issue at Harvard recently. Gov 1310, the course on Congress that resulted in the infamous "cheating scandal," was a gut. The conversations following the announcement of the cheating scandal were about student academic integrity, the withering of the final examination, the place of athletics in colleges, and lots of other things, but not one word was ever said about the fact that the particular course at the center of the scandal was embarrassingly easy and everybody knew it (or, perhaps, had been until its fateful last year).

Today again, if there are any calls to the faculty to stop teaching easy courses, I haven't heard them. (Not that I would be likely to get such a call. Nor would anyone in CS or in the Engineering School, for that matter.) Not one word has been whispered to the faculty at large that we are being too soft on our students, and we shouldn't be trolling for enrollments by lightening our workload.

With all our discussions about honor this year, wouldn't it have been more honorable to call on the faculty to stop teaching easy courses than to pretend we could prevent students from finding out about them?

The modern university is an information machine. Of course we have always created knowledge and conveyed it to our students. What distinguishes the modern university is our information control apparatus. It is now well established that a basic function of the university is to manage information about itself. It is bad enough when the public communications apparatus colors educational decision making; the more that happens, the less the public trusts that we are really prioritizing education over brand management. But now it seems that information control is an accepted tool even of the academic authorities. O tempora, o mores.

For an earlier post on a related topic, see Information Control: Yale's Power Grab.


  1. I'll be sad to see the difficulty rating go away. I found it quite helpful in my dayl. Any course beneath a 4.0 in difficulty wasn't worth my time. I can't have been the only one who found the rating helpful for avoiding guts.

    So I have to disagree with you that students shouldn't be choosing courses based on difficulty. Of course they should take it into account; the problem comes if they make their decisions primarily based on it, and primarily in favor of easy courses. Today's vote won't change anything of substance: students will always be able to find out which courses are guts from their peers -- or from the professors who let it be known, and none too subtly, in the first class session just how little work to expect.

    This move signals, rather, that Harvard thinks of its students as people who have to be tricked into working hard. But if there's a college in the country where that's less likely to be the case, I'm not aware of it.

    1. Thanks, James! Spoken like a true Harvard computer scientist, even if you are a law professor now!

  2. You mentioned that HBS has a "more respectable way of evaluating faculty." Could you elaborate on this? I'm interested in hearing how it's different from what FAS does. Thanks!

    1. A few years ago, I was told that the pre-tenure protocol worked something like this. This is from memory so it may not be perfectly accurate and it may have changed.
      1. A committee of 3 faculty is appointed to review the professor's teaching. It is given a specific list of things to look for.
      2. The committee meets with the professor to share and discuss the criteria, and to learn anything special about the course and the experience of teaching it.
      3. The committee visits the course and observes the teaching (which of course is generally interactive at HBS).
      4. The committee writes up its findings.
      5. The committee meets with the professor to go over the findings.
      6. A year later, steps 1-5 are repeated, not necessarily with the same committee members. Iterate until tenure review.
      7. The reports are all part of the tenure review, providing not just a series of snapshots but a record of the professor's success at learning how to teach well in response to constructive criticism. Also, when the faculty discuss the candidate, many are present who have seen the candidate teach (that typically is not the case in FAS).
      This is what it means to take teaching seriously, and the results show it.

  3. With regard to students and ``easy'' courses I have seen the following cycle:

    A specialized elective is taught so only students who want to learn it are there. They all work hard and get A's.

    Word gets out that everyone got A's so more people sign up for it the following semester.

    Since they aren't as interested and they are just there for the `easy A'
    the students do not do as well. Some are mad since they thought it would be an easy A.

    Students see that it is not an easy course so they stay away.

    NOW once again only those who are interested take it.

    Lather, Rinse, and Repeat