Wednesday, September 12, 2012

The Thoughtless Evolution of Final Exams, Part II

In response to the last post, several people (including a commenter below) have mentioned to me that take-home finals are not new to Harvard. And they are common at other universities, and indeed in other Harvard schools. So what is different about the College that such a disaster could occur?

First, to fill out the history. Yes, professors have been giving take home exams during Reading Period for years, probably with increasing frequency. Since what happens during Reading Period is not centrally monitored, professors who did not want to deal with the rigid protocols for administration of final exams during exam period were rolling their own exam protocols, in-class or take-home, during Reading Period. The College's decision to flip the default doubtless had something to do with frustration with having to grant so many permissions for exam substitutions. But it was dangerous to respond by letting faculty do whatever they wanted.

Some forms of take-home exam work better than others. The exam protocol for the infamous Congress class was that students had eight days to do the exam; it was posted at the beginning of Reading Period and had to be turned in by 5pm on the last day of Reading Period. So there were students who put off doing the exam, because they were writing term papers, having dinner every night with roommates who did the take-home exam right away so they could study for other exams closer to the time they were taking them. That was a recipe for disaster, under the rules that students were not to be discussing the exam with each other.

An exam that had to be done over a 24-hour period would suffer less from that failure mode. But if that 24-hour period were a particular calendar day during Reading Period, then the lack of synchronization between courses would create the possibility that some unlucky soul would have four of those exams to do in the same 24-hour period. That, of course, is the problem that Exam Period was designed to avoid! By having classes that meet at different hours centrally allocated to different exam slots, no student would have to take different exams at the same time.

Technology has created another solution. Today a professor can post the final exam in such a way that a clock starts the instant an individual student downloads it. Each student must upload his or her response within 24 hours, but the students do not all have to be synchronized with each other. Of course this does not eliminate the possibility of information flowing from student to student, but the structure of the protocol reinforces the message that collaboration is not allowed, in exactly the way the structure of the Congress exam signaled the opposite, whatever the instructions said. (HBS gives exams this way.)

All this raises a larger question, which I hinted at near the end of the previous post. Harvard has been giving exams for hundreds of years. Have we really forgotten how to think through that there are better and worse ways to do it? The protocol in this course was amateurish.

Part of what has happened is that the university administration is increasingly deregulatory---if faculty members want to do their own thing, it seems, we might as well allow them to do it. I know how hard it is to herd these cats, but the collapse of regulations, norms, and even memory is dangerous. For all that we have to say these days about pedagogical improvement, the role of the Bok Center for Teaching and Learning, the Conversations@Harvard series on the future of the university, and so on, there is remarkably little practical guidance to faculty on how to avoid these avoidable disasters.

7 comments:

  1. Actually,I have no problem with giving faculty latitude to give any kind of exam they want. I have given take-home exams with the ability of anyone to take as much time as they want provided that the exam is turned in as final by a specified date. The real problem here it seems is a loose definition and even looser enforcement of the term "collaboration." If a student had a question about an ambiguity in the exam, could they clarify this with a TA or the professor? If so, could they then share this clarification with others, or would this be prohibited collaboration? As long as everyone agrees on the rules on collaboration, and they are enforced, it makes little difference that the test is take home, during exam period, etc.

    ReplyDelete
    Replies
    1. No need in this exam for the student to have asked permission to share the TF's response, since the TF met with students in a group and gave everybody feedback on their proposed answers while the other students were listening. The premise of your last sentence is not satisfied, it seems to me, in spite of the unambiguous exam instructions.

      How about this exam scenario. The exam questions are posted on the first day of class and the exam is due on the last day of reading period, and no one is allowed to discuss the exam with anyone else during the term. Fine with you? I might agree that no rule could be written that would prohibit such a monstrosity, but surely some dean should tell the professor and department chair that such a protocol is idiotic.

      Delete
  2. In the Harvard’s defense, students were instructed to complete the exam without the help of their peers. According to Harvard President Drew Faust, “these allegations, if proven, represent totally unacceptable behavior that betrays the trust upon which intellectual inquiry at Harvard depends.” While Harvard intends on revisiting their academic policy to mitigate future instances of dishonesty, the institution must also establish an effective method to ensure exam questions are more reflective of materials taught within the classroom.
    Than Nguyen
    http://www.examsoft.com

    ReplyDelete
  3. Right. But read my post 2 down for some thoughts on the alleged clarity of the instructions, which are exactly as clear as 60 mph speed limit signs are.

    ReplyDelete
  4. For the record, I've had final exams in both CS153 and CS51 (the two undergraduate courses I've been teaching). But I've seriously considered getting rid of them in favor of a final project, paper, or programming assignment.

    There are three primary reasons for this: First, I worry about the stress that exams seem to cause some students. Perhaps having the freedom to do an assignment/exam/paper etc. when and where you want, with plenty of time is better than stuffing you in a room where you're not allowed to go to the bathroom without waiting a half hour, and signing a sheet.

    Second, for these courses at least, it's quite clear to me that most of the learning happens when the students are actually writing and testing code. And writing code in an exam isn't really feasible (at least now.)

    Third, my father and sister attended a school (Davidson) which seemed to have a strong honor code, and I feel like where I went (Richmond) had a pretty good version of that code. As a student, I remember feeling like the faculty trusted and respected me when they gave me a take-home exam. In turn, that engendered respect from me and I strove to follow the rules that were laid out.

    None of this provides a good empirical justification for abolishing exams (not arguing that), but I do think that many faculty have given this serious thought and chose to move away from exams for good reason.

    ReplyDelete
    Replies
    1. I agree of course that alternate forms of assessment are sometimes more educational and more useful. Even if the rules allowed it, I wouldn't give an exam in my freshman seminar because the research project really pulls the course together. On the other hand, if I had to have cancer surgery and I had a choice between getting it from a surgeon whose anatomy class had a closed-book exam and one whose anatomy class had a week-long, open-everything-etc.-take-home-exam with "no collaboration allowed," I'd go with surgeon #1. Call me mistrustful.

      Honor codes are another issue. I have a philosophical objection to requiring students to swear to oaths, even if it can be shown that they "work." I think when you matriculate you agree to all the rules of the place, and nothing else is needed. In the case at hand, the oath would not have helped matters because many of the students accused of cheating did not think they were cheating. I don't like the idea that the "solution" to this problem is to impose on the honest students an obligation to snitch on their peers.More generally, honor codes seem to work pretty well in general but are subject to catastrophic failure, when the trust between the students and the professor breaks down. But all this, I suppose, we will inevitably have a chance to talk through.

      Delete
  5. An exam is an exam is an exam - in a room with invigilation and no way for students to communicate.

    If you don't want the hassle of conducting actual exams, don't try to blur the definition of what an exam is - "take-home exam", "honor code exam", or whatever.

    Just use different forms of assessment that meet the requirements of their situation - as mentioned above, projects, assignments, reports, etc.

    These also help to foster communication between professors and their students, so cheating becomes less likely, or more easy to avert. And if profs are too busy writing papers to meet with students, then they shouldn't be teaching in the first place.

    That won't stop actual cheating, of course, but if an absolute, individually sealed assessment of students is required, then don't be lazy - go back to giving traditional exams.

    I'm not saying that exams are necessarily a good form of assessment, but they do meet the requirements of a certain *type* of assessment.

    ReplyDelete