Monday, December 22, 2014

Was It Really the North Koreans?

Writing in The AtlanticBruce Schneier is skeptical, not that he has a better idea, though he does lay out some other possibilities. But he reminds us that the government has not always gotten stuff like this right in the past.
I worry that this case echoes the “we have evidence—trust us” story that the Bush administration told in the run-up to the Iraq invasion. Identifying the origin of a cyberattack is very difficult, and when it is possible the process of attributing responsibility can take months. While I am confident that there will be no U.S. military retribution because of this, I think the best response is tocalm down and be skeptical of tidy explanations until more is known.
 Also, on the general question of whether this means that anybody can break into anything, Bruce writes in the WSJ, the answer is no, but anybody can break into something, and it's possible for an entity with enough resources to break into almost anything.

And in a third piece Bruce offers another piece of advice that is as good here as it is generally: the first thing to do is not panic.

Wednesday, December 3, 2014

A Phase Transition

I have been named Interim Dean of Harvard's School of Engineering and Applied Sciences. What I say in the official announcement is all true -- it's an honor and a privilege. How many people get to take the leadership role of a place to which their first connection happened almost exactly fifty years earlier? In the fall of 1964 my freshman advisor was a professor in the old Division of Engineering and Applied Physics. I remember feeling mildly insulted. An ENGINEER? I was going to be a pure mathematician! I was disabused of that fantasy by Math 55, and a couple of years later wound up in Applied Math where I belonged, and I have had some sort of SEAS affiliation ever since. Boy, the Freshman Dean's Office was good at assigning advisors (it's still done well, but no longer by the FDO).

I hope the blog won't go completely dark, but this job is going to consume all my time, and more, for the (I hope) brief period while I hold it. And yes, the subjects may change -- for certain things I might once have blogged I will now just pick up the phone to start an inside-Harvard conversation!

And NO I AM NOT A CANDIDATE FOR A PERMANENT POSITION AS DEAN. Harvard and I may both be crazy, but we are not stupid.

That said, two good op-eds in the NYT today:
Blowing Off Class? We Know (on big data and academic affairs, which it's interesting to see how other places are thinking about)
A Pox on Campus Life, in which Frank Bruni talks as though he read the 1994 Report on the Structure of Harvard College in which the committee I chaired recommended randomization of the Houses.

Wednesday, November 19, 2014

Our Anti-Business Pro-Business Conservatives

Josh Barro had a great column in the New York Times a couple of weeks ago about how schizophrenic the Republican party can seem about whether it is really the pro-business party or not. He cites the examples of Uber, and the attempt to prevent it from operating in Philadelphia, and of Tesla, which is opposed by the cartel of car dealers, since Tesla wants to sell directly to consumers. Here is the bottom line.
Anticompetitive business regulations are mostly imposed at the state and local level, and they usually have a strong built-in lobby: the owners of the businesses that are being shielded from competition.
The R.N.C. chairman, Reince Priebus, probably doesn’t get a lot of phone calls from taxi medallion owners, or car dealers, or other businesspeople who want to be insulated from competition.
But local politicians do; Republicans may be especially likely to hear from them because small business owners are a constituency that skews Republican.
As a result, in practice, it’s not clear Republicans are any more pro-market than Democrats when it comes to business regulation.
Now this is maybe not the best moment to to be touting Uber as a model unregulated small business, what with an executive seemingly power-mad over his ability to track his customers. But the bottom line stands. You either believe that competition lowers costs and improves services or you don't. If you do, you don't bring the government in every time an existing monopoly cries foul over a new entrant.

In the same vein, the Republican pro-business mantra doesn't seem to extend to the businesses that won't be able to sell their information services abroad if the rest of the world thinks they will just turn everything over to the US Government. In spite of the business arguments for the anti-surveillance USA Freedom Act, Republicans voted overwhelmingly against it. (Including Rand Paul, who, to give him credit, says he opposed the bill because it did not go far enough toward reining in the NSA.)

And the final example of the day is provided by George Leef in Forbes: Copyright Law Is Creating An Information Oligarchy, Not An Information Democracy. As Leef says,
Today, copyright does far more to create an information oligarchy than the robust information democracy the drafters of the Constitution and the first act had in mind.
I probably wouldn't go as far as Leef proposes in dismantling copyright completely, but it is so abused today that it's hard to argue we wouldn't be better off without it than with it under present law. Leef is at the John Pope Center for Higher Education Policy, where I have spoken in the past, a right-leaning education think-tank. I probably agree with what he writes no more than half the time, but he is onto something important here: it's insane how heavily copyright is wielded by the information monopolies to swat down the little guys, whose energies are supposed to be protected and encouraged by the party that allegedly so hates big government. Please explain to me how the progress of science and the useful arts is encouraged by a copyright term so long that Disney's original Steamboat Willie (aka Mickey Mouse) is still protected. (And it wasn't really original in the first place. It was based on an earlier cartoon, but that is a story for another day.)

"Codebreaker" and "Ivory Tower"

I've seen two good documentaries lately, Codebreaker and Ivory Tower. Neither gets a straight A from me, but they're both worth watching.

Codebreaker is the story of Alan Turing, the founding father and patron saint of computer science. Turing died of suicide at age 41 in 1954.

The documentary does a good job contextualizing Turing's achievements and impressing on the viewer his intellectual daring and the massive significance of his work, without getting bogged down in the whole history of mathematical logic (for a light version of which, see Logicomix). It also sets in Cold War context the brutal treatment the unworldly Turing received at the hands of the authorities once his homosexuality was discovered (he was chemically castrated). The filmmaker was able to interview some people who knew Turing -- that number is of course rapidly declining. It's very well done.

The problems with the film are almost inevitable, given that it's a documentary and therefore tries to stick to the truth! (Unlike The Imitation Game, the Hollywood version of Turing's life that is in theaters next week.) There is just not a lot of material to work with -- no films or audio recordings of Turing, few still images, and virtually all of Turing's friends dead now. So a lot of the story is told through Turing's conversations with his psychiatrist. Of course the dialog is reconstructed, but the reconstruction is grounded in solid source material, letters and so on. (The film's creator, Patrick Sammon, answered questions after the showing at Harvard last night. Sammon, I was interested to learn, is past President of the Log Cabin Republicans.) And of course the budget was limited, so there are no fancy animations, though there are quite a few clips of contemporary video to set the general themes in their historical setting.

Codebreaker is showing at Tufts tonight and is available through Netflix and iTunes. If the movie gets you interested, read Andrew Hodges's biography of Turing, Alan Turing: The Enigma. (Turing's life certainly provided material for plenty of good titles!)

Also I want to again plug Ivory Tower (see my earlier blog post), the documentary about student debt that portrays Harvard so positively. CNN will air Ivory Tower Thursday night at 9pm, so you can watch it from the comfort of home!

Friday, November 14, 2014

Can We Find a Better Way to Rank Students?

I have been complaining about GPA for years. I don't care that it's an inconstant measure -- it has been drifting upwards pretty much since the day it was invented, and there is very little reason ever to compare GPAs of today's students against GPAs of students a decade ago since they are almost never running against each other for anything GPA is used to calibrate. I don't even care much that it is compressed; in fact, that has some benefits, is it makes it easier to justify ignoring it to focus on other criteria instead. The problem I have with GPA is that even if it were constant over time, it would be almost meaningless as a measure of academic excellence, much less any of the more important kinds of excellence. 

Giving GPA the official status it has disincentives ambition. It discourages the pursuit of excellence by encouraging the pursuit of grades in a curriculum that is largely elective. When faced with a choice between two courses, the decision strategy that tends to maximize GPA, which we say we value, is clear: take the course from which you will learn less because you already know more of the material it teaches. 

I am in a very fortunate position as Director of Undergraduate Studies in Computer Science. I can tell students with complete honesty that they are better off ignoring GPA and not worry about getting Bs and Cs. Unless you apply to graduate school, I say, no one will ever see your transcript. In interviews, tech employers may just give you a problem to work on and see if you can solve it. Whether you can solve the problem is less correlated with your GPA than with whether you took challenging courses. You shouldn't let the pursuit of credentials get in the way of getting an education. And by the way, even if you DO apply to graduate school, a faculty letter praising your senior thesis is going to be more useful than a straight-A record.

I can't do anything about the way law school and medical school admissions committees screen applicants and I am not about to try. But maybe we can do something about the honors the university itself gives to high-GPA students. Just including the GPA on the official student record signals our institutional reverence for it. I can't object to supplying students with that information since we keep using it in the various ways we use it -- for example, in the award of graduation honors (cum laude, etc.). But the stupidity of the metric really hit me this fall as I sorted students for two prizes, one for freshmen and one for seniors, both aimed at rewarding true academic excellence.

These prizes are not given just for high GPA (we do have one of those too, the Sophia Freund Prize, given to the highest GPA summa -- in recent years it has been snared among multiple 4.0s). For the prizes I am talking about, GPA is used to create a pool several times larger than the number of prizes to be awarded, and then the committee reads transcripts, letters, and other supporting material to pick out the real intellectuals. The process works pretty well because the faculty on the committees take the job seriously. But several things have become evident to me.
  • Very high GPA is highly correlated with good pre-college preparation. That is, the vast majority of the pool seems to consist of students who had the good fortune to go to excellent high schools, public or private. The best public schools are either in high-income zip codes, or they are exam schools. Some of the independent school are graduating well-prepared low-income graduates, but you don't see many students in these pools from public schools in low-income zip codes.
  • Because of the compression, any one B will knock you out of contention, so freshman-year grades are among the major criteria on which, de facto, these honors are awarded. Freshman year grades tend to be lower not just because students are adjusting, but because freshmen take more large courses and grades in large courses tend to be lower.
  • Most of the transcripts were pretty easily classifiable as "hard core" or "elementary," with only a few that required more serious scrutiny. By "elementary," I mean some perfectly good course programs -- let's say, Math 1, Ec 10, Spanish A, Expository Writing, and a Freshman Seminar. Nothing wrong with that program if you landed here from one of the many American high schools that does not teach AP math and from a part of the country where they don't think foreign languages are important. But not the sort of program that should win you any prizes for superior intellectual achievement, even if you got a 4.0. The increasing variance in socioeconomic background of the Harvard student body may be making those transcripts more common, I'm not sure.
What do I mean by "hard core"? I got that phrase from Ballmer's CS50 talk. One of several pieces of good advice he offered students was to be hard-driving, intense, focused, hardworking, passionate about things. He talked about Taking Physics 55 (then, as I recall, a physics analog of the legendary super-honors Math 55 course that still exists) and getting a 33/100 on the first exam.

Fact is, the committees looking over student records can judge, reasonably well, which are hard core and which are not. Faculty at least can make that judgment for the courses in their own area. But those judgments are not easily automated. Some courses with graduate numbers are not hard core and some courses numbered less than 100 are hard core. "Everybody" knows that Math 55 is hard core but CS 20 is not. (CS 20 is a great and important and highly educational course. But it's not the course to take if you want to convince me that you are going to win the Turing Award some day.) "Everybody" knows that CS 161 is hard core (that is so well known in the tech industry that even interviewers who didn't go to Harvard listen up when interviewees say they took it) and CS 171 isn't (it's just a hugely educational course that EVERYBODY should take!).

Not every course a hard-core student takes is going to be hard core. In fact one of the blessings of guts at Harvard is that they make it possible for normal students to be hard-core some of the time, and   taking just one hard-core course can be life-changing. So when I see a transcript, I sniff at courses taken pass-fail, but I don't mind seeing well-known guts if the student took something hard core at the same time. 

So my question is, rather than fruitlessly trying to normalize grading (as Princeton just gave up doing) or trying to compute GPAs in a way that takes into account the grading curve in a course or the grades in other courses of the students taking the course, can we come up with something better, that incentivizes ambition as demonstrated by a hard-core transcript -- or even a "beautiful transcript," as Professor Elaine Scarry put it to me once? I don't want to automate the whole process of rewarding students; letters, essays, and so on are important. But I don't like the idea that students with only basic coursework are crowding out of the pool other students who have wound up with blemished records because they really stretched themselves to the max. Can we socially engineer a "hard coreness" rating for courses? What would be the incentive for students to rate courses honestly, for the Lampoon not to troll the rating system, and so on? Would faculty refuse to go along with this because they would find it too stigmatizing to have their courses classified as not-so-hard-core?

Of course the other way to handle this would be to stop giving those prizes. That ain't going to happen, but I'll leave all that for another day.

Thursday, November 13, 2014

Great Day for Harvard

Not just for Computer Science at Harvard, but that too. Steve Ballmer has just challenged us to become the #1 place for computer science research and education. We have lots of loyal alums, but not many are so determined to move us forward and so ready to put their money where their mouth is. Ballmer is funding twelve faculty slots, so we will grow from 24 to 36 full time professors. That is an amazing commitment, and I am deeply grateful and deeply humbled, because it's now over to us on the faculty to make it happen. No more complaining that we need more faculty; now we just have to hire them. Speaking of which, we are running a junior search right now. Want to be in on the ground floor? We'd love to hear from you!

This is the 51st year I have been associated with Harvard. I entered as a freshman in the fall of 1964. I fell into computer science before that was the name of anything official here. The bottom of the SEAS web site that was put up following today's announcement has a picture of me fall of my senior year, demonstrating my senior thesis to Applied Math 201 -- I was showing conformal mappings in the complex plane. (I am pretty sure this photo was taken by Bob Sproull, my Harvard classmate and another acolyte of Ivan Sutherland, who went on to be a Sun Fellow.)

By the time I joined the faculty, 40 years ago, Sutherland was gone. Computer Science existed at Harvard, but was not a priority. It was as though the Sutherland experiment had not paid off, and Harvard was going back into its more natural, more cautious mode. The first time I raised in a faculty meeting the idea that we should have a CS major was in about 1978, when I was a nontenured associate professor. Bernie Budiansky, a brilliant applied mathematician and mechanical engineer, snorted, "We've never had a major in automotive science, why would we have one in computer science?" I didn't raise the question again until my personal situation became a bit more secure a few years later. The major must have started in 1983, and I know the first degrees were awarded in 1984. Oren Etzioni, professor of CS at the University of Washington, swears he was the first CS major, which may well be true -- he may have been the first person to walk into my office and declare himself as a CS major in 1983.

Over the years we've produced an incredible series of graduates -- and non-graduates, such as Gates and Zuckerberg. The talent pool is the best in the world, but the "department," though it has improved steadily and has had a few people at the top, hasn't as a whole been at the top. (We have no departments in SEAS, just informal caucuses we call "areas." That is a huge plus for us -- it greatly reduces the amount of internecine warfare, and encourages collaboration in a discipline that is increasingly "outward-looking.")

What a change is happening! We already have a superb group of 24 faculty, brilliant and collegial, devoted to education, spanning the field, and branching into the life sciences, law, economics, and other disciplines. We now set about to change the landscape by hiring twelve more. I feel like the "next wave" just washed over me, because over the past three days I have signed more than 100 sophomores up to be CS majors (the deadline was yesterday). I am engaged in the process of planning our new building in Allston, which will be the center of an innovation hub and the promised land, not just for CS and engineering, but for other (as yet undetermined) Harvard departments that will be moving also, and for an enterprise zone that will grow up around us. CS will be at the heart of it, as it will be in so much else that lies ahead.

We are going to be #1. And the competition is going to make every other department better while we're getting there. And that will be good for Boston, for the US, and for the world.

Look at the slide show at the bottom of the "Catch the Wave" page about the future of CS -- there are a couple more that relate to me, as well as a couple featuring Henry Leitner, and one of Al Spector as an undergrad too. Bonus links: Steve Ballmer hilariously advertising CS50, the way he once advertised Windows  1.0; and Ballmer's CS50 lecture yesterday, with a whole set of good life lessons. Anyone who complains that Harvard students spend too much time on extracurriculars and not enough time in the classroom should watch that class, and then, if they dare, call Steve and tell him that he wasted his time here! And watch the video of the dedicatory program at the i-Lab -- to see Ballmer's inspirational fight-song about Harvard CS, and also to hear the spectacular speech by Harvard CS undergrad Ana-Maria Constantin, who reminds us how important it is to have a program worthy of our students.

Wednesday, November 5, 2014

Vice Provost Bol's Prepared Response to My Question

In my previous blog post, I noted that I was not authorized to comment on the Vice Provost's response to my question. I did not mean to suggest that I had been denied permission to comment, only that the protocol of faculty meetings does not allow quotation or paraphrase without permission of the speaker, and I had not sought it. It has been my practice, when something in a faculty meeting seems worthy of comment, to wait for the professionals to obtain permission and to report on it, and then to base my blog on the public reporting rather than on what I observed from being there.

I am grateful to the Vice Provost for supplying the text below. In judging this response to be unsatisfactory, I meant that it did not answer my simple yes-no question: Will students be informed that they were under surveillance? After the meeting, the Vice Provost told the Crimson that students would be informed, which settles the immediate question in a satisfactory way, for which I am grateful. I am sure there will be plenty of "next time" and "coulda-shoulda" discussions in various fora, but the question I asked has now been satisfactorily answered and I don't expect to comment further on this matter.

See also: Writeups in Harvard Magazine, the Boston Globe, and the Chronicle of Higher Ed.


To answer Prof. Lewis’s questions I want to give a bit of background.

A year ago, on taking the position as Vice Provost for Advances in Learning, part of my brief was to support the growing interest in improving teaching and learning across the campus. I began to wonder if there was a growing disconnect between how students were choosing to spend time and the expectations teachers had of their students.

Over the years I had heard colleagues assert that students in increasing numbers were skipping class, that the amount of work done outside of class (with some very notable exceptions) was decreasing, and that there was less rigorous note taking. Such anecdotes raised questions about the effectiveness of lectures as a way of helping students learn and suggested that there might be some value in exploring how new media and pedagogical techniques might be used by faculty to turn the lecture into something that was more interactive and engaging rather than simply an exercise in listening.

However it turned out that we did not have any data to support the anecdotes. I thus looked for a way of getting data on attendance, because that seemed to be the only thing that could be measured in a straightforward way that did not rely on self-reporting. I am told that there are no published multiple-course results on objectively-measured attendance to rely on.

But in designing such a study there were some very important considerations. We did not want to bias the sample. We did not want individual students to be tracked or in any way identified. And we did not want the results to be used for the purpose of evaluating the teachers. We wanted to know if we could get valid evidence on attendance, and we wanted to see if there were any patterns in the data that might support conclusions about whether or not we should care.

The protocol was sent to the Committee on the Use of Human Subjects in Research – this is the Institutional Review Board, the group responsible for deciding if research uses human subjects and reviewing that use to make sure that they are line with regulations -- which concluded that the study did not constitute human subjects research. It thus did not go to the full committee for review. The protocol was to install a camera that snapped images of the audience in a lecture hall. The images were processed through a program that counted whether seats were empty of filled. The quantities were calculated for each lecture. Once the data was in hand I made appointments, beginning in August, with course heads (two are still outstanding) to tell them what had been done and to show them generalized numerical data on their respective classes. At that time I ordered that images of students be destroyed. The course heads were asked to decide what should happen next. The course head could choose to have the numerical data removed from the study and deleted permanently. The data could be maintained without identifying the course. The data could be maintained with the identity of the course. There could be discussions with the researcher to better understand the data and consider ways of improving outcomes if so desired.

I can report that every single person I met with thought the data was interesting and potentially useful, agreed to the use of the data keeping the identity of the course, and was interested in learning more about the research. Faculty do care about their classes and their students.

The analysis did reveal patterns in the data (patterns that made sense once they were found). The results of the analysis are being shared with course heads. The aggregated data, without identifying courses, has been presented at Harvard to people interested in teaching and learning issues. Here I will only note that there was great variability in attendance.

I do understand the concern with faculty control, but ultimately course heads did have control over the data on students in their classes. Yet this has certainly raised questions about studies involving students that might not be set up to avoid identifying students. For that reason the Committee on the Use of Human Subjects in Research will automatically contact the Assistant Dean for Undergraduate Education in regard to studies that involve undergraduate students.