Gaming the Quals

Dave Tutelman  --  January 7, 2019

I took the qualifying exam for a PhD in Computer Science at University of Pennsylvania in 1967. The exam was the hardest I had ever taken, and studying for it was the most intense learning experience I ever had. My study buddy Allen Rehert and I did intense and focused study for four months leading up to the exam. Our routine was:
  • We picked a topic each week. A typical topic had to be pretty broad; remember, we had only 17 weeks to cover everything in computer science. At the beginning of the study week (which happened to be a Tuesday for us), we would divide up a list a few books and classical papers on the topic. Each of us would take half the material to read and understand that week.
  • Every Tuesday morning, we would reserve a small conference room at the Holmdel Bell Labs building where we both worked. We got a day off per week to pursue our PhD studies (very generous), and Tuesday was our day. The agenda for the Tuesday morning meeting was for each of us to summarize and teach the other the materials we had divided up the previous week. We could draw on the blackboard, use View-Graph slides (this was before white boards and PowerPoint), or just write at the desk; hey, there were only two of us. It is well said, "The best way to learn something is to teach it to someone else." We wound up learning the material really well that way.
  • The day we chose was Tuesday because we were taking Tuesday classes at Penn in the afternoon and evening. We would drive down together and drive back after classes. (I usually got home a little before midnight.) Because the drive was two hours each way, we had lots of time to argue back and forth about the fine points of the morning's study material. Additional intense learning took place.
But we didn't spend all our driving time on the material. Some days we had our minds on other things. Often it was about Allen's penchant for gamesmanship. He loved playing games, absolutely loved it! Even political games at work. Especially political games at work! We were both first-level managers, and we both liked to win. I liked to win by getting my ducks in a row and being right. Allen liked to win as if he were playing a game, almost poker. Yes, he liked to be right, of course; but it was best for him if he could anticipate the objections, lead the objectors into a trap, and spring the trap on them. Not my style, but he lived for it.

Apparently some of his game-playing stories wore off on me, and I did a bit myself -- during the examinations! I'd have been appalled at that thought a few weeks before the exams, but come exam time I indulged in a little game-playing, some planned and some improvised. Here's the story, but first a few words about how the exams worked. Some of this you'll need to know to understand the games -- and some just to realize what an ordeal it was.

About the qualifying exam

The exam was given in two parts: first a written exam, then an oral exam. Each part had five co-equal sections. I don't remember the exact names nor subject matter of the sections, but here is the rough contents:
  1. Technology -- things like transistors, logic gates, operational amplifiers (there were analog computers in those days, though the microprocessor was about to render them obsolete), and memory devices (mostly magnetic memory, cores and tapes).
  2. Logic design and automata -- combinational and sequential logic, states, stacks, and sequential memory.
  3. Discrete math -- abstract algebra, things like numbers, sets, and groups.
  4. Computation -- things like algorithms, computer languages, compilers, operating systems, and numerical analysis.
  5. Formal logic -- I bet you thought we covered all of that in #2 and #3. But there's more. Similar constructs, but use them to determine truth, falsehood, paradox, decidability, things like that. For instance, here's where we discovered that Turing, Church, and Gödel were all saying the same thing about slightly different models of reality.
The written exam was given first, for eight hours on the Monday of the exam week. It was open book. Anything you could carry (not wheel) in one trip from the class door to your exam desk you could use. Throughout my graduate studies, I kept notes on 3x5 cards. I had a briefcase set up as a card file, organized by subject with tabs. That was most of my study material. I had another briefcase with a few textbooks, for the subjects where I felt the weakest. Turns out I had studied well, and never had to open a text; the only "open book" I needed was my card files.

After the Monday writtens, Tuesday through Friday, was oral exams. There were eight candidates taking the exam that semester, so there were just enough half-day slots to go around. An oral exam was two and a half hours long. It comprised a half-hour grilling on each of the subject areas, each by a designated (and known in advance) professor who specializes in that area. The student (me) and the examiner (the professor) would stand at the board. At least one seat in the "gallery" was occupied; my graduate advisor was required to be there, to "defend" me against any unfair questions or tactics that the examiner might use. But other members of the faculty would wander in and sit for a while.

Wait! Faculty coming in and sitting for a while? You have to understand, this was the academic equivalent of gladiatorial combat. The professors were the lions, and I was -- well, you know. You could also think of it as a cage match of five 30-minute rounds, with me facing a different opponent every round. For any faculty who weren't actually examining me at that moment but happened to be in the building, this was their semiannual entertainment. There were 5-10 faculty members in the gallery at any given moment when I took my exam. Perhaps that is why my advisor had to be there as referee. I don't remember his having to defend me, which was good; but it was also good that he was there to do so.

Two things become apparent when you think about this format, things that suggest strategies for the student to keep in mind:
  • You know months before the actual exam who your examiner will be for each of the five subjects. It would behoove you to get to know each examiner a little, to understand what his (there was no "her" on the faculty at that time) favorite topics and pet peeves were. You wanted to be as expert on those things as you could get.
  • Your examiner would be the grader for your written exam in that subject, and grading done Monday evening. Anything you were not dead certain of on the written, you should quickly become an expert in before the orals. You know he's going to quiz you first and hardest about anything you got wrong on the written.
OK, that's what I was facing. While not the consummate game player that Allen was, I did more than my share -- and way more than my usual inclination -- on exam week. Here's what happened in the oral exams.

Set the agenda

I already noted, in the orals you should expect to be questioned on any weakness shown in the written exam. Just think. You can use that to set the agenda for the half-hour the professor is allowed with you. Just show some weakness in the written exam, in an area where you are actually strong.

I knew that Prof Rubinoff, my examiner for Technology, had a different opinion than I did about the relationship between precision and accuracy. I had been designing electronic instruments and test equipment for IBM and Bell Labs, as well as for my own hobby instruments, for 8 years at that point. I felt confident I could hold my own on the subject of precision and accuracy. When I saw a question on the written exam about the topic, I made sure to stress my views rather than Rubinoff's.

Rubinoff led off the oral exam by challenging me on precision vs accuracy -- exactly as I had intended. We sparred on the subject for almost half his allotted time, which was fine with me. I was comfortable arguing the subject with him, and I knew he was open-minded and didn't hold a grudge. By the time he was willing to change the subject (I'd have been happy to spend the entire 30 minutes on it), neither had convinced the other. But it was a good, well-informed argument and I got full credit for knowing the subject. (BTW, I don't think he could have gotten away with failing me on that part, even if he were so inclined. I believe I had half the faculty audience on my side in the debate.)

I may not have won the argument, but I won the game I had deliberately played. I got him to spend half his time with me on a subject that was my strength.

Meet the professor

This is one where I failed to prepare for the game, and could easily have lost it. Overconfidence to the point of arrogance.

I felt I knew Logic Design and Automata cold! I had taken several courses in the area when I was at MIT getting my Masters degree, taught by David Huffman and Fred Hennie, two huge names in the field. In fact, Huffman was my thesis advisor. On top of that, the subject was the activity I did when I worked at Bell Labs, and IBM before that. And I had successfully taught the Bell Labs course in Logic Design, taught to very talented engineers with EE Masters degrees, who were further motivated because they did it for a living and needed to be good at it. So I felt ready for anything that could be asked.

Dr Yamada was my examiner for Logic Design and Automata. I had never met him. Certainly hadn't taken his course; I had more courses than required in this area, from my MIT studies. I hadn't sought him out either, to learn his predilections. I felt I didn't have to. No, I had never even met him up to the moment he stepped up to start quizzing me. Big mistake on my part!

The exam began with Dr Yamada asking me, "You know about Kavanaugh Mat, right?" With his accent it sounded a little funny, but I wouldn't know Kavanaugh Mat if he stepped on me. I asked him to repeat the question, and he did. It didn't help. I've had oriental teachers before, and figured out their accent. But they were Chinese; this was a Japanese accent, and it was different. So I asked for yet another repeat -- looking more and more silly -- and this time I tried phonetically warping what he said in my mind. If it's an accent problem... Hey! Karnaugh map!

"Oh, you mean a Karnaugh map. Yes, I know about them." Darn right I know about them. They were presented the second hour of Logic Design 101 in every such course I had taken or taught. Very basic! Dr Yamada asked the question, I answered it, and things went smoothly after that very inauspicious start. In ten minutes, we had covered everything he had planned, from beginner stuff to the most advanced work being done. Once I understood his speech, I was on top of it. The rest of the half hour was spent on non-combative topics like, "What was the biggest system you designed using formal methods?" and "What were Huffman and Hennie really like?"

I'm just glad I was able to decipher his accent; things had been going downhill rapidly before I did. Definitely a case of overconfidence leading to not doing my homework.

The red herring

My oral exams were on an afternoon late in the week, Thursday if I remember right. I came to school in the morning rather than waiting until the last minute. So I had a chance to get the scuttlebutt that others were taking out of their own test sessions. One thing seemed to be a common thread of worry to everybody. At some point relatively early in the testing, a professor (not the same one for everybody) asked, "Tell me about McCulloch's Theorem."

Everybody was having a lot of trouble with that one. I had never heard of it either. So I rushed to the library (1967; no Google) and went from book to book trying to find it. No luck! I went to my afternoon exam knowing I'd miss at least one question.

I think it came up in my second half-hour. "Tell me about McCulloch's Theorem." I answered truthfully, at least as of when I awoke this morning, "I've never heard of it." Long silence -- probably 15 seconds. With 20:20 hindsight, the professor was probably waiting for me to say something else. I never did, so he continued with a completely different line of questioning.

I found out later (I think from Dr Carr, my graduate advisor) that they were playing games with me -- and the other candidates -- and I was the only one who refused to play. I couldn't find McCulloch's Theorem because there was no such thing! The point of the question was to see how willing the student was to BS in the absence of actual knowledge -- and how good at it they were. Take a hot guess? Say something so unspecific it couldn't be wrong? Try to draw enough out of the professor to glean what the Theorem actually was? I was the only one who wouldn't play.

Dueling banjos

My next-to-last exam was discrete math, and my examiner was Prof Harrison. I had taken his course in Abstract Algebra, and struggled a bit with it. (Got an 'A', but through sheer diligence, not insight.) And I hadn't taken it that semester, so the information was not only vague, it was stale: not a promising combination. I had studied, yes, but I was decidedly nervous about this part of the exam.

Just before the questioning started, Dr Corcoran came in to watch. He was going to be my last examiner, in the subject of formal logic. I had just finished his course in Mathematical Logic; in fact, I had just taken the final the previous week. Hadn't seen that grade yet, but I felt very good about it, and hoped I wouldn't embarrass myself in front of him during Prof Harrison's questioning. Anyway, since the questioning hadn't started yet, Dr Corcoran stood up and made a presentation to me. It was a book, a "Complete Works of Lewis Carroll", who was a mathematician and logician as well as writer of children's stories. The inscription (image at right) and his words of presentation were that I had the highest grades in the class in the course that semester. I guess that set some expectations going into the next session; I don't know whether it helped or hurt.

Anyway, Prof Harrison had at me with discrete math questions. I was handling them reasonably well, and feeling maybe I'd get through this OK. Then he asked a question about a topic that I felt very unsure about. While I didn't feel a sure grasp of it from Prof Harrison's class, I remembered his saying that algebraists and logicians looked at that topic differently, and he had gotten incensed about the difference. I did understand the logicians' view, if not the algebraists'. I decided to take a chance and answer the question as a logician might. Yes, I was picking a fight with Prof Harrison, but I knew Dr Corcoran, a logician, was in the audience and that might help limit Harrison's reaction.

It worked out way better than I had hoped. When Prof Harrison jumped on my answer, Dr Corcoran intervened to defend my position. For the next ten minutes I tried to look invisible in the corner, while Harrison and Corcoran dueled, algebraist vs logician. It was fun to watch, and I think I finally understood both sides of the topic. And yes, it might have taken as much as ten minutes before they remembered that I was supposed to be involved, not them going mano a mano. Prof Harrison turned to me again and asked a different question, mercifully one that I could handle. And the rest of that half hour went well enough for a pass -- minus the ten minutes "on the clock" while the big guns were dueling.

Conclusion

Of the eight candidates taking the PhD qualifying exams that semester:
  • Three got a "conditional pass". That means they passed at least three of the five parts. They would have to take courses in the deficient subjects, get a high enough grade, and probably have to re-take that part of the exam.
  • Three failed at least three parts, and therefore failed the test. They would have to wait a year before taking it again.
  • Allen and I passed unconditionally, and were officially through the classroom part of our PhD work. I guess we studied for it properly. And -- oh yes -- we gamed it well, too.


Last changed - 1/8/19