Gaming the Quals
Dave Tutelman --
January 7, 2019
I took the qualifying exam for a PhD in Computer Science at
University of Pennsylvania in 1967. The exam was the hardest I had ever
taken, and studying for it was the most intense learning experience I
ever had. My study buddy Allen Rehert and I did intense and focused
study for four months leading up to the exam. Our routine was:
But we didn't spend all
our driving time on the material. Some days we had our minds on other
things. Often it was about Allen's penchant for gamesmanship. He loved
playing games, absolutely loved it! Even political games at work. Especially
political games at work! We were both first-level managers, and we both
liked to win. I liked to win by getting my ducks in a row and being
right. Allen liked to win as if he were playing a game, almost poker.
Yes, he liked to be right, of course; but it was best for him if he
could anticipate the objections, lead the objectors into a trap, and
spring the trap on them. Not my style, but he lived for it.
picked a topic each week. A typical topic had to be pretty
remember, we had only 17 weeks to cover everything in computer
science. At the beginning of the study week (which happened to be a
Tuesday for us), we would divide up a list a few books and classical
papers on the topic. Each of us would take half the material to read
and understand that week.
- Every Tuesday morning, we would reserve a small conference
room at the Holmdel Bell Labs building where we both worked. We got a
day off per week to pursue our PhD studies (very generous), and Tuesday
was our day. The agenda for the Tuesday morning meeting was for each of
us to summarize and teach the other the materials we had divided up the
previous week. We could draw on the blackboard, use View-Graph slides
(this was before white boards and PowerPoint), or just write at the
desk; hey, there were only two of us. It is well said, "The best way to
learn something is to teach it to someone else." We wound up learning
the material really well that way.
- The day we chose was Tuesday
because we were taking Tuesday classes at Penn in the afternoon and
evening. We would drive down together and drive back after classes. (I
usually got home a little before midnight.) Because the drive was two
hours each way, we had lots of time to argue back and forth about the
fine points of the morning's study material. Additional intense
Apparently some of his game-playing stories wore off on me, and I did a
bit myself -- during
I'd have been appalled at that thought a few weeks before the exams,
but come exam time I indulged in a little game-playing, some planned
and some improvised. Here's the story, but first a few words about how
the exams worked. Some of this you'll need to know to understand the
games -- and some just to realize what an ordeal it was.
About the qualifying exam
exam was given in two parts: first a written exam, then an oral exam.
Each part had five co-equal sections. I don't remember the exact names
nor subject matter of the sections, but here is the rough contents:
exam was given first, for eight hours on the Monday of the exam week.
It was open book. Anything you could carry (not wheel) in one trip from
the class door to your exam desk you could use. Throughout my graduate
studies, I kept notes on 3x5 cards. I had a briefcase set up
as a card file, organized by subject with tabs. That was most of my
study material. I had another briefcase with a few textbooks, for the
subjects where I felt the weakest. Turns out I had studied well, and
never had to open a text; the only "open book" I needed was my card
-- things like transistors, logic gates, operational amplifiers (there
were analog computers in those days, though the microprocessor was
about to render them obsolete), and memory devices (mostly magnetic
memory, cores and tapes).
design and automata -- combinational and sequential logic,
states, stacks, and sequential memory.
math -- abstract algebra, things like numbers, sets, and
-- things like algorithms, computer languages, compilers, operating
systems, and numerical analysis.
logic -- I bet you thought we covered all of that in #2
and #3. But there's more.
Similar constructs, but use them to determine truth, falsehood,
paradox, decidability, things like that. For instance, here's where we
discovered that Turing, Church, and Gödel were all saying the same
thing about slightly different models of reality.
After the Monday writtens, Tuesday through Friday, was oral exams.
were eight candidates taking the exam that semester, so there were just
slots to go around. An oral exam was two and a half hours long. It
comprised a half-hour grilling on each of
the subject areas, each by a designated (and known in advance)
professor who specializes in that area. The student (me) and the
examiner (the professor) would stand at the board. At least one seat in
the "gallery" was occupied; my graduate advisor was required to be
there, to "defend" me against any unfair questions or tactics that the
examiner might use. But other members of the faculty would wander in
and sit for a while.
Faculty coming in and sitting for a
while? You have to understand, this was the academic equivalent of
gladiatorial combat. The professors were the lions, and I was -- well,
you know. You could also think of it as a cage match of five 30-minute
rounds, with me facing a different opponent every round. For any
faculty who weren't actually examining me at that
moment but happened to be in the building, this was their semiannual
entertainment. There were 5-10 faculty members in the gallery at any
moment when I took my exam. Perhaps that is why my advisor had to be
referee. I don't remember his having to defend me, which was good; but
it was also good that he was there to do so.
Two things become
apparent when you think about this format, things that suggest
strategies for the student to keep in mind:
OK, that's what I was facing. While not the consummate game player that
Allen was, I did more than my share -- and way more than my usual
inclination -- on exam week. Here's what happened in the oral exams.
- You know months before the actual exam who your examiner
will be for each of the five subjects.
It would behoove you to get to know each examiner a little, to
understand what his (there was no "her" on the faculty at that time)
topics and pet peeves were. You wanted to be as expert on those things
as you could get.
- Your examiner would be the grader for your
written exam in that subject, and grading done Monday evening. Anything
you were not dead certain of on
the written, you should quickly become an expert in before the orals.
he's going to quiz you first and hardest about anything you got wrong
on the written.
Set the agenda
already noted, in the orals you should expect to be questioned on any
weakness shown in the written exam. Just think. You can use that to set the agenda
for the half-hour the professor is allowed with you. Just show some
weakness in the written exam, in an area where you are actually strong.
knew that Prof Rubinoff, my examiner for Technology, had a different
opinion than I did about the relationship between precision and
accuracy. I had been designing electronic instruments and test
equipment for IBM and Bell Labs, as well as for my own hobby
instruments, for 8 years at that point. I felt confident I could hold
my own on the subject of precision and accuracy. When I saw a question
on the written exam about the topic, I made sure to stress my views
rather than Rubinoff's.
Rubinoff led off the oral exam by
challenging me on precision vs accuracy -- exactly as I had intended.
We sparred on the subject for almost half his allotted time, which was
fine with me. I was comfortable arguing the subject with him, and I
knew he was open-minded and didn't hold a grudge. By the time he was
willing to change the subject (I'd have been happy to spend the entire
30 minutes on it), neither had convinced the other. But it was a good,
well-informed argument and I got full credit for knowing the subject.
(BTW, I don't think he could have gotten away with failing me on that
part, even if he were so inclined. I believe I had half the faculty
audience on my side in the debate.)
I may not have won the
argument, but I won the game I had deliberately played. I got him to
spend half his time with me on a subject that was my strength.
Meet the professor
This is one where I failed to prepare for the game, and could easily
have lost it. Overconfidence to the point of arrogance.
felt I knew Logic Design and Automata cold! I had taken several courses
in the area when I was at MIT getting my Masters degree, taught by
David Huffman and Fred Hennie, two huge names in the field. In fact,
Huffman was my thesis advisor. On top of that, the subject was the
activity I did when I worked at Bell Labs, and IBM before that. And I
had successfully taught
Bell Labs course in Logic Design, taught to very talented engineers
with EE Masters degrees, who were further motivated because they did it
for a living and needed to be good at it. So I felt ready for anything
that could be asked.
Dr Yamada was my examiner for Logic Design
and Automata. I had never met him. Certainly hadn't taken his course; I
had more courses than required in this area, from my MIT studies. I
hadn't sought him out either, to learn his predilections. I felt I
didn't have to. No, I had never even met him up to the moment he
stepped up to start quizzing me. Big mistake on my part!
exam began with Dr Yamada asking me, "You know about Kavanaugh Mat,
right?" With his accent it sounded a little funny, but I wouldn't know
Kavanaugh Mat if he stepped on me. I asked him to repeat the question,
and he did. It didn't help. I've had oriental teachers before, and
figured out their accent. But they were Chinese; this was a Japanese
accent, and it was different. So I asked for yet another repeat --
looking more and more silly -- and this time I tried phonetically
warping what he said in my mind. If it's an accent problem... Hey!
"Oh, you mean a Karnaugh map. Yes, I know about
them." Darn right I know about them. They were presented the second
hour of Logic Design 101 in every such course I had taken or taught.
Very basic! Dr Yamada asked the question, I answered it, and things
went smoothly after that very inauspicious start. In ten minutes, we
had covered everything he had planned, from beginner stuff to the most
advanced work being done. Once I understood his speech, I was on top of
it. The rest of the half hour was spent on non-combative topics like,
"What was the biggest system you designed using formal methods?" and
"What were Huffman and Hennie really like?"
I'm just glad I was
able to decipher his accent; things had been going downhill rapidly
before I did. Definitely a case of overconfidence leading to not doing
The red herring
oral exams were on an afternoon late in the week, Thursday if I
remember right. I came to school in the morning rather than waiting
until the last minute. So I had a chance to get the scuttlebutt that
others were taking out of their own test sessions. One thing seemed to
be a common thread of worry to everybody. At some point relatively
early in the testing, a professor (not the same one for everybody)
asked, "Tell me about McCulloch's Theorem."
Everybody was having
a lot of trouble with that one. I had never heard of it either. So I
rushed to the library (1967; no Google) and went from book to book
trying to find it. No luck! I went to my afternoon exam knowing I'd
miss at least one question.
I think it came up in my second
half-hour. "Tell me about McCulloch's Theorem." I answered truthfully,
at least as of when I awoke this morning, "I've never heard of it."
Long silence -- probably 15 seconds. With 20:20 hindsight, the
professor was probably waiting for me to say something else. I never
did, so he continued with a completely different line of questioning.
I found out later (I think from Dr Carr, my graduate advisor) that they
were playing games with me
-- and the other candidates -- and I was the only one who refused to
play. I couldn't find McCulloch's Theorem because there was no such thing!
The point of the question was to see how willing the student was to BS
in the absence of actual knowledge -- and how good at it they were.
Take a hot guess? Say something so unspecific it couldn't be wrong? Try
to draw enough out of the professor to glean what the Theorem actually
was? I was the only one who wouldn't play.
next-to-last exam was discrete math, and my examiner was Prof Harrison.
I had taken his course in Abstract Algebra, and struggled a bit with
it. (Got an 'A', but through sheer diligence, not insight.) And I
hadn't taken it that semester, so the information was not only vague,
it was stale: not a promising combination. I had studied, yes, but I
was decidedly nervous about this part of the exam.
before the questioning started, Dr Corcoran came in to watch. He was
going to be my last examiner, in the subject of formal logic. I had
just finished his course in Mathematical Logic; in fact, I had just
taken the final the previous week. Hadn't seen that grade yet, but I
felt very good about it, and hoped I wouldn't embarrass myself in front
of him during Prof Harrison's questioning. Anyway, since the
questioning hadn't started yet, Dr Corcoran stood up and made a
presentation to me. It was a book, a "Complete Works of Lewis
who was a mathematician and logician as well as writer of children's
stories. The inscription (image at right) and his words of presentation
were that I had the highest grades in the class in the course that
semester. I guess that set some expectations going into the next
session; I don't know whether it helped or hurt.
Harrison had at me with discrete math questions. I was handling them
reasonably well, and feeling maybe I'd get through this OK. Then he
asked a question about a topic that I felt very unsure about. While I
didn't feel a sure grasp of it from Prof Harrison's class, I
remembered his saying that algebraists and logicians looked at that
topic differently, and he had gotten incensed about the difference. I
did understand the logicians' view, if not the algebraists'. I
decided to take a chance and answer the question as a logician might.
Yes, I was picking a fight with Prof Harrison, but I knew Dr Corcoran,
a logician, was in the audience and that might help limit Harrison's
It worked out way better than I had hoped. When Prof
Harrison jumped on my answer, Dr Corcoran intervened to defend my
position. For the next ten minutes I tried to look invisible in the
corner, while Harrison and Corcoran dueled, algebraist vs logician. It
was fun to watch, and I think I finally understood both sides of the
topic. And yes, it might have taken as much as ten minutes before they remembered that I was
supposed to be involved, not them going mano a mano. Prof Harrison
turned to me again and asked a different question, mercifully one that
I could handle. And the rest of that half hour went well enough for a
pass -- minus the ten minutes "on the clock" while the big
guns were dueling.
Of the eight candidates taking the PhD qualifying exams that semester:
got a "conditional pass". That means they passed at least three of the
five parts. They would have to take courses in the deficient subjects,
get a high enough grade, and probably have to re-take that part of the
- Three failed at least three parts, and therefore failed the
test. They would have to wait a year before taking it again.
- Allen and I passed unconditionally, and were officially through the
classroom part of our PhD work. I guess we studied for it properly.
And -- oh yes -- we gamed it well, too.
Last changed - 1/8/19