December 2019  
  
The most damaging thing you learned in school wasn't something you learned in
any specific class. It was learning to get good grades.  
  
When I was in college, a particularly earnest philosophy grad student once
told me that he never cared what grade he got in a class, only what he learned
in it. This stuck in my mind because it was the only time I ever heard anyone
say such a thing.  
  
For me, as for most students, the measurement of what I was learning
completely dominated actual learning in college. I was fairly earnest; I was
genuinely interested in most of the classes I took, and I worked hard. And yet
I worked by far the hardest when I was studying for a test.  
  
In theory, tests are merely what their name implies: tests of what you've
learned in the class. In theory you shouldn't have to prepare for a test in a
class any more than you have to prepare for a blood test. In theory you learn
from taking the class, from going to the lectures and doing the reading and/or
assignments, and the test that comes afterward merely measures how well you
learned.  
  
In practice, as almost everyone reading this will know, things are so
different that hearing this explanation of how classes and tests are meant to
work is like hearing the etymology of a word whose meaning has changed
completely. In practice, the phrase "studying for a test" was almost
redundant, because that was when one really studied. The difference between
diligent and slack students was that the former studied hard for tests and the
latter didn't. No one was pulling all-nighters two weeks into the semester.  
  
Even though I was a diligent student, almost all the work I did in school was
aimed at getting a good grade on something.  
  
To many people, it would seem strange that the preceding sentence has a
"though" in it. Aren't I merely stating a tautology? Isn't that what a
diligent student is, a straight-A student? That's how deeply the conflation of
learning with grades has infused our culture.  
  
Is it so bad if learning is conflated with grades? Yes, it is bad. And it
wasn't till decades after college, when I was running Y Combinator, that I
realized how bad it is.  
  
I knew of course when I was a student that studying for a test is far from
identical with actual learning. At the very least, you don't retain knowledge
you cram into your head the night before an exam. But the problem is worse
than that. The real problem is that most tests don't come close to measuring
what they're supposed to.  
  
If tests truly were tests of learning, things wouldn't be so bad. Getting good
grades and learning would converge, just a little late. The problem is that
nearly all tests given to students are terribly hackable. Most people who've
gotten good grades know this, and know it so well they've ceased even to
question it. You'll see when you realize how naive it sounds to act otherwise.  
  
Suppose you're taking a class on medieval history and the final exam is coming
up. The final exam is supposed to be a test of your knowledge of medieval
history, right? So if you have a couple days between now and the exam, surely
the best way to spend the time, if you want to do well on the exam, is to read
the best books you can find about medieval history. Then you'll know a lot
about it, and do well on the exam.  
  
No, no, no, experienced students are saying to themselves. If you merely read
good books on medieval history, most of the stuff you learned wouldn't be on
the test. It's not good books you want to read, but the lecture notes and
assigned reading in this class. And even most of that you can ignore, because
you only have to worry about the sort of thing that could turn up as a test
question. You're looking for sharply-defined chunks of information. If one of
the assigned readings has an interesting digression on some subtle point, you
can safely ignore that, because it's not the sort of thing that could be
turned into a test question. But if the professor tells you that there were
three underlying causes of the Schism of 1378, or three main consequences of
the Black Death, you'd better know them. And whether they were in fact the
causes or consequences is beside the point. For the purposes of this class
they are.  
  
At a university there are often copies of old exams floating around, and these
narrow still further what you have to learn. As well as learning what kind of
questions this professor asks, you'll often get actual exam questions. Many
professors re-use them. After teaching a class for 10 years, it would be hard
not to, at least inadvertently.  
  
In some classes, your professor will have had some sort of political axe to
grind, and if so you'll have to grind it too. The need for this varies. In
classes in math or the hard sciences or engineering it's rarely necessary, but
at the other end of the spectrum there are classes where you couldn't get a
good grade without it.  
  
Getting a good grade in a class on x is so different from learning a lot about
x that you have to choose one or the other, and you can't blame students if
they choose grades. Everyone judges them by their grades � graduate programs,
employers, scholarships, even their own parents.  
  
I liked learning, and I really enjoyed some of the papers and programs I wrote
in college. But did I ever, after turning in a paper in some class, sit down
and write another just for fun? Of course not. I had things due in other
classes. If it ever came to a choice of learning or grades, I chose grades. I
hadn't come to college to do badly.  
  
Anyone who cares about getting good grades has to play this game, or they'll
be surpassed by those who do. And at elite universities, that means nearly
everyone, since someone who didn't care about getting good grades probably
wouldn't be there in the first place. The result is that students compete to
maximize the difference between learning and getting good grades.  
  
Why are tests so bad? More precisely, why are they so hackable? Any
experienced programmer could answer that. How hackable is software whose
author hasn't paid any attention to preventing it from being hacked? Usually
it's as porous as a colander.  
  
Hackable is the default for any test imposed by an authority. The reason the
tests you're given are so consistently bad � so consistently far from
measuring what they're supposed to measure � is simply that the people
creating them haven't made much effort to prevent them from being hacked.  
  
But you can't blame teachers if their tests are hackable. Their job is to
teach, not to create unhackable tests. The real problem is grades, or more
precisely, that grades have been overloaded. If grades were merely a way for
teachers to tell students what they were doing right and wrong, like a coach
giving advice to an athlete, students wouldn't be tempted to hack tests. But
unfortunately after a certain age grades become more than advice. After a
certain age, whenever you're being taught, you're usually also being judged.  
  
I've used college tests as an example, but those are actually the least
hackable. All the tests most students take their whole lives are at least as
bad, including, most spectacularly of all, the test that gets them into
college. If getting into college were merely a matter of having the quality of
one's mind measured by admissions officers the way scientists measure the mass
of an object, we could tell teenage kids "learn a lot" and leave it at that.
You can tell how bad college admissions are, as a test, from how unlike high
school that sounds. In practice, the freakishly specific nature of the stuff
ambitious kids have to do in high school is directly proportionate to the
hackability of college admissions. The classes you don't care about that are
mostly memorization, the random "extracurricular activities" you have to
participate in to show you're "well-rounded," the standardized tests as
artificial as chess, the "essay" you have to write that's presumably meant to
hit some very specific target, but you're not told what.  
  
As well as being bad in what it does to kids, this test is also bad in the
sense of being very hackable. So hackable that whole industries have grown up
to hack it. This is the explicit purpose of test-prep companies and admissions
counsellors, but it's also a significant part of the function of private
schools.  
  
Why is this particular test so hackable? I think because of what it's
measuring. Although the popular story is that the way to get into a good
college is to be really smart, admissions officers at elite colleges neither
are, nor claim to be, looking only for that. What are they looking for?
They're looking for people who are not simply smart, but admirable in some
more general sense. And how is this more general admirableness measured? The
admissions officers feel it. In other words, they accept who they like.  
  
So what college admissions is a test of is whether you suit the taste of some
group of people. Well, of course a test like that is going to be hackable. And
because it's both very hackable and there's (thought to be) a lot at stake,
it's hacked like nothing else. That's why it distorts your life so much for so
long.  
  
It's no wonder high school students often feel alienated. The shape of their
lives is completely artificial.  
  
But wasting your time is not the worst thing the educational system does to
you. The worst thing it does is to train you that the way to win is by hacking
bad tests. This is a much subtler problem that I didn't recognize until I saw
it happening to other people.  
  
When I started advising startup founders at Y Combinator, especially young
ones, I was puzzled by the way they always seemed to make things
overcomplicated. How, they would ask, do you raise money? What's the trick for
making venture capitalists want to invest in you? The best way to make VCs
want to invest in you, I would explain, is to actually be a good investment.
Even if you could trick VCs into investing in a bad startup, you'd be tricking
yourselves too. You're investing time in the same company you're asking them
to invest money in. If it's not a good investment, why are you even doing it?  
  
Oh, they'd say, and then after a pause to digest this revelation, they'd ask:
What makes a startup a good investment?  
  
So I would explain that what makes a startup promising, not just in the eyes
of investors but in fact, is [_growth_](growth.html). Ideally in revenue, but
failing that in usage. What they needed to do was get lots of users.  
  
How does one get lots of users? They had all kinds of ideas about that. They
needed to do a big launch that would get them "exposure." They needed
influential people to talk about them. They even knew they needed to launch on
a tuesday, because that's when one gets the most attention.  
  
No, I would explain, that is not how to get lots of users. The way you get
lots of users is to make the product really great. Then people will not only
use it but recommend it to their friends, so your growth will be exponential
once you [_get it started_](ds.html).  
  
At this point I've told the founders something you'd think would be completely
obvious: that they should make a good company by making a good product. And
yet their reaction would be something like the reaction many physicists must
have had when they first heard about the theory of relativity: a mixture of
astonishment at its apparent genius, combined with a suspicion that anything
so weird couldn't possibly be right. Ok, they would say, dutifully. And could
you introduce us to such-and-such influential person? And remember, we want to
launch on Tuesday.  
  
It would sometimes take founders years to grasp these simple lessons. And not
because they were lazy or stupid. They just seemed blind to what was right in
front of them.  
  
Why, I would ask myself, do they always make things so complicated? And then
one day I realized this was not a rhetorical question.  
  
Why did founders tie themselves in knots doing the wrong things when the
answer was right in front of them? Because that was what they'd been trained
to do. Their education had taught them that the way to win was to hack the
test. And without even telling them they were being trained to do this. The
younger ones, the recent graduates, had never faced a non-artificial test.
They thought this was just how the world worked: that the first thing you did,
when facing any kind of challenge, was to figure out what the trick was for
hacking the test. That's why the conversation would always start with how to
raise money, because that read as the test. It came at the end of YC. It had
numbers attached to it, and higher numbers seemed to be better. It must be the
test.  
  
There are certainly big chunks of the world where the way to win is to hack
the test. This phenomenon isn't limited to schools. And some people, either
due to ideology or ignorance, claim that this is true of startups too. But it
isn't. In fact, one of the most striking things about startups is the degree
to which you win by simply doing good work. There are edge cases, as there are
in anything, but in general you win by getting users, and what users care
about is whether the product does what they want.  
  
Why did it take me so long to understand why founders made startups
overcomplicated? Because I hadn't realized explicitly that schools train us to
win by hacking bad tests. And not just them, but me! I'd been trained to hack
bad tests too, and hadn't realized it till decades later.  
  
I had lived as if I realized it, but without knowing why. For example, I had
avoided working for big companies. But if you'd asked why, I'd have said it
was because they were bogus, or bureaucratic. Or just yuck. I never understood
how much of my dislike of big companies was due to the fact that you win by
hacking bad tests.  
  
Similarly, the fact that the tests were unhackable was a lot of what attracted
me to startups. But again, I hadn't realized that explicitly.  
  
I had in effect achieved by successive approximations something that may have
a closed-form solution. I had gradually undone my training in hacking bad
tests without knowing I was doing it. Could someone coming out of school
banish this demon just by knowing its name, and saying begone? It seems worth
trying.  
  
Merely talking explicitly about this phenomenon is likely to make things
better, because much of its power comes from the fact that we take it for
granted. After you've noticed it, it seems the elephant in the room, but it's
a pretty well camouflaged elephant. The phenomenon is so old, and so
pervasive. And it's simply the result of neglect. No one meant things to be
this way. This is just what happens when you combine learning with grades,
competition, and the naive assumption of unhackability.  
  
It was mind-blowing to realize that two of the things I'd puzzled about the
most � the bogusness of high school, and the difficulty of getting founders to
see the obvious � both had the same cause. It's rare for such a big block to
slide into place so late.  
  
Usually when that happens it has implications in a lot of different areas, and
this case seems no exception. For example, it suggests both that education
could be done better, and how you might fix it. But it also suggests a
potential answer to the question all big companies seem to have: how can we be
more like a startup? I'm not going to chase down all the implications now.
What I want to focus on here is what it means for individuals.  
  
To start with, it means that most ambitious kids graduating from college have
something they may want to unlearn. But it also changes how you look at the
world. Instead of looking at all the different kinds of work people do and
thinking of them vaguely as more or less appealing, you can now ask a very
specific question that will sort them in an interesting way: to what extent do
you win at this kind of work by hacking bad tests?  
  
It would help if there was a way to recognize bad tests quickly. Is there a
pattern here? It turns out there is.  
  
Tests can be divided into two kinds: those that are imposed by authorities,
and those that aren't. Tests that aren't imposed by authorities are inherently
unhackable, in the sense that no one is claiming they're tests of anything
more than they actually test. A football match, for example, is simply a test
of who wins, not which team is better. You can tell that from the fact that
commentators sometimes say afterward that the better team won. Whereas tests
imposed by authorities are usually proxies for something else. A test in a
class is supposed to measure not just how well you did on that particular
test, but how much you learned in the class. While tests that aren't imposed
by authorities are inherently unhackable, those imposed by authorities have to
be made unhackable. Usually they aren't. So as a first approximation, bad
tests are roughly equivalent to tests imposed by authorities.  
  
You might actually like to win by hacking bad tests. Presumably some people
do. But I bet most people who find themselves doing this kind of work don't
like it. They just take it for granted that this is how the world works,
unless you want to drop out and be some kind of hippie artisan.  
  
I suspect many people implicitly assume that working in a field with bad tests
is the price of making lots of money. But that, I can tell you, is false. It
used to be true. In the mid-twentieth century, when the economy was [_composed
of oligopolies_](re.html), the only way to the top was by playing their game.
But it's not true now. There are now ways to get rich by doing good work, and
that's part of the reason people are so much more excited about getting rich
than they used to be. When I was a kid, you could either become an engineer
and make cool things, or make lots of money by becoming an "executive." Now
you can make lots of money by making cool things.  
  
Hacking bad tests is becoming less important as the link between work and
authority erodes. The erosion of that link is one of the most important trends
happening now, and we see its effects in almost every kind of work people do.
Startups are one of the most visible examples, but we see much the same thing
in writing. Writers no longer have to submit to publishers and editors to
reach readers; now they can go direct.  
  
The more I think about this question, the more optimistic I get. This seems
one of those situations where we don't realize how much something was holding
us back until it's eliminated. And I can foresee the whole bogus edifice
crumbling. Imagine what happens as more and more people start to ask
themselves if they want to win by hacking bad tests, and decide that they
don't. The kinds of work where you win by hacking bad tests will be starved of
talent, and the kinds where you win by doing good work will see an influx of
the most ambitious people. And as hacking bad tests shrinks in importance,
education will evolve to stop training us to do it. Imagine what the world
could look like if that happened.  
  
This is not just a lesson for individuals to unlearn, but one for society to
unlearn, and we'll be amazed at the energy that's liberated when we do.  
  
  
  
  
  
  
  
  
  
  
  
**Notes**  
  
[1] If using tests only to measure learning sounds impossibly utopian, that is
already the way things work at Lambda School. Lambda School doesn't have
grades. You either graduate or you don't. The only purpose of tests is to
decide at each stage of the curriculum whether you can continue to the next.
So in effect the whole school is pass/fail.  
  
[2] If the final exam consisted of a long conversation with the professor, you
could prepare for it by reading good books on medieval history. A lot of the
hackability of tests in schools is due to the fact that the same test has to
be given to large numbers of students.  
  
[3] Learning is the naive algorithm for getting good grades.  
  
[4] [_Hacking_](gba.html) has multiple senses. There's a narrow sense in which
it means to compromise something. That's the sense in which one hacks a bad
test. But there's another, more general sense, meaning to find a surprising
solution to a problem, often by thinking differently about it. Hacking in this
sense is a wonderful thing. And indeed, some of the hacks people use on bad
tests are impressively ingenious; the problem is not so much the hacking as
that, because the tests are hackable, they don't test what they're meant to.  
  
[5] The people who pick startups at Y Combinator are similar to admissions
officers, except that instead of being arbitrary, their acceptance criteria
are trained by a very tight feedback loop. If you accept a bad startup or
reject a good one, you will usually know it within a year or two at the
latest, and often within a month.  
  
[6] I'm sure admissions officers are tired of reading applications from kids
who seem to have no personality beyond being willing to seem however they're
supposed to seem to get accepted. What they don't realize is that they are, in
a sense, looking in a mirror. The lack of authenticity in the applicants is a
reflection of the arbitrariness of the application process. A dictator might
just as well complain about the lack of authenticity in the people around him.  
  
[7] By good work, I don't mean morally good, but good in the sense in which a
good craftsman does good work.  
  
[8] There are borderline cases where it's hard to say which category a test
falls in. For example, is raising venture capital like college admissions, or
is it like selling to a customer?  
  
[9] Note that a good test is merely one that's unhackable. Good here doesn't
mean morally good, but good in the sense of working well. The difference
between fields with bad tests and good ones is not that the former are bad and
the latter are good, but that the former are bogus and the latter aren't. But
those two measures are not unrelated. As Tara Ploughman said, the path from
good to evil goes through bogus.  
  
[10] People who think the recent increase in [_economic
inequality_](ineq.html) is due to changes in tax policy seem very naive to
anyone with experience in startups. Different people are getting rich now than
used to, and they're getting much richer than mere tax savings could make
them.  
  
[11] Note to tiger parents: you may think you're training your kids to win,
but if you're training them to win by hacking bad tests, you are, as parents
so often do, training them to fight the last war.  
  
  
  
**Thanks** to Austen Allred, Trevor Blackwell, Patrick Collison, Jessica
Livingston, Robert Morris, and Harj Taggar for reading drafts of this.  
  

