The Best Essay
paulgraham.com
paulgraham.com
And the real shame is that he gets so close: he talks about discovery, about the intersection of science and technology and the broader world, he identifies the importance of doing things to learn enough to write about them, and most importantly: having an outrageous question with a real and alarming insight lurking around the corner as an answer and thereby avoids vaporous rhapsodizing.
So I’ll ask an outrageous question about which pg has both deep insight and over a decade of lived experience to ground an answer on: is YC still a good thing? I’ll ask a few corollary questions to illustrate.
He talks about the kinds of tests you hack to get ahead in some vague elsewhere: is getting ahead in YC, or the Valley, or the technology business still about merit and capability and being relentlessly resourceful rather than optics, connections, and flexibility? Is it about garages with Ethernet cables snaking everywhere more or less than knowing one’s way around the Rosewood Sand Hill?
His first citation as a reviewer is Sam Altman: is @sama still his pick for most capable founder and best person to guide the sprawling empire that YC sits at the heart of?
I don’t find any of those things obvious, and I think a candid essay about a set of very timely questions would be the most important essay he’d ever written.
Of course one can only guess what the message for him might be.
YC's public content focuses on early-stage startups. Should it have more on late-stage startups? Maybe, but it doesn't seem as impactful. The reason to produce public content is that it can scale to reach more people than the partners can talk to one-on-one. There are fewer late-stage startups, so the scaling problem hasn't hit yet. YC follows its own advice of doing things in ways that don't scale until scaling becomes necessary.
I do not find it at all obvious that either a trillion dollars in private investment or aggressive lobbying around regulation favorable to a particular for-profit entity are necessary or even possible paths to outcomes that are globally good for humanity or locally good for the community of people who spend time in threads like these.
I don't think that I'm part of a small minority in finding those means very dubious ways to accomplish things, the ends to which those means are being put of questionable public-mindedness, or the secrecy and apparent subterfuge with which this to all external appearances is being done acceptable ways to wield vast power.
I don't know if you'd make this argument, but others have when arguing for the same outcomes, so if I'm addressing them and not you please excuse me: we have institutions with a mandate from the public to deploy wartime budgets on Manhattan Project-scale undertakings attended by extreme risk. This cannot be ethically, morally, or legally delegated to unelected and unappointed individuals or private for-profit institutions via what was once called corruption.
Clearly I speak only for myself, but many others with far better credentials are saying similar things, in many cases much more stridently.
" I've been probingly questioning of pg's essay on this very thread, and as he requested I asked a few questions that can be fairly described as "outrageous". I think they're in the spirit of his call for outrageous questions, but they aren't "softballs".
Trevor challenged me on this, and I respectfully disagreed with his assertions in the same breath as acknowledging the almost limitless power of the YC network in my profession.
I did this not because I have any desire to be ground under the boot of the YC network, but because I trust @pg and Jessica and @tlb and (especially these days) @dang. My questions are about whether or not their good intentions might have gone awry, why they might or might not be worried about that: not whether or not they have always had and still have good intentions.
I've been hanging around here for like 16 years, and I don't fear retribution for respectful dissent at all. This is in sharp contrast to some other parts of the industry that I've wandered through. "
I remember running across a tweet that said, in essence, you always have access to your own optimal policy if you simply ask. A similar technique might be the key to wresting good ideas from an ambiguous starting point: "What would be the most consistent answer to the original question, given what I've learned and written so far?"
Today within LLMs, there typicality sampling, contrastive search, top-p/top-k (nucleus sampling), and a whole massive amount of other more obscure techniques which didn't make it into huggingfaces model.generate() function.
I believe that the ontological configuration of humans has wide variety. As such, I believe that humans go through a diverse and vibrant amount of different writing processes - and often radically different processes can still result in excellent works!
Specifically, beam search is really only good at dealing with sequence-sequence tasks, i.e. summarization or translation. It's also horribly, horribly inefficient in the worst case scenarios. That said, it's also the only good way we know of for enforcing sequence level constraints (i.e. like this https://huggingface.co/blog/constrained-beam-search)
Why not? "Death of a Pig" didn't convey any new scientific ideas, and might not even have been surprising in any kind of intellectual way.
You could title this piece "Great Essays" and it would be entirely defensible. But Graham gave himself a higher goal here, and I don't think he's really presented a recipe for writing the Best essay. Look what he's up against: Baldwin, Didion, Oliver Sacks; it's easier to come up with examples of great essays that don't set out to develop surprising new ideas, and that probably didn't start out with a mischievous look in the author's eyes.
I'm not saying this isn't good advice for developing great essays, just that it's advice that narrows the solution space a bit much.
For others unaware of it, that essay was written in 1948[1], go read it in full. It starts like this:
"I spent several days and nights in mid-September with an ailing pig and I feel driven to account for this stretch of time, more particularly since the pig died at last, and I lived, and things might easily have gone the other way round and none left to do the accounting."
[1] https://web.archive.org/web/20240227003736/https://www.theat...
I'm mostly motivated to comment by that phenomenon, not by the plain text of Graham's essay.
There's another fantastic book—On Writing Well—that is a recurring read for me. An aside—I was in one of the early programing bootcamps, The Starter League. My class was held in 37signals office. Jason Fried and some of the other 37signals designers were in this class with me. One day Jason stood up and answered a few questions. One of the other students asked him about books. He walked over to a closet and opened the door. There were hundreds of copies of On Writing Well. That's the book he'd gift to people.
That always struck me as interesting. Especially since he'd written (or co-authored) several books at that point.
https://www.chronicle.com/article/50-years-of-stupid-grammar...
[EDITED to add:] I don't know how close to identical this is to the Chronicle article; at the end it says "Adapted from an article written for The Chronicle of Higher Education"; my guess is that essentially nothing was changed, but I haven't tried to check.
> “Write with nouns and verbs, not with adjectives and adverbs,” they insist. (The motivation of this mysterious decree remains unclear to me.)
The motivation is so that ponderous language like
> Following the platitudinous style recommendations of Elements would make your writing better if you knew how to follow them
Might be edited to something like "Though many are mere platitudes, following the style recommendations of Elements would make your writing better if you knew how to follow them".
The sentence, as written, suggests that the platitudes, specifically, as opposed to any which might be less uhhhh, platitudinous, would make your writing better. I don't believe this was the author's intent.
I also love “clear and simple as truth” by thomas and turner.
Another book I love is, "Towards Clarity and Grace" by Joseph Williams and Gregory Colomb (there are several expensive editions of it with modified names, but any older version would do). I elaborated more on it here[1] in the past.
I'll add some fan mail later if necessary (because I do get value from reading him), but for the moment, here's where I believe he went off course.
1. When he says the best essays are "ineffective," he's chosen the wrong word. They are "premature." They arrive before the world is fully ready to acknowledge their power. But they catch at least a modest following right away. And then their work grows and grows.
2. Essays about new technology can be quite powerful, and that's Graham's wheelhouse, so it's fine for him to talk up this cohort. But any serious survey of legendary essays needs to look wider. The most powerful essays redefine our social, moral, political and religious norms. Here are a few favorites that didn't just win their year; they stood out for centuries.
70 AD: The Gospel of Mark. Chronologically the first book of the New Testament, and look what that unleashed
1778: Common Sense by Thomas Paine. The boldest, fiercest justification for the American Revolution, and one that's still a touchstone today for anyone with a deep interest in the theory or practice of democracy.
1963: Letter From Birmingham Jail by Martin Luther King. Worth being on the list simply for its effect on the U.S. civil rights movement; even more significant as the unbreakable tuning fork for any civil or human rights movement anywhere.
We'll keep inventing new technologies, because that's what humans are good at, and I'm sure many strong essays will ensue. But it's the redefining of our social institutions that's likely to make the future so incredibly different from today. Anyone who can write a prescient reflection about society's new rules will get my vote for "Great Essays."
"Almost any question can get you a good essay. Indeed, it took some effort to think of a sufficiently unpromising topic in the third paragraph, because any essayist's first impulse on hearing that the best essay couldn't be about x would be to try to write it. But if most questions yield good essays, only some yield great ones."
Sounds like you have a challenge in front of you.
And that isn't even getting into things like "must tell you something you didn't know." By that, I can never find an essay that I've read before to be great? Or to learn from one I have already read before, but from a different perspective?
Will be fun to read more of what others hear in this essay. I suspect I should give more credence to the idea of the best essay for today. And each day can be another search for today's winning entry.
“While breadth comes from reading and talking and seeing, depth comes from doing. The way to really learn about some domain is to have to solve problems in it”
You can easily accumulate depth of knowledge from reading in areas that are well documented.
In fact that’s what I would consider the critical dividing line between a regular genius and a bonafide super-genius. Someone who almost supernaturally acquires expertise/intuition/depth/etc. with very little visible effort.
Do you have an example of bonafide super-genius?
He once mentored a guy who was so ridiculously good at writing programs, that he wrote out (once) a 5 foot stack of punch cards (back when that was how you would code), to create a program that was needed by the company for some purpose.
It worked, flawlessly, the FIRST time.
This was his MO.
Once, he wrote a program for an internal client, and it got shipped. They ran the program and ran into a problem. My friend told him to debug it for them… he replied “I don’t know how”. Up to that point in his life, He had never had to debug a single piece of software…
My friend helped him debug the program: Turns out, the problem was not an issue with his code, the customer had given him the wrong spec for a critical interface… and that was the only reason it had not worked the first time.
There are truly people that qualify. The only note I will make is that generally if you are a super genius in one dimension, you likely have something you are absolutely terrible at in another. Hopefully it is in a dimension that either doesn’t matter, or you have enough complementary people around you to mitigate it.
You're pulling our leg or in denial.
FWIW, apparently back before Intel released a particular 4 bit processor, this guy made an emulator and compiler for the chip so they could start writing code in anticipation of its release.
Once again, not direct experience… but trusted source.
Every CS department has these legendary tales that grow more hyperbolic over time (See "never" above.)
They are fun to tell!
Two alternate stories I didn’t tell:
First is about my two friends (both ridiculously smart) who in undergrad became the TAs for the Operating System class at Carnegie Mellon as undergrads (normally taught by grad students)… this is the hardest CS course taught there. As part of their summer prep, they wrote a new file system example… I believe based on a b-tree.
The next was my AI professor, Andrew Moore [0], he was legendary at the school… eventually becoming dean after doing a stint at Google (he has since stepped down). By far, he is the smartest person I have ever personally known. To give you context (and we did this regularly in his class), you could ask him a question on anything and he would pause, think about it, and come up with a well reasoned answer that would be both insightful and illuminating… from first principles. On any subject. You could not throw him (we tried). I am still in awe of him.
While I agree that legendary tales grow… they are almost always based on a kernel of truth. The reality is most people don’t often interact with folks at these levels. I was very lucky, and I have only interacted with a handful.
[0] https://www.forbes.com/sites/peterhigh/2017/10/30/carnegie-m...
There aren't any Tony Starks who become experts in thermonuclear astrophysics overnight.
Richard Feynman was as close to this trope as you can get in reality, but insisted that his reputation for being able to solve difficult problems was due to having a "different box of tools" than others. And he obtained that by studying rather obsessively, well beyond assigned texts.
I have been lucky enough to know a few people who also might qualify as geniuses able to produce miraculous results. One of them decorated his laptop with the logos of defunct computer companies of the 1950s-1980s. He drew a lot of inspiration from papers and books that few others have read in thirty years.
I'm an atheist but when someone like him tells me that god gave him all of his equations - I'm not going to respond with "well ackshully". I'm going to ask them how we can make it easier for them to listen to god.
Assuming you don’t have a direct line to divinity yourself, you would need to have that profound genius in the first place to evaluate…
The original claim was that a genius should be able to easily and quickly attain mastery of any topic, given a grounding in the basics. Like, you give Mr. Super Genius IQ a book on quantum chemistry and the next day they’re putting things together so cleverly they are on the precipice of new discoveries.
Feynman actually did this a couple of times! But I think that just shows the “unreasonable usefulness” of mathematics, as well as his own spirit of curiosity. (And at least one time that we know of, he faked it by reading a colleague’s research before he presented it. Not to take credit, but just as a prank, so he could interrupt and predict every conclusion before the presenter finished.)
True “depth” comes from the know-how acquired in digging through all the minutia nobody else though to document, on your way to producing a new creation nobody else thought to build.
The trope that software projects are poorly documented is so pervasive as to be saying nothing at this point. Despite extremely useful and celebrated documentation projects (MDN and Python off the top of my head) that dramatically accelerate other peoples' learning and productivity.
But when the topic is raised, the response is always "But we don't have time for documentation".
And when that time is given it becomes "It will be useless or out of date soon anyway, just go read the code" (As if there is nothing between non-practical high-level docs and per-line code comments.)
And when forced it becomes "Fine, but it won't be any good" or "Other people won't read/benefit from it anyway".
And when it's done, often as little effort as possible is put into it.
All of which smell to me like cover stories for: "I just don't want to do it."
Supremely narrow minded. Knowledge has many facets.
In other words: example?
Henry Heimlich used "the heimlich maneuver" for the first time when he was 96 (in 2016), 40 years after inventing it, in the "senior home" where he was staying. But according to you, having invented was nothing in the first 40 years, because he didn't actually use it, never mind that it was used by others and taught as a life saving technique.
You're just beathtakingly ignorant of how progress actually happens. It's like your understanding of the world comes exclusively from thinking about the world in abstract hypotheticals, rather than interacting with it, which is ironic coming from someone stating that only action matters.
If all he did was read books about things other people did, he certainly would not have invented that.
Your following attempt at armchair psychology is humorous, but not much beyond that I’m afraid. Don’t quit your dayjob.
And my specialty is spotting morons on social media.
Do you have a real counter example? Someone that through reading the material of others alone, with no practical/“hands on” experience, was able to develop what you consider “deep” knowledge?
Edit: A day ago you were arguing talent can’t come without training. https://news.ycombinator.com/item?id=39656444
In fact much of your comment history is dedicated to claiming knowledge requires product. How bizzare.
> In fact much of your comment history is dedicated to claiming knowledge requires product.
Your comment is a great example of someone putting their (mis-)interpretation into something.
No, I didn't say anything about products. I leave that for tinkerers and sales people.
> Do you have a real counter example?
Already gave you one, no idea why you're asking again.
^Or at least something with approximately that level of intellectual capacity/honesty.
Given that specific knowledge relevant to a field may be of a highly specific and hard to understand nature it increases the risk that any attempt to understand that knowledge by simply reading it will fail due to simple misunderstanding of what one reads.
In documenting things there are always points in which documenting minute details of a thing starts to detract from the purpose of documentation, that is to say the more in depth and detailed one documents the less readable the documentation becomes, therefore one leaves out things that should be easily understood by others when trying to use the documentation to actually work in the field or will quickly be imparted by other practitioners in the field if it is one with easy access to others.
Documentation by its nature is aimed at everyone, but there may be particular things that would be obvious to many people but not some specific person, and that specific person when reading the best documented guides to the area of knowledge will still not be as knowledgeable as they believe, because everybody is different.
Very many areas of knowledge have specific relation to things that people do with their bodies, martial arts, sex, cooking, etc. etc. In such cases there is of course muscle memory, thus no matter how precise and painstaking the documentation will be in these areas you will not be as knowledgeable as one that builds up muscle memory in the field by doing if you rely on only reading the documentation.
I could go on, but given my point about minute details it might be self-defeating.
The default action for an autonomous entity or system is to do nothing. It’s really odd to phrase doing nothing as a choice. Anything other than doing nothing is a choice. Especially when it comes to work, most humans just don’t want to bother spending time thinking about work outside of paid work time.
I'm so tired of (mostly boomers) talking about learning how to work on cars like it's easy. Sorry guys, but the companies who make those DIY repair manuals that you guys keep talking about using to learn wrenching don't seem to make stuff for any modern vehicles. Don't believe me? Go check out how pathetic their selection is right now: https://haynes.com/en-us/
It is nigh impossible for a non-car person to learn how to wrench without direct literal hand-holding from those who do.
It's also lead to the mechanic industry being FAMOUS for scamming grandmas, mothers, US soldiers, and other captive audiences. I think the only groups with a worse reputation are lawyers and car salespeople.
The boomer way wasn't that easy, but it actually finally IS easy to learn to work on cars.
There's also this, I can recommend it: https://www.howacarworks.com/
Watch even experienced mechanics works on new cars. They tend to bumble around for awhile.
After all, the term "essay" was invented/popularized by Montaigne in a book of his writing called "Essays" - and "essay" is French for "to try." So from that context, essays shouldn't be concerned with finding answers or be the "best" but to make discoveries in the process of trying.
Which looks to be where the English word assay comes from too: https://www.etymonline.com/word/assay
To "assay" is to engage in an "investigative (analytic) procedure" upon some thing [0].
To "essay" (je vais essayer) is an attempt to achieve a specific thing.
The former seems more like an exploration without initial expectations, like a fact finding, focus group, or initial prototype.
"Assay" came via Norman, "essay" came via French.
There are lots of other examples like this where one word came from Norman and the other from French, e.g. warden + guardian, warranty + guarantee
From that same Wiki definition I notice that "In English, essay first meant a trial or an attempt.
That implies an iteration on some seed, or maybe just the intent to grow and iterate on the initial essay.
Personally I find that true. Most of my "essays" I keep for myself, or sometimes share just with close friends who I trust to bounce ideas off as they mature.
Later they become collections of essays in a folder, and I might join the dots between them to perhaps see the potential for something of a book, or a new essay that synthesises them.
So I think calling something an "essay" grants the author one special privilege; the right to revise and "re-attempt" the same piece without moral judgement.
That's problematic in the Internet age, where the provenance of a document is judged by dates, by diffs to archived versions etc. It wouldn't seem right to actually change the contents at the same URL.
Nor would it seem right to insist on tracking edits, because a writer always has the right to forget and disown old and weak ideas they've moved beyond.
So I think essayists have our permission to keep approaching the same idea over and over. The way composers would sometimes write ten different versions of the same piece.
What makes a "best" essay then is its improvement with respect to the author's previous attempts.
All we can ask is, "Is this Mr Graham's best essay so far?"
pg does specifically address this objection btw:
> I love questions that seem naughty in some way — for example, by seeming counterintuitive or overambitious or heterodox. Ideally all three. This essay is an example. Writing about the best essay implies there is such a thing, which pseudo-intellectuals will dismiss as reductive, though it follows necessarily from the possibility of one essay being better than another.
Preemptive dismissal of any criticism of the idea of a best essay is stupid. In particular, even though I buy the "possibility of one essay being better than another", the criteria for ordering will be necessarily subjective.
As someone who has read many essays in the European tradition (Montaigne, Sartre, etc.), I cannot bring myself to see the essays by E. B. White, for example, to be superior in any sense. Yet, I know that White's essays are held in high-regard in this part of the world. This points to the fact that there is no universal ordering function for essays, which mostly invalidates Graham's argument above.
This is false unless you further assume that "better than" is a total order.
Also what you said, but they aren't mutually exclusive.
I think he's speaking broadly about what counts as an essay, and does have at least one example in there to illustrate his point. He's also an esteemed essay writer, and I think it's fair for him to talk about what he values in an essay without needing to justify any of it by saying "and also, this person that some people respect did it like that"
At the same time, to strengthen the practice, make a statement and then follow with a question. Make the impact, leave the reader with a question back to it. It's two sentences, paragraph break.
Those are the best essays.
He practiced this technique many times in this article. But is this the format of best essays [of the Internet]?
Women are so superficial (cringe)
This essay reads as if someone has too much time on their hands and is so self-absorbed that you get blog post titles like this and my god what on earth did I actually read?
I feel like even if you guess right, you can't help but do a lot of rewriting and exploration because otherwise you'll always feel that "something's missing", and that you're further from the truth than you could be, that there's a better explanation around the corner if you study this subtree a bit more.
In the process of an essay I'm currently writing, I've probably written a short book of meandering thoughts / notes. I can't imagine it any other way because doing so would be imagining me actively not exploring an interesting subtree / implication of something I wrote, that I feel could impact the crux of the essay.
And of course none of this is wasted effort, as PG talks also mentions. Identifying the next essays from those ideas I've left out of this one is simply an act of observation and organization based on the exploration I've already done.
And likely when I start the next essay, this process will repeat itself.
Not to be a pseudo-intellectual, but this confuses partial orderings and total orderings. It makes for a fun discussion, but I hope Paul would agree that there's obviously no "best essay", real or hypothetical.
His writing is consistently flat, uninspired, and egotistical. He's ok at writing clear prose and this plus his success in relation to YC has made him popular among technical people that also embrace capitalism. Anyone outside of this little cult realizes that pg says very little of value, particularly when he decides to comment on subjects outside of his capitalistic and technical wheelhouses. I have not read a single pg essay that touches on culture, ethics, or general epistemology that has given me anything other than a complete groan of a reaction and astonishment that this is the sort of thinking that passes the "high-quality" bar for the average technologist.
pg can try as hard as he can to write the "best essay". He will never convince this reader since he occupies a completely different world and basically isn't even capable of speaking to this audience in the first place.
...just to lengthen this comment more than is due, the quote is astonishing. I mean, how arrogant do you have to be to claim that the past hundreds of years of literary development, in which the notion of "best essay" is considered at best completely relative and at worst nonsense, must be wrong and that anyone who thinks otherwise must just be a "pseudo intellectual" (read: someone who disagrees with paul graham). He's really high on his own supply.
You always have to try or you wouldn't have a chance. But you can't just sit down and determinedly write your best material ever, with any kind of certainty, right when you decide to.
Good fortune is going to need to smile.
Suppose you were trying to answer the question: "what is the best way to make as much as money as possible in the next year?" You imagine optimizing yourself or your company or your algorithm or whatever to answer this question. Maybe it does a bunch of calculations and decides the answer is "day-trading commodities futures" or something. Or maybe a company does this and thinks the answer is "spin up a new product and sell it". (Or any situation where you're optimizing some other variable, like "altruism" or "security".)
Well, in almost every case, some of the best answers are "do something impossible". For instance, "write a story that is so good you get a 1 billion dollar book deal out of it". Or "write an essay that convinces everyone to give you all their money". Or "hack something that's considered unhackable and take all the money". What if the most altruistic thing you can do is... convince everyone to become pacifist? Or start a new religion? Etc.
Each of these has the property that rationality alone can't really model it. An algorithm can't analyze reality and say "write a story that is so good you get a 1 billion dollar book deal", or "write an essay that convinces everyone to become pacifist". (Maybe an actually-sentient AI can. But that's about it.) The only way you could come up with that as an actual strategy is to have an unrealistic belief that it is possible. Basically to have faith in yourself and your vision of the world despite the evidence for that strategy not being solid.
Yet the strategy can work. History shows that sometimes it does. And for a particular person at a particular time it might be the right answer. But it will never be the "right strategy" according to an outside computation.
I dunno. Always felt like that was interesting. Not sure if there's a word for it. If not I would call it the "miracle paradox": miracles occur, but you can't rationally justify them as a strategy, yet believing you can pull one off is necessary for them to happen, in which case they are the correct strategy.
very very roughly the ideas you need are:
1. survivorship bias
2. self-interested optimisation is not guaranteed to lead to a global maxima, e.g. self-interested agents may be systematically less risk-taking than would be societally optimal.
3. it's fine for rational agents to account for private information, e.g. "I did really well at Stanford probably I have a better shot at a company than a random person." is a fine thing to include in reasoning.
Yes, of course rational agents can account for private information. I am trying to describe a case where a strategy cannot be rationally justified, yet is a better strategy than others because of "having faith in yourself". Now you can always try to flip this around and say: "well maybe the internal calculation that they do involves having evidence that they ought to have faith in themselves, like past success, consistently good mental models, a model of how faith helps, etc". And I'm not talking about those: I'm talking about cases where there is not a good argument, even in an internal calculation using private information, why a person should have faith in themselves, yet doing so anyway helps.
Basically it seems like there are cases where believing that something has a more-than-rational chance of working makes it more likely to work because of the power that irrational faith in yourself provides. But suppose you tried to quantify it: okay, an irrational faith in yourself makes you, I don't know, 20% more like to succeed, so you can account for that. But now it's not an irrational faith, it's a rational one, so you can still have more irrational faith than that, and it can still have the same effect.
My pattern-matching brain says it has the form of a Halting Problem/Godel-incompleteness-type theorem for rational calculation. Roughly: "It is possible to construct scenarios in which non-rational strategies lead to improved success in ways that rational calculations cannot account for."
This would not apply in a toy problem where the range of strategies are completely mathematically known: if all outcomes can be understood probabilistically then it doesn't apply. It applies in cases that are more like: some outcomes are unknown and the probability distributions of them are shielded behind "unknowable information", like new ideas that you have no way of being sure of the existence of. In these cases an irrational belief that you can find a new idea can improve your EV even though there is mathematically no way to justify it, because you can't really model "the probability of finding a new idea" without actually knowing what the idea is, even hypothetically.
"Moonshot" type endeavours are not paradoxes. They do involve considerable uncertainty and usually some kind of large investment (time, money or something else). Most can't afford them and don't have any particular reason to believe they will be more successful than everyone else who has tried. Relative to most people's resources, time preference and risk tolerance they are bad bets.
If you are starving then your strategy for getting food should usually be the one with the highest overall chance of success, you care much less about factors like effort/reward ratio or getting the highest possible payoff.
The most common strategy for people who want a tiny chance at an enormous payoff is to buy a lottery ticket. Millions of people do that every week.
Such bets absolutely do feel paradoxical at times because multiplying a very small chance of success times a huge (usually easier to reason about) payoff seems to break people's brains a bit. This is often described as a "bug" in human cognition, though I have always felt it's an instinct for "exploration" in the sense of occasionally making small bets with huge potential payoff "just in case" your world model is wrong. Also that the way humans process probabilities and payoffs seems more logarithmic than linear (making small probabilities "feel" bigger than they really are).
Claude 3 Opus:
Here is a story that I hope captures your interest and imagination:
The Memory Thief
In a world where memories could be extracted, bought and sold, Lina was the most skilled Memory Thief in the business. The wealthy elite paid handsomely for the chance to acquire the experiences and skills of others. And Lina had no qualms about stealing memories from the unwitting. It was just business.
Her latest mark was Jonathan Thorn, a reclusive tech billionaire. The bounty on his memories was the biggest Lina had ever seen. Enough to set her up for life.
Infiltrating Thorn's mind was surprisingly easy. But as she slipped into his memories, Lina found herself drawn in by the raw beauty and staggering brilliance she discovered there. She witnessed Thorn's greatest triumphs and his deepest heartbreaks. And she began to question everything about her life as a thief.
Then, in the innermost recesses of Thorn's mind, Lina uncovered a memory unlike any she had ever encountered - a memory of the future. In it, she saw a world in flames, its population enslaved by a ruthless dictator. And she saw herself and Thorn, leading the last desperate resistance against the tyrant's armies.
Shaken to her core, Lina fled Thorn's mind without the bounty memories. But she took something else with her - a newfound resolve and a dangerous knowledge.
Now, Lina must track down Thorn in the real world and convince him that his prophetic memory is true. Together, they must figure out how to prevent the apocalyptic future only they know is coming.
Even if it means Lina must do the one thing a Memory Thief is never supposed to do - reveal who she really is.
What follows is a race against time and a battle against fate as an unlikely pair - the world's most notorious thief and the man whose mind she invaded - become the only hope for salvation. All the while, a forbidden love blossoms between them, forged by the intimate knowledge of each other's deepest secrets and the heavy burden they now share.
In the end, they will have to sacrifice everything to rewrite the future. But in the process, they might just find that the most valuable memories are the ones you make with someone else.
In the end, the true Billion Dollars was the friends we made along the way.
I never knew SOTA LLMs have gone this far. Wow!
From this it seems that the best strategy for doing impossible things is to assume that impossible things are actually possible. It's only by doing that that you'll find out.
It's a risky strategy though, since many things are actually impossible, and even for possible things, they may be difficult, or the easy way may be hard to discover.
Maybe there's some hueristic for sorting the actually possible, and from there, for sorting the reasonably doable.
But I'd be wary of hueristics because (barring unusual personal experience) they're likely to suffer from the same bias that led people to dismiss things as impossible in the first place.
To a hammer, everything is a nail. I love technology and science, but that doesn't mean that is what makes the best essay. Relationships are key to our survival, so I suggest personal relations could be just as good an essay as anything scientific - yes, I'm suggesting that social science isn't really a science.
This also then leads to timelessness. The essay on natural selection is timeless. It wouldn't be written today, but it is still a viewport into the discovery, and of the time, and the knowledge is still valid. Maybe I am misunderstanding the point about timelessness.
> The other reason the initial question matters is that you usually feel somewhat obliged to stick to it.
I also disagree with this comment. I'm currently writing a talk I've been asked to present, and in the process of my writing and researching, I've discovered a whole new and better question. I find the point of a good question is that it can lead to better questions. I do feel that PG is suggesting this as well, but maybe this one sentence just stuck out to me.
Some of my favorite books are old literature that describe relationship dynamics I still see playing out between people today. Swann’s Way, The Brothers Karamazov, even Canterbury Tales…
I think it was Vonnegut that wrote that the best books are the ones that tell you what you already know [but maybe you didn’t know you knew till you read it]. That’s something like surprise, and yet something like the opposite.
In reality and practice, no effort at science is 100% "really a science".
But for "social science":
(1) It can be a goal to have some of social science be real science.
(2) My wife's Ph.D. was in social science, and her dissertation was real science. The work started with a real question, in social science, gathered some real data, analyzed the data statistically, and came to some real conclusions. The conclusions were "new, correct, and significant". Some science? Yup.
That sounds like a trekie concluding that the best literature is "Star Trek" licensed novels...
Sorry, but PG didn't invent or define the essay.
It is a form that is centuries old (and in prior incarnations, millenia old) and have its own long cultural history, classics, and canon, few of the latter being about "important scientific or technological discoveries".
Maybe if instead of "best" he changed it to "most impactful scientifically" he'd have a point.
This feels like that paper, except the author isn’t a stressed-out college sophomore but a 59 year old “thought leader” with a large audience who gets Sam Altman to read his drafts. No attempt to engage with the history of the genre at all, or with the many other writers who have considered the form, and no examples cited except a famous scientific article by Darwin.
> I blame Eric Raymond and to a lesser extent Dave Winer for bringing this kind of schlock writing onto the Internet. Raymond is the original perpetrator of the "what is a hacker?" essay, in which you quickly begin to understand that a hacker is someone who resembles Eric Raymond.
> The whole genre reminds me of the the wooly business books one comes across at airports ("Management secrets of Gengis Khan", the "Lexus and the Olive Tree") that milk a bad analogy for two hundred pages to arrive at the conclusion that people just like the author are pretty great.The author (who is on HN, btw, too) does go into a more substantial difference of the nature of the two endeavours too, but the whole post is in a jocking tone. And yes, it has "an axe to grind", but it's not about who gets the girl. It's about taking down essays they consider pompous and self-congratulatory.
Disney can predict with a high degree of certainty that its superhero movies will do well, as there is a large, built-in market for those movies, and they tend to be conceived on the same creative blueprint or foundation. Even its 'duds' are still profitable.
But this is not possible with writing, especially not internet writing. What is the market for short-form contrarian non-fiction? Who knows. It's hit or miss, mostly miss
Google something like "Marvel superhero fatigue". It seems that either the movie quality is decreasing or the audience is becoming more and more bored by these movies.
- David Lubars
I know Paul does have some trusted readers that he shares early drafts with. Maybe he uses those readers as his focus group?
Editors exist for a reason.
I believe it is possible/likely that the best essay in the world could not be realized as such (thus, it wouldn't/couldn't be the best essay in the world), because most people couldn't even get by one of these let alone all three...and, there's likely to be many other hurdles one would have to make it over.
Instead, the best movie [0] is about this.
Michel de Montaigne, "Essays": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
David Hume, "Of Public Credit:": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
David Thoreau, "Civil Disobedience": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Virginia Woolf, "A Room of One’s Own": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
George Orwell, "The Lion and the Unicorn": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Simone Weil, "Human Personality": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
James Baldwin, "Notes of a Native Son": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Susan Sontag, "Against Interpretation": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Joan Didion, "The White Album": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
David Foster Wallace, "Up, Simba!": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Umberto Eco, "Thoughts on Wikileaks": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Ta-Nehisi Coates, "The Case for Reparations": https://www.lrb.co.uk/podcasts-and-videos/podcasts/history-o...
Q. How well--perhaps on a scale of 1 to 10--does this essay follow its own recommendations? Is it a great essay by its own standards?
A. The essay "The Best Essay" does follow many of its own recommendations, but it also falls short in a few areas. Let's evaluate the essay based on the 10 rules it sets out for writing great essays.
Important topic with surprising insights: 8/10. The topic of what makes a great essay is important for writers, and the author provides some surprising insights, such as the idea that the best essay would be about a groundbreaking scientific or technological discovery.
Compelling question: 7/10. The essay starts with the question of what the best essay would be like, which is interesting but not as compelling as it could be.
Thinking out loud and refining: 9/10. The author does an excellent job of thinking out loud and refining their ideas throughout the essay.
Recursive process: 8/10. The essay follows a recursive process, with responses generating further questions and exploration.
Cutting content: 6/10. While the author mentions cutting a 17-paragraph subtree, it's unclear how much unnecessary content remains in the essay.
Following novelty and generality: 7/10. The essay explores novel ideas but could have delved deeper into some of the more interesting tangents.
Writing about interesting topics: 9/10. The author is clearly interested in the topic of essay writing and what makes essays great.
Breadth and depth of knowledge: 8/10. The author demonstrates a broad knowledge of essay writing and related topics, but the depth could be further developed in some areas.
Aiming for timelessness: 7/10. The essay discusses the concept of timelessness but doesn't fully achieve it, as some of the ideas may not be surprising to readers well-versed in essay writing.
Discovering ideas through questions and answers: 8/10. The essay does explore ideas through asking questions and refining answers, but the most valuable questions could have been pursued further.
Overall, the essay scores well in following its own advice, with an average score of around 7.7/10. However, it falls short of being a truly great essay by its own standards. The author could have pushed further in exploring the most compelling questions, cutting unnecessary content, and aiming for more timeless insights. Nonetheless, it remains a thought-provoking and insightful piece on the art of essay writing.
> If caring about it is one of the criteria for a good initial question, then the optimal question varies from person to person.
with another submission posted some 15 minutes later [0] noting J. R. Ackerley’s ability to make anything compelling through the sheer weight of care
The "best essay" for an author is certainly cantered on their care. Yet I feel in the current nonchalant, urbane and cynical climate if the audience sense any sniff of "care", it is to be savaged.
Unless it tickles something sentimental or ironic, care is despised and dismissed as weakness that "doesn't make a living" or is some wonky "ideology".
But then people would call me cynical for saying that :)
This is pure gold.
What is the best car? Some cars are real stinkers. It's not hard to find cars that are better than Zastava Yugo in every way. For most cars you can find other cars that are in the same class, but just better. Clearly, a comparison between cars can be made. Car reviewers make this their job.
But from the existence of bad and mediocre cars it doesn't follow that there is such a thing as a "best car". How large is your family? Do you want to tow a boat? Do you like to go fast on curvy roads? Do you care more about acceleration or range? How tall are your rear passengers?
There are good essays and there are bad essays. Some essays, perhaps, can be considered the very best in their category. But you can't rank order essays across categories, just like you can't argue that a Ford F150 is objectively better or worse than a Mazda Miata.
(And if this makes me a small-minded pseudo-intellectual so be it.)
Perhaps if you limit yourself to a hidebound traditionalist view of the form but this topic has been explored by the Moving Picture Essay https://www.youtube.com/watch?v=oMDcIApBtL8
"I love questions that seem naughty in some way — for example, by seeming counterintuitive or overambitious or heterodox. Ideally all three. This essay is an example. Writing about the best essay implies there is such a thing, which pseudo-intellectuals will dismiss as reductive, though it follows necessarily from the possibility of one essay being better than another."
That's not what "follows necessarily" means Mr. Graham. Are these pseudo-intellectual spectres literally just you strawmanning your own counterfactual? I can't imagine you actually listening to anyone you would call a pseudo-intellectual.
Then I guess after I went to grad school and actually read a lot and tried to do some science and research. I find this kind of popular writing superficial, flawed, inefficient, and even conceited. That's just my subjective experience. But it seems some young people in tech just lap it up, he functions as a kind of thought leader for the industry.
I'm not saying that his arguments are 'wrong' persay, it's just that it's not particularly original or interesting.
You're spot on. Looking to this very comment section, you have people calling this line out as being enlightening:
>While breadth comes from reading and talking and seeing, depth comes from doing.
I would hope this is self evident to anyone who has ever tried _anything_.
Wouldn't it be easier on the reader to simply include the Notes in the body part of the essay? Especially when the notes are short. For example the aforementioned note no. 4 goes:
"[4]Corollary: If you're not intellectually honest, your writing won't just be biased, but also boring, because you'll miss all the ideas you'd have discovered if you pushed for the truth."
Which is short, relevant, and insightful. What's the criteria for which piece of text goes into the body of the essay, and which goes to Notes? I guess if a substantial percentage of readers skip the Notes part, then it's a time-saving practice to separate the two.
I believe that the reading flow is quite important, probably near the top of importance, because if the reader gets into the reading flow they're much more likely to read it all and enjoy it!
> The Mezzanine tells this story through the extensive use of footnotes—some of them comprising the bulk of the page—as the narrator travels through his own mind and past. The footnotes are quite detailed and sometimes diverge into multiple levels of abstraction. Near the end of the book, there is a multi-page footnote on the subject of footnotes themselves.
I'm reminded of a philosophy joke:
An angel came down for a meeting of the American Philosophical Association. Greeting the assembled philosophers, the angel offered to answer any single question for them. Immediately the philosophers set to arguing about what they should ask. So the angel said, “Alright, you figure out what you want to ask. I’ll come back tomorrow.” And he left the philosophers to deliberate.
As the philosophers argued about the best question to ask, one philosopher suggested: “What is the best question to ask?”, in the hope that some day another angel might make a similar offer, at which point they could then ask the best question. But this suggestion was rejected by those who feared that no such opportunity would arise and did not want to waste their only question.
Another philosopher suggested asking "What is the answer to the best question we could ask?" Which was roundly criticized because of the possibility that the answer would give no hint to the question it answered. One philosopher cheekily pointed out that the angel might reply "42 of course."
Finally, the philosophers agreed on the following question: “What is the ordered pair whose first member is the best question to ask, and whose second member is the answer to that question?” Satisfied with their decision, the philosophers went to bed.
The next day, they posed their question, and the angel replied: “Of course: it is the ordered pair whose first member is the question you just asked, and whose second member is the answer I am now giving.” And then he disappeared.
The philosophers could have asked "What is the meaning of life?" and the angel would answer "Of course, it is the meaning of life," without any of this ordered pair nonsense.
But it's not a joke about programmers, so I don't get it either, ha!
> What is the ordered pair whose first member is the best question to ask, and whose second member is the answer to that question?
Is isomorphic to:
> What is the highest cardinality set in the set of all sets?
"What is the best question in the set of all questions?" is itself "what is the best question in the set of all questions?"
"What is the highest cardinality set in the set of all sets?" is itself "the set of all sets".
"What is the answer to that question?" is an assumption of bounded size, and therefore isomorphic to saying "the largest set that is not itself."
> What is the answer to the best question we could ask?" Which was roundly criticized because of the possibility that the answer would give no hint to the question it answered. One philosopher cheekily pointed out that the angel might reply "42 of course."
42 is a valid answer because if you assume two contradictory statements are true at the same time, then all statements are true. The cheeky philosopher is pointing out that "What is the answer to the best question we could ask?" implies self reference, which implies a contradiction.
The philosophers asked a self referential question and got a self referential answer.
At least that's my take.
Solving the halting problem requires solving the halting problem's halting problem.
Asking the best question requires the answer to what the best question is.
— some shiny demon in the middle of some long and lonely road
All it takes is motivation. A lesson there for all of us, I think
The premise is wrong (or at least not obviously right) IMO, so I have a hard time taking any of the rest of it seriously. Could the best essay not be the most emotionally moving? The best when heard aloud? The most convincing call to action? The most accurate? The highest grossing? Driving the most engagement? What about the topic (any topic) that you could tell the /most interesting surprise/ about?
If Paul Graham didn’t run this company he certainly would not make it to the front page for his lazy philosophy.
- Gödel, a bastard and a cynic, having a bad essay day after attempting to out drink Hemingway, yet somehow expressing a surprising deep truth
On the most important topic you could tell people something surprising about. And all those things you said. And … and (yes, that “and”) … so much more!
We are talking about the join of all essay joins. The essay supremum. By definition
I think we can agree there is a limit to the length of this essay. Proof by absurdum: if the essay is too long to read, that would imply unread word choices make a difference. So there is finite length N.
Now we just search for it. Unleash the monkeys! Or today, unleash the competitive adversarial AI cohort!
We can find it, just not copyright it. Which I think, is the best outcome for humanity. Assuming this essay is aligned with our interests, but we have set that unboxing in motion now so it’s not worth worrying about
I think it’s best to maintain some humor when talking about mythical bests, one way or another
I think the grammar we want here is:
"What would make it special would be what it is about."
Did he step off a refugee boat as Pavel Gramko?