2,422 karma · joined November 9, 2020
Former head of product and lead for assessments @ Triplebyte.
To be fair, this is true of any equity model, which is precisely why most engineers (at least in our data set) place very little weight on equity.
- 23% of candidates tell us outright that they don't want us to match them with crypto (see chart here: https://bsky.app/profile/otherbranch.bsky.social/post/3l4wod... )
- About 36% of candidates who didn't, that we end up reaching out to about a crypto role, decline it explicitly because it's crypto.
So: 23%, plus 36% of the remaining 77%, equals just north of 50%. Add in a few who don't turn it down outright but who end up withdrawing later, and you get a bit north of half.
If you were doing it that way, you could also set distributional values for the variables. Rather than x = {500, 1000}, you could set (say) X ~ N(750, 100) and have it pull samples from that distribution. If you really wanted to get fancy you could take advantage of known results on operations on distributions to keep things exact for a lot of common calculations, then turn to numerical methods for uglier operations (at which point you're basically building a wrapper around one of the usual statistical libraries, I suppose).
(EDIT: apparently this already exists, see other comments in this thread.)
This is borne out by results downstream with clients. No client we've sent more than a couple of people has ever had concerns about quality, so we're fairly confident that we are in fact detecting the cheating that is happening with reasonable consistency.
I actually just looked at our data a few days ago to see how candidates who listed LLMs or related terms on their resume did on our interview. On average, they did much worse (about half the pass rate, and double the hard-fail rate). I suspect this is a general "corporate BS factor" and not anything about LLMs specifically, but it's certainly relevant.
That is sometimes true, but as I said in another comment, I think this is on the weaker end of criticisms because it doesn't really apply to the best of that community's members and the best of its claims, and in either case isn't really a consequence of their explicit values.
> Logic, like software and for the same reason, is “garbage in, garbage out.” If even one of the inputs (premises, priors) is mistaken the entire conclusion can be wildly wrong. Errors cascade, just like software.
True, but an odd analogy: we use software to make very important predictions all the time. For every Therac-25 out there, there's a model helping detect cancer in MRI imagery.
And, of course, other methods are also prone to error.
> That's why every step needs to be checked with experiment or observation before a next step is taken.
Depends on the setting. Some hypotheses are not things you can test in the lab. Some others are consequences you really don't want to confirm. Setting aside AI risk for a second, consider the scientists watching the Trinity Test: they had calculated that it wouldn't ignite the atmosphere and incinerate the entire globe in a firestorm, but...well, they didn't really know until they set the thing off, did they? They had to take a bet based on what they could predict with what they knew.
I really don't agree with the implicit take that "um actually you can never be certain so trying to reason about things is stupid". Excessive chains of reasoning accumulate error, and that error can be severe in cases of numerical instability (e.g. values very close to 0, multiplications, that kind of thing). But shorter chains conducted rigorously are a very important tool to understand the world.
The reason is that murdering people for organs has massive second-order effects: public fear, the desire to avoid medical care if harvesting is done in those contexts, disproportionate targeting of the organ harvesting onto the least fortunate, etc.
Even the oracle version is already really dangerous in the wrong hands. A totalitarian government doesn't need to have someone listening to a few specific dissidents if they can have an AI transcriber write down every word of every phone conversation in the country, for example. And while it's certainly not error-proof, asking an LLM to do something like "go through this list of conversations and flag anything that sounds like anti-government sentiment" is going to get plenty of hits, too.
-------
So, toy model: imagine the brain has a single 1-dimensional happiness value that changes over time. You can be +3 happy or -2 unhappy, that kind of thing. Everyone knows when you're very happy you tend to come down, and when you're very sad you tend to eventually shake it off, meaning that there is something of a tendency towards a moderate value or a set-point of sorts. For the sake of simplicity, let's say a normal person has a set point of 0, then maybe a depressive person has a set point of -1, a manic person has a set point of +1, that sort of thing.
Mathematically, this is similar to the equations that describe a spring. If left to its own devices, a spring will tend to its equilibrium value, either exponentially (if overdamped) or with some oscillation around it (if underdamped). But if you're a person living your life, there are things constantly jostling the spring up and down, which is why manic people aren't crazy all the time and depressed people have some good days where they feel good and can smile. Mathematically, this is a spring with a forcing function - as though it's sitting on a rough train ride that is constantly applying "random" forces to it. Rather than x'' + kx = 0, you've got x'' + kx = f(t) for some external forcing function f(t), where f(t) critically does not depend on x or on the individual internal dynamics involved.
These external forcing functions tend to be pretty similar among people of a comparable environment. But the internal equilibria seem to be quite different. So when the external forcing is strong, it tends to pull people in similar directions, and people whose innate tendencies are extreme tend to get pulled along with the majority anyway. But when external forcing is weak (or when people are decoupled from its effects on them), internal equilibria tend to take over, and extreme people can get caught in feedback loops.
If you're a little more ML-inclined, you can think about external influences like a temperature term in an ML model. If your personal "model" of the world tends to settle into a minimum labeled "completely crazy" or "severely depressed" or the like, a high "temperature" can help jostle you out of that minimum even if your tendencies always move in that direction.
Basically, I think weird nerds tend to have low "temperature" values, and tend to settle into their own internal equilibria, whether those are good, bad, or good in some cases and bad in others (consider all the genius mathematicians who were also nuts). "Normies", for lack of a better way of putting it, tend to have high temperature values and live their lives across a wider region of state space, which reduces their ability to wield precision and competitive advantage but protects them from the most extreme failure-modes as well.
I broadly agree with this criticism, but I also think it's kind of low-hanging. At least speaking for myself (a former member of those circles), I do indeed sit down and write quantitative models when I want to estimate things rigorously, and I can't be the only one who does.
> Indeed, as well as just ignoring that uncertainties about the state of the world or the model of interaction utterly dominate any "calculation" that you could hope to do.
This, on the other hand, I don't think is a valid criticism nor correct taken in isolation.
You can absolutely make meaningful predictions about the world despite uncertainties. A good model can tell you that a hurricane might hit Tampa but won't hit New Orleans, even though weather is the textbook example of a medium-term chaotic system. A good model can tell you when a bridge needs to be inspected, even though there are numerous reasons for failure that you cannot account for. A good model can tell you whether a growth is likely to become cancerous, even though oncogenesis is stochastic.
Maybe a bit more precisely, even if logic cannot tell you what sets of beliefs are correct, it can tell you what sets of beliefs are inconsistent with one another. For example, if you think event X has probability 50%, and you think event Y has probability 20% conditional on X, it would be inconsistent for you to believe event Y has a probability of less than 10%.
> The world at large is does not spend all its time in lesswrongian ritual multiplication or whatever... but this is not because they're educated stupid
When I thought about founding my company last January, one of the first things I did was sit down and make a toy model to estimate whether the unit economics would be viable. It said they would be, so I started the company. It is now profitable with wide operating margins, just as that model predicted it would be, because I did the math and my competitors in a crowded space did not.
Yeah, it's possible to be overconfident, but let's not forget where we are: startups win because people do things in dumb inefficient ways all the time. Sometimes everyone is wrong and you are right, it's just that that usually happens in areas where you have singularly deep expertise, not where you were just a Really Smart Dude and thought super hard about philosophy.
Take, for example, 80,000 Hours, among the more prominent EA organizations. Their top donors (https://80000hours.org/about/donors/) include:
- SBF and Alameda Research (you probably knew this),
- the Berkeley Existential Risk Initiative, founded (https://www.existence.org/team) by the same guy who founded CFAR (the Center for Applied Rationality, a major rationalist organization)
- the "EA infrastructure fund", whose own team page (https://funds.effectivealtruism.org/team) contains the "project lead for LessWrong.com, where he tries to build infrastructure for making intellectual progress on global catastrophic risks"
- the "long-term future fund", largely AI x-risk focused
and so on.
The problem with rationalists/EA as a group has never been the rationality, but the people practicing it and the cultural norms they endorse as a community.
As relevant here:
1) While following logical threads to their conclusions is a useful exercise, each logical step often involves some degree of rounding or unknown-unknowns. A -> B and B -> C means A -> C in a formal sense, but A -almostcertainly-> B and B -almostcertainly-> C does not mean A -almostcertainly-> C. Rationalists, by tending to overly formalist approaches, tend to lose the thread of the messiness of the real world and follow these lossy implications as though they are lossless. That leads to...
2) Precision errors in utility calculations that are numerically-unstable. Any small chance of harm times infinity equals infinity. This framing shows up a lot in the context of AI risk, but it works in other settings too: infinity times a speck of dust in your eye >>> 1 times murder, so murder is "justified" to prevent a speck of dust in the eye of eternity. When the thing you're trying to create is infinitely good or the thing you're trying to prevent is infinitely bad, anything is justified to bring it about/prevent it respectively.
3) Its leadership - or some of it, anyway - is extremely egotistical and borderline cult-like to begin with. I think even people who like e.g. Eliezer would agree that he is not a humble man by any stretch of the imagination (the guy makes Neil deGrasse Tyson look like a monk). They have, in the past, responded to criticism with statements to the effect of "anyone who would criticize us for any reason is a bad person who is lying to cause us harm". That kind of framing can't help but get culty.
4) The nature of being a "freethinker" is that you're at the mercy of your own neural circuitry. If there is a feedback loop in your brain, you'll get stuck in it, because there's no external "drag" or forcing functions to pull you back to reality. That can lead you to be a genius who sees what others cannot. It can also lead you into schizophrenia really easily. So you've got a culty environment that is particularly susceptible to internally-consistent madness, and finally:
5) It's a bunch of very weird people who have nowhere else they feel at home. I totally get this. I'd never felt like I was in a room with people so like me, and ripping myself away from that world was not easy. (There's some folks down the thread wondering why trans people are overrepresented in this particular group: well, take your standard weird nerd, and then make two-thirds of the world hate your guts more than anything else, you might be pretty vulnerable to whoever will give you the time of day, too.)
TLDR: isolation, very strong in-group defenses, logical "doctrine" that is formally valid and leaks in hard-to-notice ways, apocalyptic utility-scale, and being a very appealing environment for the kind of person who goes super nuts -> pretty much perfect conditions for a cult. Or multiple cults, really. Ziz's group is only one of several.
Yet another of a series of recent lessons in listening to people - particularly powerful people focused on PR - when they claim a neutral moral principle for what happens to be pragmatically convenient for them. A principle applied only when convenient is not a principle at all, it's just the skin of one stretched over what would otherwise be naked greed.
The fact is that even highly intelligent people are not smart enough to avoid deliberate disinformation efforts by actors with a thousand times their resources. Not reliably. You might avoid 90% of them, but if there's a hundred such efforts on at a time, you're still gonna end up being super wrong about ten things. You detect the Nigerian prince phone call, but you don't detect the CFO deepfake on your Zoom call, that kind of thing.
When you say it's a "skill issue", I think you're basically expecting a skill bar that is beyond human capability. It's like saying the fact that you can get shot is a "skill issue" because in principle you could dodge every bullet like you're in the Matrix - yeah, but you're not actually able to do that!
> but you can't hold the position that we're all sheeple and deserve to be free-thinkers at the same time.
I don't. I believe it's mostly the first one. I don't know what other conclusion I can possibly take from everything that has happened in the history of the internet - including having fallen rather badly for disinformation myself a couple of times in the past.
You should be a freethinker when it comes to areas where you have unique expertise: your specific vocation or field of study, your unique exposure to certain things (say, small subgroups you happen to be in the intersection of), and your own direct life experiences (do you feel good today? are the people you know struggling?). Everywhere else, you should bet on institutions that have otherwise proved to earn your trust (by generally matching your expectations within the areas where you do have expertise or making observably correct past predictions).
Yes, it is partially a problem with improper use. But as a practical matter, we know that convenience and confidence are powerful pulls to very large portions of the population. At some point, you have to treat human nature (or at least, human nature as manifested in the world we currently have) as a given, and consider things in light of that fixed background - not in light of the background of humanity you wish we had. If we lived in a world where everyone, or even where most people, behaved reasonably, we'd do a lot of things differently.
Previous propaganda efforts also didn't automatically construct a roughly-self-consistent worldview on demand for whatever false information you felt like feeding into them, either. So I do think LLMs are a powerful tool for that, for roughly the same reason they're a powerful tool in other contexts.
This particular approach is honestly kind of funny, though. It's so transparent it reads like parody.
The lesson of everything that has happened in tech over the past 20 years is that what tech can do and what tech will do are miles apart. Yes, AGI could give everyone a free therapist to maximize their human well-being and guide us to the stars. Just like social media could have brought humanity closer together and been an unprecedented tool for communication, understanding, and democracy. How'd that work out?
At some point, optimism becomes willfully blinding yourself to the terrible danger humanity is in right now. Of course founders paint the rosy version of their product's future. That's how PR works. They're lying - maybe to themselves, and definitely to you.
No. A good-faith failure is different from a lie.
The rule of thumb I use to handle ambiguous situations is "if my incentives were different, would I be saying something else right now?"
> Of course there are fraudsters out there, but I view most founders as rampant optimists instead of liars.
Most founders are both.
They're rampant optimists in the sense that they believe in their thing so much that it overrides all other concerns. But that often makes them liars via an argument of the form "my thing is so important and will change the world so much that I have to lie now to make sure it can be so great".
I'm not saying founders are ogres. The kind of lying they do derives from fairly ordinary human failures. But it's still lying, it's still normalized, and it still has terrible consequences all the time.
(As always, you should trust what a founder says publicly about their company approximately not at all. If you want the answer for yourself, you gotta do it yourself, because you only know if you're lying or not. But I have my answer, I think.)
Roughly put:
- A particle is a "minimum stretching" of a field.
- The "stiffness" corresponds to the energy-per-stretch-amount of the field (analogous to the stiffness of a spring).
- So the particle's mass = (minimum stretch "distance") * stiffness ~ stiffness
The author's point is that you don't need to invoke virtual particles or any quantum weirdness to make this work. All you need is the notion of stiffness, and the mass of the associated particle and the limited range of the force both drop out of the math for the same reasons.
> The struggles of single parenthood for both the child-rearing parent and the children of divorce are very real and well-documented
The question isn't "does it suck to be a single parent or the child thereof". It's "is it worse than the alternative?" This is "people who see a doctor are more likely to die"-style reasoning that conflates a preexisting problem with an imperfect solution.
Kids need examples of loving and trusting relationships. That's how they learn how to build them themselves. They learn conflict resolution, compromise, and communication by observing their parents' relationship. And when that relationship is at best one of civil distance, a child can't learn what they need to learn. It's even worse when - as in my case - the kid is the channel through which a lot of the marital conflict plays out.
When my parents finally did split up (after I was already an adult), it was a relief to everyone involved. They're both better off. If they ever tried to get back together, I'm pretty sure I and my brothers and sisters would go slap them and tell them to not do the dumb thing.
> Keeping you away from illegal drugs meant you had the opportunity to get properly prescribed and managed psychiatric medication instead of the too-common path of self-medicating with the recreational drug-du-jour, with much worse long-term consequences.
Yes, but you're leaving out the part where unmanaged mental illness almost killed me before I got on properly prescribed and managed psychiatric medication. In almost every timeline but this one, it probably did kill me.
> You know the YOLO fad passed so quickly because kids realized the permanent scars left by “experimenting”, especially if there are no rich parents to pick up the pieces.
I take a different lesson from this. I think your point about "no rich parents to pick up the pieces" is one of the reasons that millennials and zoomers are struggling: we/they've grown up in a competitive world that doesn't allow them room for normal human error.
Making mistakes - or the safety to make them - is a critical part of growing as a person. It's an investment, the same way a company invests in R&D. It pays dividends. But it has short term costs you can't pay if you're always trying to make ends meet.
Yes, there are experiments you shouldn't perform because their costs outweigh their benefits, but most youthful indiscretions are not irreversibly damaging. One way to tell is that many of the richest and most powerful people around had fairly wild youths and tended to be fairly aggressively risk-taking.
Did it protect me from driving drunk when I was in college? Yeah, but it also "protected" me from having a healthy social life because I couldn't engage with any sort of normal behavior. Did it protect me from getting on drugs? Yeah, but it also "protected" me from getting on desperately needed psychiatric medication because that was for Other People, Who Are Too Weak To Handle Their Problems Properly. Did it protect my parents from sleeping around? Yeah, but it also locked them into a miserable marriage for half their lives, leaving both them and their children with heaping scoops of extra trauma.
Maybe that trade-off is worth it, but if you're going down this route, make sure your kids know how to experiment and screw up sometimes, too.
I'm inclined to say that a better solution is to recognize that none of us exist in a vacuum. When our societies are full of toxicity and manipulation and brainrot, we can't escape those things without cutting off a part of ourselves. Sometimes we have to do that, but ultimately what we need is a healthy culture to live in - and if we don't have one, we should be working to make one.
You won't see PG writing an article on how homeless people in SF should be more pro-breaking-the-rules. Because it's OK to steal from your users, to inflate your growth numbers, to make false promises to build your initial userbase, but it's not OK not to shoplift from the Safeway or do drugs on the BART. That's the kind of breaking the rules that isn't cool, edgy, smart, and most importantly high status and beneficial to Paul Graham. Don't you think that double standard is a bit suspect, that he's "pro-breaking-the-rules" exactly when the rules restrain him and not others, when it's the rules he thinks are stupid and not the rules someone else thinks are stupid?
You won't see PG writing an article about how it's bad to deny a 15 year old medical information on puberty blockers. That is, undeniably, censorship in its very simplest form: it's the suppression of information out of a belief that it is in some way dangerous to let people know about it. But most of the people who claim to be so concerned with censorship won't say a word against it, and a lot actively support it, because it stops being "censorship" when it's something they like.
And, of course, the idea that the anti-woke crowd is "anti-authoritarian" is kind of laughable right now, given their response to the incoming administration.
The change isn't that his (or other tech elites') ostensible values have changed. It's that their ostensible values have become increasingly transparently hypocritical. PG hasn't become less of a hacker: it's "hacker culture" itself, especially as represented and hijacked by venture capital, that is not what we (or at least what I) hoped it was.
Rule breaking for me, and not for thee sums up my objection, I suppose.
Depression isn't a failing on the person's part, and it isn't stupidity. Nor is suicide resulting from depression. It's a disease, and you "die from suicide" the same way you "die from cancer" - from the effects of your disease disrupting vital functions of your existence until you can no longer survive.
For me, at least, understanding and healing from severe mental illness required understanding that the illness wasn't "me". It was this crappy thing I had to live with because some part of my brain Just Does That Sometimes. See [1] among other posts, but the only way I've ever found to beat my own tendencies towards mental illness - and they are extremely strong - is to treat them like a chronic disease. The same way that a person with liver disease has to avoid drinking, I have to avoid the things that trigger my own chronic depression.
A Cherokee man named Sequoyah ended up serving as a soldier in one of the early conflicts between English settlers and other Native American groups. In this case, his people were on the side of English settlers against rival tribes (see [2]), so he was serving alongside English soldiers in the conflict. He saw them reading and writing - something he couldn't do, since the Cherokee language had no written script at the time - and went "wait this seems like it would be pretty useful".
So Sequoyah, who until this point had never encountered a writing system, decides he's going to make one. He starts by trying to create a symbol for every word, but decides that's too difficult, so he switches to symbols for syllables instead. It took him ten years to finish it, but once he did, it was adopted by nearly every speaker of Cherokee within only a few years. It's still the primary writing system for the Cherokee language today.
[1] http://www.stat.columbia.edu/~gelman/research/published/AOAS...