Plenty of legitimate things to criticize EA for, no need to smear them by association with someone who's never claimed to be an EA and hasn't obviously behaved like one either.
686 karma · joined January 11, 2021
They threatened to block the employee who pushed back on the non-disparagement from participating in tender offers, while allowing other employees to sell their equity (which is what the tender offers are for). This is not a "market term".
Extremely reasonable position, and I'm glad that every time some idiot brings it up in the EA forum comments section they get overwhelmingly downvoted, because most EAs aren't idiots in that particular way.
I have no idea what the rest of your comment is talking about; EAs that have opinions about AI largely think that we should be slowing it down rather than speeding it up.
I don't see Anthropic making very confident claims about when they're going to achieve AGI (however you want to define that). Predicting how long it'll take to produce a specific novel scientific result is, by its very nature, pretty difficult. (You might have some guesses, if you have a comprehensive understanding of what unsolved dependencies there are, and have some reason to believe you know how long it'll take to solve _those_, and that's very much not the case here. But if you're in that kind of situation, it's much more likely you're dealing with an engineering problem, not a research problem.) Elsewhere in the comments on this link, their CISO predicts a 50% chance of hitting capabilities that'll trigger their ASL-3 standard in the next 6 months (my guess is on the strength of its ability to find vulnerabilities in open-source codebases). That's predicting the timeline for a small advancement in a relatively narrow set of capabilities where we can at least sort of measure progress.
State your accusation plainly: you think that Anthropic is spending a double-digit percentage of its headcount on pretending to care about catastrophic risks, in order to better fleece investors? Do you think those dozens or hundreds of employees are all in on it too? (They aren't; I know a bunch of people at Anthropic and they take extinction risk quite seriously. I think some of them should quit their jobs, but that's a different story.)
Uh, no? In fact humans do not currently have a way of causing human extinction (nukes wouldn't do it with current stockpiles). That would require scientific advancements which we have yet to make, and most possible routes seem pretty non-trivial... except, unfortunately, AI.
Sam Altman: "Development of superhuman machine intelligence (SMI) [1] is probably the greatest threat to the continued existence of humanity. " (https://blog.samaltman.com/machine-intelligence-part-1, published before he co-founded OpenAI)
Dario Amodei: "I think at the extreme end is the Nick Bostrom style of fear that an AGI could destroy humanity. I can’t see any reason and principle why that couldn’t happen." (https://80000hours.org/podcast/episodes/the-world-needs-ai-r..., published before he co-founded Anthropic)
Shane Legg: (responding to "What probability do you assign to the possibility of negative consequences, e.g. human extinction, as a result of badly done AI?") "...Maybe 5%, maybe 50%. I don't think anybody has a good estimate of this." (https://www.lesswrong.com/posts/No5JpRCHzBrWA4jmS/q-and-a-wi...)
Technically Shane's quote is from 2011, which is a little bit after Deepmind was founded, but the idea that Shane in 2011 was trying to sow FUD in order to benefit from regulatory capture is... lol.
I wish I knew why they think the math pencils out for what they're doing, but Sam Altman was not plotting regulatory capture 9 years ago, nearly a year before OpenAI got started.
Note that this is not a valid argument against any argument that superintelligent AI might kill everyone; it's just a character attack.
Yes, indeed, which is why we should avoid creating something that will kill everyone...
...so I guess I'm glad to hear there's no difference in the amount of damage that can be inflicted by a baby, and by Vladimir Putin, and superintelligent AI really can't be any worse than either.
Astonishing claim, when some such researchers have (unfortunately) been responsible for many of the most impressive capabilities advancements in the last few years, and the three most cited AI researchers of all time are all doing extinction-risk-mitigation work (with Bengio and Sutskever both doing technical work; Hinton mostly seems to be focusing on outreach).
...does not follow? "If an atom bomb is actually a bomb, then it's no worse a threat than any other existing bomb known to man."
In fact both of those seem quite bad, both by regular industry standards, and even moreso as applied to OpenAI's specific situation.
It'd be a more interesting claim if it was argued that mutacin-1140 was particularly dangerous in the relevant doses (and this somehow never came up in the wild-type S. mutans that produce it).
Hanania is just some dude on twitter. He's not prominent in the rationalist community in any meaningful sense.
We have effectively no mechanistic understanding of frontier ML systems, in the sense that we have no idea how they do what they do and could not e.g. write human-readable code to perform comparable tasks, nor can we predict ahead of time what capabilities such systems will have when they're trained (being able to predict e.g. log-loss is _not_ the same as being able to predict specific capabilities).
Agreed. It seems remarkable that she dealt with her conflict of interest so thoroughly that she published an article that was mildly critical of the organization. I do not think her duty was to let Sam Altman seize power by firing her from the board on a flimsy pretext, though.
Criticism (especially valid criticism) is not sabotage, and in any case the board's responsibility is to advance the charter, not to support the (non-profit) company regardless of what it's doing.
I mean, yes? The board is explicitly there to replace the CEO if necessary. If the CEO stuffs the board full of their allies, it can no longer do that.
> First: most boards are accountable to something other than themselves. For the exact reason that it pre-empts that type of nonsense.
Boards of for-profits are accountable to shareholders because corporations with shareholders exist for the benefit of (among others) shareholders. Non-profit corporations exist to further their mission, and are accountable to the IRS in this regard.
> Second: the anti-Sam Altman argument seems to be "let's shut the company down, because that will stop AGI from being invented". Which is blatant nonsense; nothing they do will stop anyone else. (with the minimal exception that the drama they have incepted might make this holiday week a complete loss for productivity).
No, the argument is that Sam Altman trying to bump off a board member on an incredibly flimsy pretext would be an obvious attempt at seizing power.
> Third: in general, "publishing scholarly articles claiming the company is bad" is a good reason to remove someone from the board of a company. Some vague (and the fact that nobody will own up to anything publicly proves it is vague) ideological battle isn't a good enough rationale for the exception to a rule that suggests that her leaving the board soon would be a good idea.
This might be true w.r.t. for-profit boards (though not obviously so in every case), but seems nonsensical with non-profits. (Also, the article did not reductively claim "the company is bad".)