3,228 karma · joined February 5, 2018
Email: [firstname]@fehri.ng
We're a team of frontier AI (including ex-Anthropic, METR, OpenAI), insurance, and governance people building incentive infrastructure to help ASI go well. We red-team, certify, and insure AI agents - working with AI category leaders like Cursor, ElevenLabs, Fin, Harvey, Lovable, and UiPath; auditors like Schellman and KPMG; insurance partners at Lloyd's of London; and the AIUC-1 Consortium https://www.aiuc-1.com/consortium.
We've raised $55m from investors including Ribbit, First Harmonic, and Nat Friedman, and we're hiring across the board in San Francisco.
Apply to our MTS role: https://jobs.ashbyhq.com/aiuc/2816bb05-2a1f-4600-8780-deb152...
Browse all of our open roles: https://jobs.ashbyhq.com/aiuc
[0] https://bw.bse.eu/wp-content/uploads/1564_compressed.pdf
[1] https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3635535
Still sensible to mark proprietary for now though.
[0] https://thinkingmachines.ai/tinker/
[1] https://www.joelonsoftware.com/2002/06/12/strategy-letter-v/
My reading of this is that OpenAI's contract with the Pentagon only prohibits mass surveillance of US citizens to the extent that that surveillance is already prohibited by law. For example, I believe this implies that the DoW can procure data on US citizens en masse from private companies - including, e.g., granular location and financial transaction data - and apply OpenAI's tools to that data to surveil and otherwise target US citizens at scale. As I understand it, this was not the case with Anthropic's contract.
If I'm right, this is abhorrent. However, I've already jumped to a lot of incorrect conclusions in the last few days, so I'm doing my best to withhold judgment for now, and holding out hope for a plausible competing explanation.
(Disclosure, I'm a former OpenAI employee and current shareholder.)
I have two qualms with this deal.
First, Sam's tweet [0] reads as if this deal does not disallow autonomous weapons, but rather requires "human responsibility" for them. I don't think this is much of an assurance at all - obviously at some level a human must be responsible, but this is vague enough that I worry the responsible human could be very far out of the loop.
Second, Jeremy Lewin's tweet [1] indicates that the definitions of these guardrails are now maintained by DoW, not OpenAI. I'm currently unclear on those definitions and the process for changing them. But I worry that e.g. "mass surveillance" may be defined too narrowly for that limitation to be compatible with democratic values, or that DoW could unilaterally make it that narrow in the future. Evidently Anthropic insisted on defining these limits itself, and that was a sticking point.
Of course, it's possible that OpenAI leadership thoughtfully considered both of these points and that there are reasonable explanations for each of them. That's not clear from anything I've seen so far, but things are moving quickly so that may change in the coming days.
[0] https://x.com/sama/status/2027578652477821175
[1] https://x.com/UnderSecretaryF/status/2027594072811098230
I think a competent opposition party would be great for the US. But regardless of the candidate, US voters had three clear choices in the 2024 Presidential election: (1) I support what Trump is going to do, (2) I am fine with what Trump is going to do (abstain/third-party), (3) Kamala Harris. I think it’s extremely clear 3 was the best choice, but it was the least popular of the three.
Still a sad outcome for now. People will die because of this decision.
The fertility rate has decreased significantly for US-born women of every race and ethnicity since the 1990s. I couldn't quickly find good stats on trend in birth control usage or labor force participation by race, ethnicity, or immigration status, but I'm skeptical that the trend is in the opposite direction for any particular demographic.
So I expect the claims in my previous comment still hold even for, e.g., native-born whites as a subgroup: flat-to-decreasing birth control usage, declining labor force participation, but still declining fertility rate. Obviously the magnitudes of those changes may be different at the subgroup level, but I don't see how the data is compatible with the claims of the comment I initially replied to.