NIST staffers revolt against expected appointment of Paul Cristiano to NIST | Hacker News Reader

NIST staffers revolt against expected appointment of Paul Cristiano to NIST

venturebeat.com

48 pointsjoe_the_user32 comments

It seems concerning that he would be in charge of determining AI policy when he was a high-level OpenAI employee for 4 years and presumably has a huge financial interest in their success (https://www.levels.fyi/blog/openai-compensation.html).

Maybe; hopefully he can sell whatever stake he has in OpenAI to avoid conflict-of-interest concerns. Personally I can hardly think of any better person for the role.

That would be great, but I don't think federal appointees are ever forced or even asked to divest anything. At most he'd just have to disclose the conflict of interest.

I can think of an entire body of people who should be excluded from such a role; people who spent multiple years working at OpenAI.

Yeah being appointed to such a body should come with a mandatory “you must completely divest from the industry you will be regulating, include and aggregate fund or similar”, alongside “any compensation, hiring, or rehiring contracts and agreements offered prior to or during tenure are explicitly invalid”, and a “you cannot work on the board or executive of any business you I. The relevant industries for five years post departure, or at or with any previous employers for 10 years”

A policy as likely to stick in our marvelous environment of endemic corruption as banning trading by members of the legislative branch and their families.

"Rules for thee, not for me."

Wait until you hear where most finance regulators come from

> huge financial interest in their success

I never understood this. There is money to be made whichever direction the stock moves.

OpenAI doesn't give stock, they give "Profit Participation Units" that are directly tied to how much profit the company makes. If they aren't making a profit and you can't find a buyer who thinks they will in the future, then they are worthless.

How is this compatible with being a non-profit?

> SAMUEL ALTMAN, an individual, GREGORY BROCKMAN, an individual, OPENAI, INC., a corporation, OPENAI, L.P., a limited partnership, OPENAI, L.L.C., a limited liability company, OPENAI GP, L.L.C., a limited liability company, OPENAI OPCO, LLC, a limited liability company, OPENAI GLOBAL, LLC, a limited liability company, OAI CORPORATION, LLC, a limited liability company, OPENAI HOLDINGS, LLC, a limited liability company, and DOES 1 through 100, inclusive

Kinda seems like it's not

https://www.courthousenews.com/wp-content/uploads/2024/02/mu...

The non profit is one of the profit participants of the for profit entity.

They abandoned the non-profit model 5 years ago.

This article says only "a crucial, though non-political, position". "Determining AI policy" is not in the reporting, such as it is.

(It's also extremely unclear who and how many staffers are 'revolting' against this prospective appointment.)

Yes sorry I should have linked to this: https://www.nist.gov/artificial-intelligence/artificial-inte...

> develop science-based and empirically backed guidelines and standards for AI measurement and policy, laying the foundation for AI safety across the world.

This article would be better if it reported on the topic of the article, rather than stopping at stating the topic. Seriously, the whole thing could easily fit in 240 characters.

NIST staffers are revolting against his appointment... why, exactly? Ok, it also says he's associated with EA, and EA people have done bad things. Is that what the NIST staffers are saying? What are the staffers saying, anyway? If you're going to claim to be reporting on them, maybe at least try talking to them.

I got pretty much nothing from this article other than the headline. It both piqued and frustrated my curiousity. Perhaps I need to read more about how EA is being problematic in practice. The high-level description seems fine, but then again, every single religion, cult, movement, rebellion, revolution, etc. could be described as "do good stuff" at a high level (or at least "don't do bad stuff"—Don't Be Evil, anyone?) The part between "do good stuff" and "therefore we must commit genocide" is kinda important.

I had not heard about that movement, but here is their own introduction: https://www.effectivealtruism.org/articles/introduction-to-e...

At first glance, terms like "long-term" and "altruism" resonate with me, as I care much more whether we can avoid the heat death in 500 million years than about what happens tomorrow.

Also the main "doctrine" seems to be one I can agree with, namely to find "unusually effective ways" of helping so that with the same ressources, more positive impact can be achieved. (I did the same when I had to decide which charity to support - since I am not a millionaire, but experienced at research, I focused on identifying highly effective charities to avoid wasting my humble charitable contributions.)

However, where things get problematic is when utilitarian ethics get adopted, a paradigm considered problematic in philosophy because it tries to quantify good in order to measure how to maximize it, and the most good to the most people can often result in outcomes that people of good will agree are unethical, namely the suppression or extermination of minorities. For example, the link above compares pandemic deaths with terrorism deaths and concludes that pandemic deaths are the big ticket numbers and in comparison terrorism deaths are small.

While the numbers reported are possibly statistically correct, I would personally object if one drew the conclusion from them that a small number of deaths does not matter, or that they mattered less than larger number of casualties. Utilitarian ethics, which is often appealing to engineering minds who are used to think in terms of optimizing a variable, has issues with protecting minorities and with justice [1].

From a recent article in the WSJ, it appears that NIST have larger problems than a potentially strange leader: the building substance of the Gaithersburg campus is rotten to the core, to the extent that equipment got destroyed from dampness or water damage etc. A complete overhaul seem due, yet plans exist to cut -- rather than expand -- its funding.

[1] https://www.scu.edu/ethics/ethics-resources/ethical-decision...

> as I care much more whether we can avoid the heat death in 500 million years than about what happens tomorrow.

Isn't this just an egotistical mechanism that allows the bearer to freely justify any means they please by the supposed loftiness of their stated ends?

It has to be, considering anyone who took such a nonsense philosophy remotely seriously would not be so many orders of magnitude wrong about when heat death is expected to occur. 500My isn't even in the right order for solar expansion!

Yes. In Christianity it's pejoratively phrased as "Trust in God and do as you please".

>I focused on identifying highly effective charities to avoid wasting my humble charitable contributions

(1) If you rank charities by effectiveness, how do you avoid making moral comparisons?

(2) If you are experienced at research into charity efficacy, how is it possible you have not ever encountered effective altruism before?

Related to the EA cult: AI doomsayers funded by billionaires ramp up lobbying

https://www.politico.com/news/2024/02/23/ai-safety-washingto...

Their estimations of how likely we are to have a Dyson sphere by 2040 should be sufficient to clarify that this movement has no interest in rigor.

Reply on news.ycombinator.com