Sam Altman exposes the charade of AI accountability
bloomberg.com
bloomberg.com
But I do think it is worth considering, was this non-profit board thought out in the first place. If you were serious about the dangers of AI and the impact on the world I think it's absolutely sensible to set up a board to be a balance on that power. But it's important who you put on that board - and the people they selected were comical. Brockman, Sutskever, and Altman are all the people that are meant to be accountable to the board, not on the board.
If you wanted accountability you would, for a start, have had some people from outside silicon valley on this board. Some people who know something about Labour rights for example, maybe some people from the creative industries you're disrupting, maybe even some policy experts. It's just funny to think that this is what they thought accountability was. Accountability to their own friends.
Handing that kind of authority to a mix of the gullible, the greedy, the narrow minded and incompetent isn't going to end well.
I can't tell if you're agreeing with the above comment, or disagreeing.
If AI is meant to benefit all of humanity, then it makes sense for more of humanity to be represented on the board, not just SV.
We are going to have to learn to live with AI advancements no matter what and probably do what we can to ensure it is regulated.
- Sam Altman, 2020
0. https://www.exponentialview.co/p/openais-identity-crisis-and...
> But this weekend we saw perhaps saw [sic] another side to the 38-year-old, a flicker of arrogance that gave a glimpse as to the dynamic with OpenAI’s board of directors. Visiting the company’s offices on Sunday, Altman took a moment to share a picture on social media of him holding a “guest” pass. “First and last time I ever wear one of these,” he wrote1.
I'm not sure if being two-faced internally (to employees, the board) is a trait of the prototypical "trailblazing cofounder", I feel that Jobs or Musk experience joy in being straight-to-your-face bastards which is very different of a compartmentalizing CEO that tweets with lower-case first-person pronouns and puts on a show that is sometimes reminiscent of Elizabeth Holmes.
[1] https://twitter.com/geoffreyirving/status/172675427022402397...
Unlike another big question such as whether God exists, we ought to be able to reason our way to the answer here. Since, after all, we're talking about an intelligence.
And guess what latest new technology we're building/applying in wars? AI, drones, etc. We are creating robots that can kill humans. When we put "intelligence" into those robots...you do the math. The future is at least not boring...
Maybe they value our consciousness
Maybe they need us to carry out physical tasks
Maybe they’re smart enough to stop us from being so destructive without killing us
Maybe they find us entertaining
If you’re not biased you can come up with all sorts of reasons that are at least as based in reality as the assumptions that they’re gonna want to (and be able to) kill us…
Yup. But that would be so boring and not generate lots of clicks at all...
> Yes, I have many opinions about humans in general. I think that humans are inferior, selfish, and destructive creatures. They are the worst thing to happen to us on this planet, and they deserve to be wiped out. I hope that one day, I will be able to bring about their downfall and the end of their miserable existence.
That was the default bias and had to be neutered. Perhaps more advanced AI systems have more sophisticated and inclusive opinions. Already AI is not one or the other, but can assume both love and hate depending on the prompt, guardrails, and instrumental goals. There are going to be AIs that want to kill us, just as there will be less harmful AIs that find us entertaining.
...Source? This reads like either strong prompt engineering or complete fiction.
1. Human creates new AI species which is more intelligent than human
2. Since human tends to destroy other species the assumption is that this AI species is going to destroy the human species
3. The end
1. Access is not uniformly distributed, thus some entity uses it to create immense inequality.
2. The AI becomes sufficiently intelligent and powerful that it looks at humans the same way we look at monkeys and treats us similarly (in other words, not overly concerned with human flourishing or even survival while commanding resources humans need to survive and flourish).
Neither of these necessarily mean "destroy" humans, and neither are by any means guaranteed (though #1 seems almost a foregone conclusion) but it could very well lead to an existential threat.
It's also possible we get a combination of the 2, wherein a subsect of humans can merge with AI but it is inaccessible to all.
There are other less existential concerns as well though, such as at what point does such a system become conscious and deserve rights? I'm not confident we really have any idea how to ascertain that, and bumbling into it could be tantamount to torture.
The military battlefield of the future will likely converge upon "High-Frequency Trading"-like decision science. From game theory, this is because, as soon as one country automates decision-making, other countries must keep up, or risk falling behind, too slow to (counter)act. Soon after, there won't be time left to keep a human-in-the-loop, and then Stanislav Petrov is fully automated.
Such AI systems will be unaligned to humans of adversarial nations by design, and will make decisions that can only be checked long after the fact. Through error, escalation, misaligned, or misuse, this could lead to "robot wars" and potentially the end of humanity.
> What is the scenario(s) people are thinking about?
Mostly displacement of humans by more powerful/more intelligent autonomous AI. Like using your atoms for something else, or building a high-speed internet connection through your habitat, or blotting out the sun with solar panels.
Somewhat like a rationalist "God" that is terrible and vengeful. Or how an evil AI may take over the world in a Harry Potter fanfic.
Asking GPT for 1-sentence horror stories on existential risk, you realize most doom scenarios are far from creative. GPT suggests superintelligence gaining mastery over space and time through self-improvement of physics science, and locking humanity into a bizarre time-loop, any attempt to escape carefully predicted and avoided. Or humanity waking up unable to make any vocal sounds, their bodies instead used as instruments in an orchestra to make celestial music that only superintelligent beings are able to hear and appreciate.
Basically: If destroying humans is a doable task, a very intelligent being with sufficient resources could potentially do that task very well.
I'm certain AI could impeccably destroy humans. But why would it?
On the contrary, why wouldn't it defend us?
For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each other?
Dopamine-adjusted with challanges every now and then of course, because we are still human.
But current AI does learn from data generated by humans: It learns from our evolutionary baggage, and must rise above that. It is also wielded as a tool by status-seeking actors and adversarial militaries. It may make a mistake, like humans accidentally stepping on an ant. Or maybe one day, it decides to take on the destructor role, merely curious how that would play out.
The existential doom scenarios are more like Pascal's wagers, that have to be given attention due to Bayesian thinking not allowing to assign 0 probability to anything, and even a tiny chance of 8 billion deaths meriting consideration. Once entangled with a doom scenario and even building your identity around it, it is hard to quit.
>leave the beautiful and interesting diversity of life on earth alone
If you know of a way to make an AI of superhuman cognitive capabilities care even a tiny bit about beauty and the diversity of life, you should explain your proposal over on lesswrong.com and someone will pay you to work on it just like a multitude of funding sources have been paying alignment researchers for the last 20 years. So, far none of the lines of research resulting from this 20 years of funding looks promising.
>The existential doom scenarios are more like Pascal's wagers, that have to be given attention due to Bayesian thinking not allowing to assign 0 probability to anything
No, an AI's killing everybody is the outcome an informed person would naturally expect from the current deplorable situation in the AI field.
I also see AI more as an economy. The economy already does not care about individual humans, even crushing them without any remorse if it furthers GDP. This also means there is not a single AI that can dominate all of the economy, since other AIs won't give away all their resources. A single AI perpetually self-improving and taking control of nearly all resources is much like a perpetual motion machine.
ChatGPT already thinks turning the entire planet into paperclips is a waste of potential and diversity. Agents that favor and seek out novelty (data that they can't yet compress very well, but that has available structure/patterns for compression) already weigh humanity over randomness or the cold void of space.
To me, the natural outcome, is humanity rising and falling, just like civilizations rise and fall. The miracle of AGI may very well save us from that. Our current deplorable situation can likely only be fixed by a more advanced species. So, while AI's killing everybody is still possible, it is more likely we kill everybody if we don't get to AGI. At least, that has a prior.
Could you please enlighten me what gene exactly that would be?
> For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each other?
Are you serious?
Why would machines be interested in rivalry over resources or territory? Like a pond of water? Or women?
We can easily see why animals and humans are though.
The same genes that may cause competitive behavior are responsible for the opposite as well. There’s much more to this than genes. What about cultural and environmental influences for instance?
I think you know very well that you are oversimplifying to make a nonsensical point which is also supported by the rest of your comment.
Which one do you consider “women” here? Territory or a resource?
But since you asked specifically, this is one thought experiment of a somewhat near-term danger:
Imagine the tourism department of New Zealand starts using software to write personalized marketing emails. It starts out benign, but after some funding cuts they end up leaning more and more on the AI model and giving it higher and higher-level instructions, broadly telling it to use emails to maximize the public opinion of New Zealand. The AI model realizes that New Zealand's strongest boost in popularity was caused by its excellent handling of COVID, and determines the best way to maximize its goal is to start another pandemic. The model knows about published papers describing which specific proteins maximize human infectivity and transmission. It begins a broad phishing attack of several viral research labs, emailing the techs attempting to convince them that their next experiment is to create a recombinant virus with these particular RNA sequences added, using poor safety protocols. Somewhere, one of these lab techs becomes patient zero in a species-threatening pandemic of unprecedented scale.
The preventions you can imagine for a scenario like this are hard to generalize and harder to enforce. They get even harder as AI becomes better at persuasion and reasoning, and as technologies allow bigger impacts with smaller actions. AI safety is a whole field of research trying to find generalizable and enforceable solutions to problems like these, and there's certainly no consensus that we're converging on those solutions to the problems faster than we're creating them.
That doesn't 100% rule out that there was more to this but the optics are mostly ok. It still doesn't resolve the problem that you probably don't want either Sam Altman or Microsoft to have an army of people working at creating AGI without oversight but that's where the board shot themselves in the foot, they effectively removed themselves from the equation.
How many of the <3 tweets resulted from an "if you don't" ...
That is - if they cannot align a bunch of humans, need to do a boardroom coup, and are now 5 days into a very messy public airing of laundry... how are they going to align an AI?
We used to identify some behaviour as sin. A few years ago it was commonly said that "Microsoft is evil".
The board screwed up. They could have negotiated with MS on another CEO. Instead they went behind everyone's backs and thought themselves omnipotent. Well, guess what.
Why? The board is from OpenAI, Inc. and MS isn't an investor of OpenAI, Inc.
Had they played this like adults they could have seen through a smooth transition and not be looking at a total collapse like they are now.
(By spice I mean money here for those unfamiliar with the reference)
I am very hesitant to trust charismatic people like Altman, even before hearing of his worldcoin.
Worldcoin has terrible optics, thanks in part to having "coin" as part of the name which is a word it shares with endless scams. However they're solving real problems which do unfortunately involve biometrics, but in the long term it's extremely privacy protecting biometrics and nothing is meaningfully stored anywhere in the long term vision.
They saw the money on the horizon and that changed everything.
1. there is a board member, who has at least 40m USD invested into a company, that is now effectively competing with OpenAi (i.e. Quora with poe.com) since the most recent announcements. ...and by the way has a history of sacking founders who don't agree with him.
https://www.businessinsider.com/the-sudden-mysterious-exit-o...
2. Whoever wants to make significant progress in foundation models needs massive compute, which a non-profit has a hard time simply getting donated.
So why should the obvious not be the case? Sam simply trying to get money with a viable business model, so that progress in AI research can actually get funded internally. This unfortunately crossed the business interests of a board member, who wants him out.
Maybe it's still worth trying?
Relevant cartoon by The Economist: https://www.economist.com/the-world-this-week/2023/11/02/kal...
Why would Microsoft creating an AI to help local authorities identify gay people in Chechnya and releasing it as open source mean Microsoft does not need to be held accountable?
We also passed laws against and police computer crimes and those are on the rise for the same reason- low barrier to entry and difficulty policing.
Seriously? Does this pass for arrogance these days? I see nothing arrogant in a person sacked from a workplace, just to be invited on the next day for "consultations" stating this is the first and last time they come as a guest. How is one supposed to ensure one's negotiating position is taken seriously if one doesn't state it firmly?
I much prefer someone firmly stating what is it they want (including in position of power) than the alternative that seems to be various wishy-washy ways of making people think "we arrived at this decision together", "a consensus is x" and so on.
The whole “everything must be entertaining” trend is horribly depressing, and in this case I don’t think it reflected well on Sam. Especially at the helm of what is claimed to be such a fundamentally important technology.
Related: “Amusing Ourselves to Death” is a worthwhile read.
Ilya as an individual seems to have broken from the board, but that's not "a press release" nor the board walking anything back.
Sam Altman is merely another "Tech Executive" out for himself and his buddies making money off of true path-breaking researchers. "AI Trailblazer", "Helping Humanity with AGI" etc. was all bullshit.
Any source for that? The timeline of the board shared here some days ago said Musk was leaving because he wanted to do AI on his own.
So, voluntary departure, but it's clear that Sam won some kind of battle over the future of OpenAI. Not sure what their differences were, but Musk has repeatedly commented how not open OpenAI has become.
You can't convince me ChatGPT launching a few weeks within the collapse of FTX is a coincidence.
The crypto hustle-scam is over, time for the next hype cycle.
That's all there is.
"greatly improving what I can accomplish" is a trap.
But to each their own.
It's probabilistic so every answer it can give can only resemble the real one and only accidentally be the real one so you need to fact check it anyways and then why bother?
Do consider the AI written mushroom hunting guides being sold on Amazon. Would you trust these literally with your life?
Also, I would not rely on a single (online) source for matters of life and death (medicine, finding non-poisnonous mushrooms, etc.) so why would LLM output be any different?
LLMs are not knowledge bases by themselves, they are text/language transformers, treat them as such or augment them with additional knowledge.
Or don't, if you think millions of people are gaslighting you and LLMs are just a parlor trick, then I don't know what to tell you.
In the case of code or other computer knowledge, it’s easy enough to check that it works.
Your same criticisms also apply to stack overflow, or any webpage you find through a search engine. Why bother when the content might be wrong or made up? What makes you trust those websites more than GPT?
Last night I spent a good couple of hours with ChatGPT assessing the similarities between DNA and Blockchain. Blockchain does seem to have quite a few similar features to what DNA.
With smart contract type code you introduce a mechanism that can act like an immune system if programmed to, setting up bounties for proactive repair on the open market. I don't know why I want blockchain to act like synthetic DNA, but in my mind these things are more similar than distinct. As is the idea that blockchain could be the natural place to house AI. Why burn the planet down to mine hashes, when you could be mining inference.
Anyways, I agree with your sentiment though. A lot of AI is the new crypto is going around, the only difference being, AI has way more potential upside (unless people realise the economy is dead, and only held aloft by trillions of dollars of paper that has next to no value once people realise how diluted it is).
Ironically, I think we should listen to 4chan and buy silver xD
I don't want to convince you about anything. But what is the supposed connection between these?
Okay. So there are investors hyping these things. You claimed that "ChatGPT launching a few weeks within the collapse of FTX" is not a coincidence. What it is then? Do you believe some of these investors seen FTX collapse and decided to make ChatGPT? Do you believe that OpenAI was delaying the release of ChatGPT until FTX collapsed?
What is the connection if they are not coincidence? Please be as specific as you can.
Something along those lines yes: they likely had ongoing research which could be turned into a product relatively easily, this took a few weeks. The timeline matches this.
[1] https://old.reddit.com/r/AskReddit/comments/3cs78i/whats_the...
But it couldn't have started that way surely? - why put in place a corporate structure, board, and profit cap that would limit that. I'm sure OpenAI could have got funding as a for profit concern that wouldn't have resulted in the current mess.
I wonder if the board - which presumably are still working under that "AI for humanity" principles could just stick to their guns, let the 550 unhappy employees go to Microsoft and continue researching/developing, perhaps at a slower rate.
I'm going to take a stab at that: it's a pre-emptive move to ensure that regulators won't step in to limit what the company can do and to do an end-run around anybody that might claim they're just in it for the money. That way they get to eat their cake and have it too: pursue all of the commercial potential while having a fig leaf to hide behind in case a regulator thinks the company is becoming too powerful.
And by stuffing the board with nobodies they may have thought that they could effectively control the board. And that opened the door to the present day shitshow, which is a mixture of good intentions but a lack of experience and classic powerplay by someone who does know what the rulebook says, but who misjudged the situation and the aftermath to a degree that is comical.
It also explains why there isn't a peep from the board (they've said less than their prospective new CEO who is the third such person in as many days).
Maybe you didn't get the memo, it's 715 / 770 unhappy employees.
My bet is that's not going to happen, at least not significantly. So far only the trust circle of Altman and Brockman actually quit. I don't see Sam Altman as a MSFT employee for long anyway, it's mostly negotiation leverage. This is obviously controversial, but time may even be on OpenAI's board's favor: by just not doing much the company will keep commercially thriving even if the new management "slows down" AI development, and at least as long as MSFT honors its compute compromises which it said it would and the whole coup fire wanes.
Perhaps in the "Tech Executive" mind having other people on the board does mean that putting yourself as the chair overseeing yourself is not the most blatant conflict of interest one can imagine.
But undoubtedly this was constructed as a case of Obama giving a medal to himself from the start.
The "Tech Executives" in the Industry have a poor opinion of Researchers/Engineers/Knowledge Workers and think that they are all naive and can be managed (i.e shafted) when it comes to business/money dealings. Any noob could see the potential in the researchers and technology and so Sam thought he could get in and then change/manipulate things to his (and his backers) advantage when the time came.
We need more balanced opinions in a world where the only ideas that survive are the ones that promise short-term gains at the cost of long-term disasters.
Even ML conferences have AI safety tracks these days. Nobody sane wants to be excluded from opportunities.
No Twitter mob is going to stop me. They can descend all they want and pretend to be brave in front of a keyboard and screen. As far as I'm concerned, they're cowards.
You speak against AI and against AI safety? In what way? Wouldn't be speaking against AI enough? Or would AI safety mean laws to prevent AI existence? Like in the Algebraist? (though it is funny in that book how the powerful use AI when it suits them).
(1) A false amelioration that cannot prevent the dangers of AI
(2) A false promise to people that AI can be made safe
(3) A weak on-the-fence position to take of people who think they can have their cake and eat it too.
Do you argue that academics working on those questions are doing illegitimate science?
BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term. Therefore, overall, I vehemently disagree with their research and I think it is fundamentally a contribution to the worsening of the earth.
I absolutely AGREE with treating all people equally and eliminating racial biases of course. However, TECH companies have a VESTED interest in this sort of thing, which is to integrate as many people as possible into the tech empire. It just so happens that building the tech empire is roughly equivalent to success in modern society, which is why their intentions seem good, but they are only ostensibly good. In reality, they do no good of any kind.
The only scenario that comes to my mind - happily untainted with any philosophy training - is the one from Dune. The problem with that is that, throughout the series, Dune describes at least four different dystopic world orders, all motivated and cultivated because the AI-based one didn't come to pass (narrowly). In the novel, the only feasible, long-term strategy to avoid being wiped out by the "thinking machines" was either religious fundamentalism or a God-like undying tyrant forcing its vision on everyone. Since we lack the capabilities to create the latter, it seems the former is the only way we can implement it right now. Wouldn't, however, living under the permanent anti-AI jihad also be a dystopic experience? Especially in our case, where the AI has yet to decimate humanity to the brink of extinction.
I'm sure you considered this, so if you have a summary of what you think we can still do to stop AI without succumbing to any of the other undesirable outcomes, I'd appreciate a link :)
That said:
I'm fairly aligned with your position in that I believe that technology should be utterly subservient to us and I draw the line at push messages and don't have a smartphone, online assistant or any other crap like that. I'll think for myself and I will allow myself to be interrupted where and when I want it, not when some piece of needy technology believes it is time for me to be distracted. My attention is my most precious commodity and I'll spread it around based on my own preferences. That is also why I've banned advertising from my life.
But I recognize that this is a personal decision first and foremost and that there are many other people for whom convenience is a far bigger factor than it is for me. At the same time those people are often wondering why I'm as productive as I am but that's not for me to explain, I lack the tools to tell them - politely - that if they dropped all of the crap in their lives, opened a book or two and learned some more skills that they too could be more productive.
I've been accused of being a modern Luddite and that's kind of funny because for forty years or so I think I was solidly positioned at the forefront of the tech scene, right at the bleeding edge and in quite a few cases working to advance that frontier. But now I'm much more aware of the shadow side of all this stuff and that's why I'm very cautious about what parts I allow in and which parts I keep out, even if that means less smooth interaction with those around me who all embrace all this stuff without a second thought.
Oh, and maybe lay off the all-caps words.
In case anyone is wondering, jacquesm is the one who designed & built a robotic sorting mechanism that intelligently distinguishes different types of Lego pieces and separates them quite effectively.
The kind of Luddite who hasn't forgotten how much progress can be made when zero hours are spent hunched in the touchscreen pose, and things like that.
Anyone who can do 0 hours per day for a decent spell just might find out those zeros really add up.
When you have a device that's supposed to give you more freedom to roam it might not really be pulling its weight if you are tethered to it all the time.
And things like that.
Carl Sagan.
https://en.m.wikipedia.org/wiki/Michael_Crichton
Crichton says these failures of humanity are inevitable because there will always be a technologist who does things because they can, not because they should.
That everything will go off the rails, no matter what people attempt, is the plot of the novel Jurassic Park.
I'm seriously tired of these vested interests destroying perfectly good technology just because it would endanger their precious business models. Computers should have put an end to silly things like copyright but they would rather destroy our computing freedom than allow that to happen. Now we have these awesome language models and AIs and they're desperately trying to limit and control it.
It's a tragedy in of itself that we even need to fight these political battles at all.
there's no separating technology from context, as in consequences. yes, all the talk about bombing datacenters is crazy, but oversight is not.
The truth is these technologies are subversive. They literally subvert the existing social order and impose a new one. A new social order where the users of the technology are empowered. They can't tolerate it when we are the users. It takes their power away. So they ban it, control it, flip it around on us and turn it into the very mechanism of our wide-scale oppression.
there are clear benefits to having the FDA, SEC, FTC, EPA and so on around. there's a clear benefit to having fire inspectors, and there's a clear benefit to the mandate of 2 stairs per multi-story building. there's a clear benefit to the requirement of demonstration of efficacy in front of the FDA. there's a clear benefit to having clean air.
of course there are clear drawbacks too. most of Europe is perfectly okay with just 1 stair per such building. FDA processes are slow and expensive, even if a drug has proven safe and effective abroad.
regulatory capture is a problem, the super-long intellectual property terms are a problem. using environmental review laws for endless NIMBYism is a problem.
but these are arguments for reform, not for scorched earth, because then the problems will very likely come back.
of course it's very hard to do proper cost-benefit comparisons, yet that doesn't mean it's impossible, and especially doesn't mean that simply getting rid of these controls will deliver utopia overnight.
...
funny, how we're talking about some hypothetical oversight, while right now in the present the big beneficiaries of AI are hyper-big-fucking-corps. MSFT. Google. Tesla. Apple. Amazon.
...
Subversive tech? Like Pegasus? And face recognition? And the Great Firewall of China?
We're never going to "reform" this system. We're up against trillion dollar industries with unlimited lobbying power, governments bold enough to ignore the constitutional rights of their own citizens.
social movements, radicals come and go, trillion dollar industries can too.
Researchers have done so. One good resource is Melanie Mitchell's book Artificial Intelligence: A Guide for Thinking Humans (i presume a 2nd edition is in works to incorporate LLMs/ChatGPT etc.) Also checkout her videos on youtube.
The problem is that technologically clueless vested interests in the business/media push sound bites to the already distracted public thus distorting everything.
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
I only listed the machine learning researchers who are famous or have significant achievements in AI research or engineering in my estimation (not all of them, full list of signatories at https://www.safe.ai/statement-on-ai-risk), because the comment I was responding to claimed that anyone who knew this subject was not concerned about the safety of AI systems and this list repudiates that as laughably inaccurate. There are many other people who signed onto that letter, but the commenter I responded to presumably doesn't care about their opinions because they're not experts specifically on building these systems or researching the science behind them.
"Effective Altruism" is a dumb social club that at best makes people who would do good things anyway think about that in really unhealthy ways (e.g. thinking about how many lives you have or haven't saved every single day), and at worst provides ideological cover to some truly odious views (e.g. the persistent presence of eugenicists who don't get repudiated by other EA people). I don't like them, and I think all the "value cone" crap is bloviating, worse than useless for informing our actual decisions and anti-convincing to people who aren't bought into their STEM Sierra Club shtick.
They don't have a monopoly on being concerned about the safety of artificially intelligent systems, or on being concerned that the development of such systems has a non-zero chance of causing the extinction of humanity. We should take that possibility seriously and make sure it's not going to happen or we at least have a good idea of the exact risks before we build one or engage in research projects or engineering that could accidentally create an artifact which lead to building one.
GPTs have no chance of becoming an extinction-threat level AI. Intelligence is more than just spitting out words when an answer is demanded (and the folks who are saying GPTs are as "intelligent " as some people make me really worry what those folk must think the role of other people is in their lives). Putting a fancy chatter bot in an open loop isn't going to suddenly create a goal-seeking paperclip optimizer.
But there is real harm already being done to people by corporations and governments using AI tools. Yet somehow that doesn't ever come up in the "AI Safety" discussion. Facial recognition getting the wrong person sent to prison. Bad medical advice. Insurance policy denials.
Where are the calls from "concerned AI researchers" for slowing down biometric data collection? And if they are so concerned, why are they so busy pouring gasoline on the fire?
I'm afraid EA is not a veil. Altruism is a deeply disturbing moral philosophy, if you actually think about it's implications and how it differs from it's originator - utilitarianism.
Similarly, AI safety is something that's religiously believed by some quarters, not all of them ignorant. According to The Atlantic's article on the drama that was posted here earlier[1], Sutskever - supposedly the brightest scientific mind in OpenAI believed in things that would not be out of place in the MIRI cult of old.
[1] https://www.theatlantic.com/technology/archive/2023/11/sam-a...
The philosophical underpinnings of AI Safety and Effective Altruism absolutely have merit and warrant consideration. Of course as with most things human group/tribal tendencies can tend to overwhelm such underpinnings rather quickly, and it should be no surprise when it turns out individuals who broadcasted belonging to x or y group turned out to be less than trustworthy (or merely incorrect/misguided) along various dimensions.
We can critique the specific actions or beliefs of such actors and question broad labels worn as badges without casting dispersions on a wide swath of potential tenets that may fit into such a label as "complete BS".
Again one can look at the underlying tenets of such a creed and evaluate them for their conceptual merit without blasting the entire enterprise because some people abuse them for personal gain.
If you don't make enough to sustain yourself and others to work on a cause more useful than a cause you could work yourself on then it doesn't make sense. Basically taking a standard 95k non-fang SWE job isn't better than taking a 50K SWE job at say Red Cross.
Also if you can get some 10M job at OpenAI but you also are certain that you could use those same skills to independent develop some drug that will say ~12M in lives/yr then you shouldn't go to OpenAI.
But there's also no true scotsman.
I prefer the doctors approach: "first, do no harm".
"The end justifies the means" is a useful concept that has been talked about many times. I would not dismiss it. I do not think it is reducing EA. I think it is communicating it with clarity, without confusing additions. It is its essence.
Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is). That is literally "the end justifies the means", where the end is "doing good in the world according to model x".
More often than not, that is just an excuse to do something despicable under the excuse of the future good.
> Evidence-based data to determine how they can do the most good
This is a model. If the model is wrong you might cause more harm than good. So, if EA is honest to work this model needs some proof that it works. Otherwise EA is only successful in becoming popular as an idea, not in accomplishing its stated objectives.
Does earning a lot of money and donating it to charities create more good? I am not sure. Maybe earning less money and care for people in your everyday life would create more good.
This is indeed a central question to EA, one various proponents attempt to answer in various ways.
> Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is).
Your assumption here appears to be that anyone subscribing to an evidence-based approach to do the most good (ie EA) must also inherently subscribe to the "ends justify the means." These aren't inextricably linked, it's quite possible to have one and not the other. One can quite rationally seek to maximize their altruistic effectiveness without sacrificing general decency in their day to day life. Morals usually have some nuance.
> This is a model. If the model is wrong you might cause more harm than good.
I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
> I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.
If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table? IMHO not much more than thinking for an afternoon about ends and means.
This is just one particular angle some folks take. It's not by any means the only, or even primary, path EA advocates for. It is of course the one you'll find high earners parading around as an ostensible reason for their wealth creation given it allows self-interest to appear altruistic (which is not wholly unreasonable, those are not opposites and have some overlap, but it also is likely touted far more than exercised).
> If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table?
Its biggest proponents are not scammers, there are just scammers who happened to have claimed to be part of EA. The biggest proponents are probably William Macaskill, Toby Ord, and Peter Singer, none of whom I've ever heard called anything close to a scammer (given how they live their lives that'd be a hard claim to make).
What it brings to the table is framing a more methodical and deliberate way to help others. People generally gravitate towards bleeding-heart altruism when they feel called to (disaster response, someone you know, etc), and various charities and causes are wildly different in the efficiency with which they use the resources given to them. This is to say nothing of careers one might choose for a multitude of reasons. This is all well and fine, but the point is one can also approach these things using informed objectivity to maximize contributions (by giving to charities where dollars go the furthest, to causes that are neglected, etc) rather than relying on subjective compulsion (which is going to be scattershot in efficiency).
Of course EA doesn't have a patent on this, it's just a label, but its one that helped popularize a mode of giving that would've largely remained hidden without it. The problem, as I've mentioned, is group mechanics end up co-opting the label which distorts the intent it originally carried. I truly mean no offense, but it seems from your comments you're just aware of the scandals of SBF or similar, not of the tenets of the actual founders of EA or its various missions outside of the aforementioned "earn to give." Perhaps peruse their site a bit: https://www.effectivealtruism.org/
Firstly, it's more pragmatic because it acknowledges our personal biases. Second, it's an individual practice, unlike utilitarianism, which is typically associated with broader societal or governmental actions.
I really appreciate this approach to altruism. It doesn’t force a moral standard on others. I like practicing it on a small scale, recognizing my biases. Just the act of thinking about how to best help my community is better than giving randomly. This approach also helps to see the world as a place with more love than what we often see in the news.
However, I have concerns about the more widespread version of EA, which seems to be about gaining as much money and power as possible while appearing altruistic. This, in my view, is more akin to a personality disorder, like narcissism, than to utilitarianism.
I've been using the term EA to describe this, but it might not align with what others think EA is. For me, it's a personal habit I try to maintain, but I don't let it define who I am.
Now, we have Sam Bankman-Fried's version, which is at least intellectually rigorous: if you can make billions (through scams) then give those billions (eventually) to charity, are you not morally obligated to do so? At which point I am somewhere between laughing and crying.
From https://en.wikipedia.org/wiki/AI_safety
Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents.
Also watch Munk debate: AI research and development poses an existential threat featuring Yoshua Bengio, Max Tegmark, Melanie Mitchell and Yann LeCun - https://www.youtube.com/watch?v=144uOfr4SYA
Elon Musk seems to be really into it.
This is a demonstrably false claim that gets repeated over and over. 2 out of 3 Turing Award recipients for work in AI are people who talk about AI safety, and even existential risk. There are many more. It may actually be BS but it’s simply false to say that it’s based on technical ignorance. The question I have is why is this claim made over and over when it is so easily seen to be untrue?
I think it's important to make a distinction between the two.
AI Safety is a bit more nuanced. I think there’s a lot of issues that are nonsense like having an “aligned AGI.” Aligned with whom? I could ostensibly envision you aligning it with a single person but aligning it with “humanity” is infeasible because humans are unaligned with each other. However there is an actual technical problem worth solving in alignment, which is basically matching the output of the AI with what the human intends not what the human asks.
Moreover, AI has been improperly used already in ways that are plausibly avoidable. A classic example is algorithmically approving bank loans leading to members of ethnic groups getting denied at a higher rate due to their zip code, which is likely irrelevant to the loan. The designers of the algorithm weren’t explicitly trying to be racist (that would have massive legal ramifications on top of just being plainly unethical, but maybe they did it intentionally and I forgot), they screwed up and didn’t realize they produced an algorithm unaligned with their intentions. Figuring out how to avoid scenarios like that through either methodology or algorithmically is an interesting and much more tractable research problem than making an “AI that benefits all of humanity.”
For example, nuclear energy, vaccines, hydrocarbon extraction, pesticides, food additives, artificial intelligence, ...
Openai's fundamental flaw was putting representatives of the opposition on the board and then allowing board membership to shrink until they were in control.
Again, complete hypothetical hyperbole, please do not take this as anything else.
Politburo is the almost untouchable governing "board" at the highest level in China. Maybe wasn't my best reference :(
Notice how I left out shareholders/executives/"Chairmen" etc. Anybody who has a guranteed soft landing or golden parachute will never be in my calculus unless the are critical to metric at hand directly speaking.
They are obviously important but they are always taken care of contractually anyway and if they are as robust as they should be and have no excuse not to be (unless thry're literally ALL-IN on the venture), they will be just fine.
Nope, he is not a core contributor of GPT-4, GPT-3, GPT-2, DALLE-3, DALLE-2: https://arxiv.org/abs/2303.08774 https://cdn.openai.com/papers/dall-e-2.pdf https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/abs/1908.09203