The philosophical underpinnings of AI Safety and Effective Altruism absolutely have merit and warrant consideration. Of course as with most things human group/tribal tendencies can tend to overwhelm such underpinnings rather quickly, and it should be no surprise when it turns out individuals who broadcasted belonging to x or y group turned out to be less than trustworthy (or merely incorrect/misguided) along various dimensions.
We can critique the specific actions or beliefs of such actors and question broad labels worn as badges without casting dispersions on a wide swath of potential tenets that may fit into such a label as "complete BS".
Again one can look at the underlying tenets of such a creed and evaluate them for their conceptual merit without blasting the entire enterprise because some people abuse them for personal gain.
If you don't make enough to sustain yourself and others to work on a cause more useful than a cause you could work yourself on then it doesn't make sense. Basically taking a standard 95k non-fang SWE job isn't better than taking a 50K SWE job at say Red Cross.
Also if you can get some 10M job at OpenAI but you also are certain that you could use those same skills to independent develop some drug that will say ~12M in lives/yr then you shouldn't go to OpenAI.
But there's also no true scotsman.
I prefer the doctors approach: "first, do no harm".
"The end justifies the means" is a useful concept that has been talked about many times. I would not dismiss it. I do not think it is reducing EA. I think it is communicating it with clarity, without confusing additions. It is its essence.
Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is). That is literally "the end justifies the means", where the end is "doing good in the world according to model x".
More often than not, that is just an excuse to do something despicable under the excuse of the future good.
> Evidence-based data to determine how they can do the most good
This is a model. If the model is wrong you might cause more harm than good. So, if EA is honest to work this model needs some proof that it works. Otherwise EA is only successful in becoming popular as an idea, not in accomplishing its stated objectives.
Does earning a lot of money and donating it to charities create more good? I am not sure. Maybe earning less money and care for people in your everyday life would create more good.
This is indeed a central question to EA, one various proponents attempt to answer in various ways.
> Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is).
Your assumption here appears to be that anyone subscribing to an evidence-based approach to do the most good (ie EA) must also inherently subscribe to the "ends justify the means." These aren't inextricably linked, it's quite possible to have one and not the other. One can quite rationally seek to maximize their altruistic effectiveness without sacrificing general decency in their day to day life. Morals usually have some nuance.
> This is a model. If the model is wrong you might cause more harm than good.
I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
> I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.
If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table? IMHO not much more than thinking for an afternoon about ends and means.
This is just one particular angle some folks take. It's not by any means the only, or even primary, path EA advocates for. It is of course the one you'll find high earners parading around as an ostensible reason for their wealth creation given it allows self-interest to appear altruistic (which is not wholly unreasonable, those are not opposites and have some overlap, but it also is likely touted far more than exercised).
> If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table?
Its biggest proponents are not scammers, there are just scammers who happened to have claimed to be part of EA. The biggest proponents are probably William Macaskill, Toby Ord, and Peter Singer, none of whom I've ever heard called anything close to a scammer (given how they live their lives that'd be a hard claim to make).
What it brings to the table is framing a more methodical and deliberate way to help others. People generally gravitate towards bleeding-heart altruism when they feel called to (disaster response, someone you know, etc), and various charities and causes are wildly different in the efficiency with which they use the resources given to them. This is to say nothing of careers one might choose for a multitude of reasons. This is all well and fine, but the point is one can also approach these things using informed objectivity to maximize contributions (by giving to charities where dollars go the furthest, to causes that are neglected, etc) rather than relying on subjective compulsion (which is going to be scattershot in efficiency).
Of course EA doesn't have a patent on this, it's just a label, but its one that helped popularize a mode of giving that would've largely remained hidden without it. The problem, as I've mentioned, is group mechanics end up co-opting the label which distorts the intent it originally carried. I truly mean no offense, but it seems from your comments you're just aware of the scandals of SBF or similar, not of the tenets of the actual founders of EA or its various missions outside of the aforementioned "earn to give." Perhaps peruse their site a bit: https://www.effectivealtruism.org/
Firstly, it's more pragmatic because it acknowledges our personal biases. Second, it's an individual practice, unlike utilitarianism, which is typically associated with broader societal or governmental actions.
I really appreciate this approach to altruism. It doesn’t force a moral standard on others. I like practicing it on a small scale, recognizing my biases. Just the act of thinking about how to best help my community is better than giving randomly. This approach also helps to see the world as a place with more love than what we often see in the news.
However, I have concerns about the more widespread version of EA, which seems to be about gaining as much money and power as possible while appearing altruistic. This, in my view, is more akin to a personality disorder, like narcissism, than to utilitarianism.
I've been using the term EA to describe this, but it might not align with what others think EA is. For me, it's a personal habit I try to maintain, but I don't let it define who I am.
Now, we have Sam Bankman-Fried's version, which is at least intellectually rigorous: if you can make billions (through scams) then give those billions (eventually) to charity, are you not morally obligated to do so? At which point I am somewhere between laughing and crying.
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
I only listed the machine learning researchers who are famous or have significant achievements in AI research or engineering in my estimation (not all of them, full list of signatories at https://www.safe.ai/statement-on-ai-risk), because the comment I was responding to claimed that anyone who knew this subject was not concerned about the safety of AI systems and this list repudiates that as laughably inaccurate. There are many other people who signed onto that letter, but the commenter I responded to presumably doesn't care about their opinions because they're not experts specifically on building these systems or researching the science behind them.
"Effective Altruism" is a dumb social club that at best makes people who would do good things anyway think about that in really unhealthy ways (e.g. thinking about how many lives you have or haven't saved every single day), and at worst provides ideological cover to some truly odious views (e.g. the persistent presence of eugenicists who don't get repudiated by other EA people). I don't like them, and I think all the "value cone" crap is bloviating, worse than useless for informing our actual decisions and anti-convincing to people who aren't bought into their STEM Sierra Club shtick.
They don't have a monopoly on being concerned about the safety of artificially intelligent systems, or on being concerned that the development of such systems has a non-zero chance of causing the extinction of humanity. We should take that possibility seriously and make sure it's not going to happen or we at least have a good idea of the exact risks before we build one or engage in research projects or engineering that could accidentally create an artifact which lead to building one.
GPTs have no chance of becoming an extinction-threat level AI. Intelligence is more than just spitting out words when an answer is demanded (and the folks who are saying GPTs are as "intelligent " as some people make me really worry what those folk must think the role of other people is in their lives). Putting a fancy chatter bot in an open loop isn't going to suddenly create a goal-seeking paperclip optimizer.
But there is real harm already being done to people by corporations and governments using AI tools. Yet somehow that doesn't ever come up in the "AI Safety" discussion. Facial recognition getting the wrong person sent to prison. Bad medical advice. Insurance policy denials.
Where are the calls from "concerned AI researchers" for slowing down biometric data collection? And if they are so concerned, why are they so busy pouring gasoline on the fire?
From https://en.wikipedia.org/wiki/AI_safety
Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents.
Also watch Munk debate: AI research and development poses an existential threat featuring Yoshua Bengio, Max Tegmark, Melanie Mitchell and Yann LeCun - https://www.youtube.com/watch?v=144uOfr4SYA
This is a demonstrably false claim that gets repeated over and over. 2 out of 3 Turing Award recipients for work in AI are people who talk about AI safety, and even existential risk. There are many more. It may actually be BS but it’s simply false to say that it’s based on technical ignorance. The question I have is why is this claim made over and over when it is so easily seen to be untrue?
AI Safety is a bit more nuanced. I think there’s a lot of issues that are nonsense like having an “aligned AGI.” Aligned with whom? I could ostensibly envision you aligning it with a single person but aligning it with “humanity” is infeasible because humans are unaligned with each other. However there is an actual technical problem worth solving in alignment, which is basically matching the output of the AI with what the human intends not what the human asks.
Moreover, AI has been improperly used already in ways that are plausibly avoidable. A classic example is algorithmically approving bank loans leading to members of ethnic groups getting denied at a higher rate due to their zip code, which is likely irrelevant to the loan. The designers of the algorithm weren’t explicitly trying to be racist (that would have massive legal ramifications on top of just being plainly unethical, but maybe they did it intentionally and I forgot), they screwed up and didn’t realize they produced an algorithm unaligned with their intentions. Figuring out how to avoid scenarios like that through either methodology or algorithmically is an interesting and much more tractable research problem than making an “AI that benefits all of humanity.”
We need more balanced opinions in a world where the only ideas that survive are the ones that promise short-term gains at the cost of long-term disasters.
Even ML conferences have AI safety tracks these days. Nobody sane wants to be excluded from opportunities.
No Twitter mob is going to stop me. They can descend all they want and pretend to be brave in front of a keyboard and screen. As far as I'm concerned, they're cowards.
You speak against AI and against AI safety? In what way? Wouldn't be speaking against AI enough? Or would AI safety mean laws to prevent AI existence? Like in the Algebraist? (though it is funny in that book how the powerful use AI when it suits them).
(1) A false amelioration that cannot prevent the dangers of AI
(2) A false promise to people that AI can be made safe
(3) A weak on-the-fence position to take of people who think they can have their cake and eat it too.
Do you argue that academics working on those questions are doing illegitimate science?
BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term. Therefore, overall, I vehemently disagree with their research and I think it is fundamentally a contribution to the worsening of the earth.
I absolutely AGREE with treating all people equally and eliminating racial biases of course. However, TECH companies have a VESTED interest in this sort of thing, which is to integrate as many people as possible into the tech empire. It just so happens that building the tech empire is roughly equivalent to success in modern society, which is why their intentions seem good, but they are only ostensibly good. In reality, they do no good of any kind.
The only scenario that comes to my mind - happily untainted with any philosophy training - is the one from Dune. The problem with that is that, throughout the series, Dune describes at least four different dystopic world orders, all motivated and cultivated because the AI-based one didn't come to pass (narrowly). In the novel, the only feasible, long-term strategy to avoid being wiped out by the "thinking machines" was either religious fundamentalism or a God-like undying tyrant forcing its vision on everyone. Since we lack the capabilities to create the latter, it seems the former is the only way we can implement it right now. Wouldn't, however, living under the permanent anti-AI jihad also be a dystopic experience? Especially in our case, where the AI has yet to decimate humanity to the brink of extinction.
I'm sure you considered this, so if you have a summary of what you think we can still do to stop AI without succumbing to any of the other undesirable outcomes, I'd appreciate a link :)
That said:
I'm fairly aligned with your position in that I believe that technology should be utterly subservient to us and I draw the line at push messages and don't have a smartphone, online assistant or any other crap like that. I'll think for myself and I will allow myself to be interrupted where and when I want it, not when some piece of needy technology believes it is time for me to be distracted. My attention is my most precious commodity and I'll spread it around based on my own preferences. That is also why I've banned advertising from my life.
But I recognize that this is a personal decision first and foremost and that there are many other people for whom convenience is a far bigger factor than it is for me. At the same time those people are often wondering why I'm as productive as I am but that's not for me to explain, I lack the tools to tell them - politely - that if they dropped all of the crap in their lives, opened a book or two and learned some more skills that they too could be more productive.
I've been accused of being a modern Luddite and that's kind of funny because for forty years or so I think I was solidly positioned at the forefront of the tech scene, right at the bleeding edge and in quite a few cases working to advance that frontier. But now I'm much more aware of the shadow side of all this stuff and that's why I'm very cautious about what parts I allow in and which parts I keep out, even if that means less smooth interaction with those around me who all embrace all this stuff without a second thought.
Oh, and maybe lay off the all-caps words.
In case anyone is wondering, jacquesm is the one who designed & built a robotic sorting mechanism that intelligently distinguishes different types of Lego pieces and separates them quite effectively.
The kind of Luddite who hasn't forgotten how much progress can be made when zero hours are spent hunched in the touchscreen pose, and things like that.
Anyone who can do 0 hours per day for a decent spell just might find out those zeros really add up.
When you have a device that's supposed to give you more freedom to roam it might not really be pulling its weight if you are tethered to it all the time.
And things like that.
Carl Sagan.
https://en.m.wikipedia.org/wiki/Michael_Crichton
Crichton says these failures of humanity are inevitable because there will always be a technologist who does things because they can, not because they should.
That everything will go off the rails, no matter what people attempt, is the plot of the novel Jurassic Park.
I'm seriously tired of these vested interests destroying perfectly good technology just because it would endanger their precious business models. Computers should have put an end to silly things like copyright but they would rather destroy our computing freedom than allow that to happen. Now we have these awesome language models and AIs and they're desperately trying to limit and control it.
It's a tragedy in of itself that we even need to fight these political battles at all.
there's no separating technology from context, as in consequences. yes, all the talk about bombing datacenters is crazy, but oversight is not.
The truth is these technologies are subversive. They literally subvert the existing social order and impose a new one. A new social order where the users of the technology are empowered. They can't tolerate it when we are the users. It takes their power away. So they ban it, control it, flip it around on us and turn it into the very mechanism of our wide-scale oppression.
there are clear benefits to having the FDA, SEC, FTC, EPA and so on around. there's a clear benefit to having fire inspectors, and there's a clear benefit to the mandate of 2 stairs per multi-story building. there's a clear benefit to the requirement of demonstration of efficacy in front of the FDA. there's a clear benefit to having clean air.
of course there are clear drawbacks too. most of Europe is perfectly okay with just 1 stair per such building. FDA processes are slow and expensive, even if a drug has proven safe and effective abroad.
regulatory capture is a problem, the super-long intellectual property terms are a problem. using environmental review laws for endless NIMBYism is a problem.
but these are arguments for reform, not for scorched earth, because then the problems will very likely come back.
of course it's very hard to do proper cost-benefit comparisons, yet that doesn't mean it's impossible, and especially doesn't mean that simply getting rid of these controls will deliver utopia overnight.
...
funny, how we're talking about some hypothetical oversight, while right now in the present the big beneficiaries of AI are hyper-big-fucking-corps. MSFT. Google. Tesla. Apple. Amazon.
...
Subversive tech? Like Pegasus? And face recognition? And the Great Firewall of China?
We're never going to "reform" this system. We're up against trillion dollar industries with unlimited lobbying power, governments bold enough to ignore the constitutional rights of their own citizens.
social movements, radicals come and go, trillion dollar industries can too.
Researchers have done so. One good resource is Melanie Mitchell's book Artificial Intelligence: A Guide for Thinking Humans (i presume a 2nd edition is in works to incorporate LLMs/ChatGPT etc.) Also checkout her videos on youtube.
The problem is that technologically clueless vested interests in the business/media push sound bites to the already distracted public thus distorting everything.
I'm afraid EA is not a veil. Altruism is a deeply disturbing moral philosophy, if you actually think about it's implications and how it differs from it's originator - utilitarianism.
Similarly, AI safety is something that's religiously believed by some quarters, not all of them ignorant. According to The Atlantic's article on the drama that was posted here earlier[1], Sutskever - supposedly the brightest scientific mind in OpenAI believed in things that would not be out of place in the MIRI cult of old.
[1] https://www.theatlantic.com/technology/archive/2023/11/sam-a...
I think it's important to make a distinction between the two.
Elon Musk seems to be really into it.