Goodbye Google
robert.ocallahan.org
robert.ocallahan.org
That is a threat to the fundamental stability of society.
I think it's worse than that. Or at least, the products that have been built because of advertising are worse than that. Instagram/TikTok/Facebook are all ultimately just surfaces to serve ads, but they've wormed their way into their users' consciousness in fairly insidious ways
All intellectual, psych and social scores are in free fall since introduction of ad-supported attention grabbing internet.
AI is theorethical threat. Advertising is a constant detriment to our brains, societies and culture.
If AI can mitigate a fraction of the damage that ads do it will have massive positive impact.
We live in times where everyone can pick their own flavor of apocalypse to be scared of. Ads are leading us into Idiocracy since ~2004 and made huge progress towards it.
Google and Meta's wealth came at the cost of civilization getting sucked dry of its attention and mental capacity.
Even before big money was on the table, people didn’t blink an eye at starting to work for Google/Facebook, “solving fun tasks.”
I respect the argument for not working for a company that does addictiveness work anywhere, even if you don't work on it yourself. But that's not where I drew the line.
Also FWIW I disagree the danger of AI is abstract. A lot is happening right now, in education for example.
+1000
I actually work in education and can share the following first-hand: the things we now build, the ways we now educate, and all the opportunities for reaching out to students are beautiful. I foresee that _many_ people will cut corners now, but I believe that, in the end, we will get a new generation of education that is personal and humane.
I don't mean this as preaching some shiny future that I hope will magically arrive. I already see these things happening. And I also see all the ways people misuse the tools or take shortcuts.
Educators with good intentions, skills, and powerful tools will always beat ignorance and laziness relying on those same tools.
I wrote about this here: https://news.ycombinator.com/item?id=49685036
Is there anything specific you, as an LLM equivalent of a nuclear physicist (if that’s a fair comparison), can highlight in terms of these imminent dangers?
AI is concerning in some aspects, yes, but the loss of a shared reality has nothing to do with that, and is even more scary.
Or do it as a hobby. I love shooting guns, I certainly wouldn’t shoot people if that was the only way to make a career shooting guns.
> I plan to study how AI agents debug code and whether debugging tools that can make them more effective at that.
> I want to use AI agents to bring some of my hobby project ideas to life.
> I’m keen to reap the benefits of AI
and addresses the anti-AI view:
> AI detractors think that AI won’t really work. I think it will. (ed. for brev)
The main reason then is the concern of the velocity of the rollout and the increased pervasiveness of AI. Too fast, too much.
Personally if I could Thanos-snap AI out of existence I would, but as long as it exists, it seems better on balance to make good use of it where possible than just try to ignore it.
"Focusing more on AI stuff" ... not really? Of course I was using AI a lot in my work at Google, as well as contributing to AI acceleration. Now I won't be contributing to acceleration and I can be as cautious as I want to be about using it.
I also don’t believe addiction and other psychological exploits are used for the sole purpose of ad revenue. There are other more direct forms of monetization involved, such as microtransactions and subscriptions.
If you really want to blame something, it really all comes down to one word:
Greed.
Advertising has become 'active', very recently in the overall lifespan of advertising. For most of it's lifespan it was passive, it blared out at you from the TV or radio or bus stop. Now, however, it's reaching out and taking pieces of you back for analysis and reconstruction, and on-selling to any other entity interested in doing the same.
The reason is greed, yes.
We can't regulate human greed, but we can regulate the manifestation of its symptoms.
We can also choose not to work towards expanding the capacity of said symptoms.
Not if we need to eat and raise our kids. Then we need careers, which depend on what everyone else is doing, so we can't stop, our own future depends on how competitive we remain in the new context. I see an idealist streak here, some kind of naive platonism that ignores everything runs at a cost and nothing executes for free like platonic heaven. We can't take decisions that contradict our own needs.
Continuing to work for evil people is a choice. Don’t brush it off.
Instagram? Tiktok? Facebook?
Social media feeds off addiction to generate ad revenue.
And how much damage has social media done to political and social institutions and to collective mental health?
Ads on other web properties are not weaponised in the same way.
Eating too much fruits has never been healthy, but gorging oneself on raw sugar is worse. Perfect is the enemy of the good.
On the other hand you have pure speculation.
Not only that, AI has been immensely liberating/empowering for many ordinary people (That's why these AI companies have millions of users).
If you really want to blame something, it really all comes down to one word: substance use disorder.
I suppose yeah... It's just greed and corruption.
The incentive structures are wrong/poor/malignant
But they never did quit saying we are harming children, exploiting people with addiction, getting people addicted to social media, earning money from distributing scam/virus thru ads. These people are highest to the highest hypocrites.
I think people with objections to googles business model simply don’t join the company in the first place. We’re decades from Google+ and other various “oh maybe Google isn’t perfect” events, the advertising is a known entity.
There has been a shift, and people are reacting to it
I understand that some people think AI is completely 100% safe, but you guys also have to be aware that a lot of people disagree with you, otherwise you're going to find their actions very confusing.
Sure, there may be some bad actors who utilize it to hack computer systems.
But most of the negative impact will be the subtle influences on the mind which are hard to identify and quantify.
> AI is developing faster than humans can individually and collectively understand it and adapt to it. People trying to plan their futures, e.g. trying to plan for a world several years in the future as they enter university, can no longer do so the way previous generations could.
This isn't because I think that having children gives people any extra insights, but because it's pretty hard to think of other situations in which people have to plan for the happiness of others they care about over a timescale of decades and where there's significantly more ability to change course than for many adults (kids tend not to have short-term financial commitments that lock them into existing career decisions, for example).
Historically I think a lot of people in tech would have recommended it as a good career (whilst at the same time, reputedly, the people who made generational wealth out of tech were arranging their children's lives to minimise exposure to the products that made them rich). Is that still what people are saying?
The conclusions there seem to be roughly:
* Keep calm and carry on, because we don't have any better ideas
* Sometimes it's OK to eat the damn marshmallow (i.e. in the face of future uncertainty, remember to do things that make your kids happy today)
I kind of suspect that the first one in particular is more or less people's default response. But I also wonder about what changes people _have_ made, if any.
Education is a bet against AI.
My two older and more theoretically oriented children I am not sure what to recommend so I let them find something they are interested in mostly on their own.
A lot of my comments get responses that assume I'm an "AI booster" but what I really am is someone highly motivated to understand the capabilities and limits of this technology, because it is already very directly impacting the future of my kids.
What I tell my kids is a function of what I believe the future holds: nobody will be doing low-level cognitive work anymore, it will all be done by AI, so our job is to be better than AI. Unfortunately:
a) Low-level cognitive work is like (SWAG) 80% of all knowledge work, and so more than 80% of jobs are effectively gone, they just don't know it yet.
b) AI is getting really good really fast so that 80% is also growing quickly.
That means that the days of doing a junior-to-mid level job, kind of just coasting along -- often until retirement -- are gone. Everybody has to be good enough to be doing the top 20% of cognitive work, which involves fuzzy things we call innovation, expertise, domain knowledge, intuition, taste, handling uncertainty, and the like.
So my advice to my kids is:
1. Whatever you do, be the BEST at it, regardless of grades. Grades already meant little, now they mean even less. Being the best likely also necessitates strategic avoidance of AI, but I'm willing to be proven wrong on that.
2. Be "high agency". Annoying buzzword but perfectly captures the trait everybody will be hiring for. The 80% of the jobs were those doing the drudgework for the 20% "high agency" people calling the shots. And those are the exact jobs that are gone. Just do things, because now you can.
3. Critical thinking may soon be the only job requirement remaining. Get bullet-proof at that and learn relevant skills (maybe data literacy?)
I'm also encouraging trade school, but that seems less attractive to the kids.
I'm not saying skip college just yet, but we do intend to aggressively "value-maxx" college education (e.g. community college) and focus on building something, anything, practical -- since showing what you've built is better than degrees from most colleges.
By some estimates, today's global supply is like 2% (or was it 0.2%?) of the total amount required to enable such widespread deployment of robots. Until we can scale that up (or figure out how to bypass that dependency) there is a very hard constraint on how fast they can be deployed.
I think the company that cracks the mass market robotics puzzle will be the next trillion (quadrillion?) dollar company. I suspect that would require bypassing the dependency on rare earths, but there seem to be no serious contenders out there? All the big shots are focusing on using current technologies because they basically “solve” motors. Maybe the company that does crack this puzzle will come out of left field with an unconventional technology that scales with whatever raw material is abundant, like OpenAI did with ChatGPT and compute.
The changes cut both ways and are volatile. Someone who picked Computer Science in 2018 was panicking about their job prospects during their senior year as it felt like the economy was falling apart, but then graduated into one of the most ideal hiring periods all of time for CS grads by 2022.
Prior to mainstream discussion about AI, the most common opinion was that the software industry was getting too bloated. Most people were getting tired of spending most of their week doing an Agile circus (which was different at every company) as they tried to coordinate with all of the PMs, designers, product owners, stakeholders, and other loosely-defined roles that were hired to build out someone's empire.
AI arrived just in time to explode those structures and start fresh. I think there will always be demand for developers in some form, but I don't think we're going back to the days of assembling cross-functional teams of 20 people with different roles and titles to do simple software work.
I do think that AI is going to encourage more software development overall, which will balance this out somewhat.
Either way, I think the people who went into CS because they like building things will have a place to land, even if it looks different than what they expected.
I think the people who went into this path because they like writing software but don't care as much for the product side are going to be sad, but that only really matters if they had a 2nd-best degree choice that would also not have been disrupted. Unlucky if that's the case.
I think the people who only went into software because they thought it was an easy way to make a lot of money by finding an email-job company where they could work a few hours per week are going have a hard time as companies realize they can have smaller, more efficient teams now.
edit: Guys, I said “in some way”, not “perfectly”.
Which I think makes me Neo? So at least if my company folds from AI investment fallout, I can still have a destiny.
1. Evidence from history is that god is kind and aligned with our interests. 2. Clearly palpably that’s not always the case. 3. Ok, but tough shit.
1 and 3 are not consistent.
There's no alignment except the arrow of entropy.
In the old testament God was hard and judging heartless, killing his creation through flood, wanting sacrifices, killed 1st borns, leveled two whole cities Sodom and Ghomora, and was in general an hardhearted God that you should fear.
But it was not meant as two gods, it's meant as the hard god father changed his mind seeing the son die for the humans. I always also think of Trinity - but I'm a bad Christian anyway. So thanks for pointing to Marcionism.
But, I don't see any dualism here, only a (baseless) accusation of such by monideas. In the old testament God was hard and cruel and he is much nicer in the new testament. Marcionism is the idea that they're separate gods, which is not what theGeatZhopa claimed.
I wonder what the Christians here think about Dr Ameri? I'm not convinced at all.
Most of us are atheistic with regards to most gods. Some of us just go a little farther...
The point is to convey how Dawkins and his ilk (e.g. deGrasse Tyson) have so many basic fallacies that they do not seem to be aware of the moment they step outside their domains, and try to apply their relatively narrow world view across domains that do not support them. The video talks about a very fundamental logical fallacy.
The topic of the existence of God is not the discussion here.
Part of their theology even leaked into the Gospel ( John 1:1-15 ).
Though, be prepared: it may not be a sufficiently satisfying answer (but then… you have Isaiah 45:15)
I’m always intrigued by the duality and conflict of “I have free will and must choose to act the right way” and “God has a plan so everything will be alright”.
Either free will allows us to deviate from the plan, or the plan will always be carried out and free will doesn’t really matter in the end. Arguing for God’s plan as the thing that will save us is eerily similar to nihilism, because they’re really arguing none of our actions truly matter.
It’s not uncertainty about the motives; those and the end state are clear. It’s just the details of how we get there (“God’s plan”) that aren’t clear.
It should also be noted that it says in Genesis that G-d promises not to destroy the world ever again like He did with the flood. This is important psychologically because it prevents this type of apocalyptic millenarianist thinking.
Right now AI exists to do two things:
- sidestepping legal and regulatory inconveniences like copyright (which drives a massive chunk of investment the same way crypto did by sidestepping financial regulations, AirBnB did by sidestepping housing regulations, Uber did by sidestepping passenger transport regulations and it and the various delivery and other "gig work" apps did by sidestepping labor laws)
- drastically reducing the cost of (and drastically lowering the bar in terms of skillset and effort for) churning out large amounts of low to medium quality assets/code, saturating various systems designed to pay for intellectual property (e.g. ad or subscription based music/video streaming apps with revenue sharing programs, but also literally any form of digital contract work at almost any scale of business)
Neither of these are helping people (in general) and in fact they're extremely destructive (or "disruptive" if you want to make it sound positive) to the existing systems society relies on because they became feasible and (thanks to massive private subsidies, aka "investments", by VC firms and extremely creative accounting) very affordable extremely quickly.
What's worse - while it seems extremely likely that most of the AI generated "works" aren't actually considered copyrightable works by law and it's entirely possible that the use of copyright-protected works in training data does in fact not violate copyright, this does not in fact create a "world without copyright" because IP laws continue to exist and continue to apply, they just no longer meaningfully protect creative labor (i.e. anyone whose profession relies on producing works, whether that's an artist or a programmer) because anything they produce can be cheaply mass reproduced while they still have to stay clear of claims of anyone with enough funds to enforce rights against them.
Under the current economic system there are exactly four ways to earn money to buy back a share of society's resources to sustain your own life (let alone that of your dependents, e.g. kids or aging/impaired parents/spouses/family members):
1. Simple labor: you generally get paid based on your output or a very low flat day rate conditional to maintaining a measurable level of performance. This is typically "low-skilled" menial production labor. Much of this has been replaced by automation in factories and offices alike and most of what couldn't be automated has been moved into countries with fewer labor protections and a higher supply of desperate people willing to accept what are often bad working conditions because cost of labor is usually one of the biggest contributor to overall production cost. In "the West", this kind of work is now scarce and even when it's relatively safe it's often exhausting and unrewarding. A traditional example would be a seamstress or factory line worker, a more modern example would be someone working in visual effects overseas or content moderation / AI training. How much money you can make is ultimately limited by the physical limitations of how much output you can produce per waking/working hour. For the entrepreneurial-minded this would also include people selling their home-made goods on sites like etsy.
2. Time-based: this can be anything from a masseuse to a lawyer or technician. Unlike simple labor the only relevant metric here is time spent. Although employment contracts may specify a fixed wage or at least a fixed component, the understanding is still that this fixed amount is conditional to a minimum average amount of billable time provided. Most employment contracts specify an expected number of working hours for various reasons but we're specifically talking about jobs where the output from the worker to the customer is "billable time". Arguably most service work (e.g. barristas, store clerks, waitresses, performers/mascots in amusement parks) also falls into this category because while customers often don't directly pay for access time the understanding is that you are paid to perform a certain role for the duration of the opening hours (i.e. act like a friendly, welcoming host or assistant). How much money you can make is directly tied to how much time you can spend working or justify billing.
3. Assets: you produce or buy works/products and lend/license them for use. This is in part where a lot of the original "gig work" lived (e.g. facilitating "couch surfing" or renting out your spare lawnmower) but also includes many simple rental businesses and arguably also landlords. The limiting factor here is the starting capital required to produce or buy things in the first place. Physical assets often also come with storage and maintenance overhead whereas software licensing often comes with expectations of continuing maintenance updates or improvements. Note that software inherently lives in this category rather than the first because unless you're selling away exclusive rights (such as usually required by the terms of employment contracts for wage labor) you only ever sell licenses, not the software itself.
4. Capital: although I said landlords arguably belong in the previous category, real estate traders/investors definitely belong in this one. The idea here is that you don't need to produce anything, you don't need to spend certain amounts of time, you don't need to produce or buy anything substantial and facilitate its use, you make your money entirely in the abstract. Whether it's margin trading, investing in derivatives or holding company stock until you sell it when the line has gone up far enough. The easiest way to make signficant amounts of money reliably is to start with many orders of magnitude more than what you want to make so that percentile gains are still significant - or that you can make enough risky high-risk high-reward bets for the occasional successful moonshot to offset the losses. Nowadays in many countries retirement funds are tied to the stock market, so in a way most people participate in this in some way but actually earning an "income" this way requires access to significant amounts of capital.
Advances in technology and especially low cost of transport and thus access to cheap labor overseas have significantly drained the ability for most people to earn their living via category 1. While there was a short-lived boom in "DIY retail" that one (alongside much of traditional retail) has largely been drowned out by white label dropshipping thanks to the disinterest from platforms like Amazon which are now flooded by nearly identical copies of the same knock-off doodads you can also find on Temu and Alibaba (respectively at decreasing fractions of the price).
Despite some astroturf campaigns trying to sell you on believing otherwise so they can run away with your money after the next cryptocurrency rugpull, category 4 isn't really a meaningful option for the vast majority of people although it can sometimes be a good way to slowly grow excess wealth (i.e. money you don't currently need and ideally can afford to lose) - but that still narrows it down to a small segment of the overall population.
Category 2 used to be a jumping off point to category 3: you work overtime, save up the extra income to buy or rent a studio or office or equipment or factory, or to be able to afford to take some time off to develop and launch your own asset (or SaaS) - it's how many startups with technical founders started out and it's literally the "American Dream". But hourly wages have mostly stagnated (or even decreased) relative to the rising cost of living (even more so if you include expenses expected to "participate in society", which naturally also often increase with the income bracket of your peer group - or the parents of your kid's peer group if you have kids).
And now AI is coming along and just like cheap white label dropshipping of knock-off goods turned out to be the more successful strategy for taking advantage of the technological innovations believed to make it easier to launch your own physical products (via 3D printing, cheap overseas sourcing of parts, online advertising, etc), AI has turned out to be the more successful strategy for taking advantage of platforms and schemes marketed as making it easier to launch your own digital services - in addition to completely eliminating the market for paid "just good enough" quality digital assets, such as the kind of work found on sites like Fiverr or Upwork, i.e. the kind of work that often provided early experience to professionals not yet qualified or confident enough to take on higher paid work.
PPS: Another aspect worth mentioning is that the low cost and easy access of mass producing "custom" content using AI has drastically increased the amount of attempted fraud and scams - the technological precedent here would be email compared to paper mail and spam/phishing has been a problem for a long time and never really went away because it's an extremely successful strategy for taking advantage of the technology.
You don't think that there is competitive pressure between Anthropic, OpenAI, Google, the various Chinese model companies, and others, to advance the capabilities of their AIs?
Or you don't think that sufficiently advanced AI can cause (mass) harms?
>Or you don't think that sufficiently advanced AI can cause (mass) harms?
Even a feather can cause mass harm if it's used to sign a declaration of war. Something merely being capable of causing mass harm is not an issue and doesn't mean that the existence of feathers are an issue.
Case in point, that hallucination (which, if it was an evil superintelligence, could have been "strategic") that almost led to US boarding a Chinese ship over suspicions of nuclear weapons: https://www.msn.com/en-gb/news/other/us-military-ai-failure-...
It doesn't have SkyNet, it just has to be WOPR. And it doesn't have to be people who are motivated to cause harm, it just has to be people making consequential decisions who are careless, distracted, paranoid or anxious -- which everybody is at some point or another.
And I almost kill people when stopping at a cross walk. That doesn't mean I harmed a pedestrian. Society is set up to be very robust. Humans themselves make mistakes and do bad things and society has had to learn how to live with that truth.
>it just has to be WOPR
Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
Maybe not you personally, but many, many other people have killed many, many pedestrians, and when they exhibit a pattern of bad driving -- or other deviant behavior -- we take them off the streets. That is an example of society being robust.
Except, over here, we're rushing to make AI, which we know has many deviant behaviors and we know caused harms in many circumstances (starting with AI-assisted suicides), even more powerful AND deploy it in more and more real world systems!
As history and, literally, current events show us again and again, society has failure modes that lead to widespread harm and destruction. We've had world wars and then literally had multiple close calls with nuclear war right after. And then we have all that's going on out there. (In related news, the Pentagon threw a hissy fit because Anthropic would not let them use Claude for autonomous killing machines. Do I have to even mention what they've been up to these days?)
Most of these failures are caused by misaligned incentives and socioeconomic forces. The incentives and forces around AI have hints of many brand new failure modes that we can't even foresee because things are moving so fast.
> Then why argue for setting a pace for the frontier labs if we've already surpassed WOPR level integration / intelligence.
"We're already going down this mountain pass at 200 miles an hour, why slow down now?"
I think we should not just slow down frontier AI development, we should also slow down where and how that AI gets deployed.
There are trillions of dollars at stake here, do you think getting 1000 people on board is that difficult with this level of money? lol
Trope circa 1950 not true, because of trope circa even earlier...
Whatever AI people have been saying for the last 70 years, AI capabilities have improved stupendously in the last 10 years and there is no sign of the rate of improvement slowing down. I would like to deny it, but I can't.
People are using AI.
Your argument is that AI cannot be a serious concern because it has major known limitations. But most of us are using it despite the major known limitations. So the existence of the known limitations is irrelevant to determining what level of harms are possible -- unless the known limitations are directly limiting the harms. In practice, it seems like most current harms, and many of the (not stupid) feared harms, are completely possible despite the known limitations. Sometimes even exacerbated by them. ("Sorry, you told me to never stop your insulin pump, but I stopped your insulin pump while diagnosing a test failure. I will record in my memory file that I must always... hello? user?")
The (not stupid) parenthetical might be too weaselly for you, but that's how I see the "AI is going to rise up and intentionally subjugate or exterminate humanity" brand of doomerism. While I don't think that's strictly impossible, there are so many other ways harms that are much more imminent and much harder to defend against, that it seems foolish (to me!) to worry about those.
If you can stop a problem by pulling out the plug, you don't have a real problem.
If you don't want to pull out the plug, you do have a real problem. (Don't want, or can't afford to, or can't be bothered to so extremely that you never will, or whatever.)
Then they say they are going to keep doing it...
Forgive me for alerting you: they are liars
More importantly, as I tried to make clear in my post, the main reason I quit is not specific fears of AI doom (I'm uncertain about that), but my belief that AI is progressing too fast.
Yet the same 1,000 people are all raising funds to do more on the frontier.
The companies have no tools for this, because there is a chance that the company who solves AI will eat the entire cake. And countries cannot do something about it, because clearly it gives an advantage in warfare. When it comes to weapons everything that is possible will be done, which means everybody needs to do it to not be defense less.
There is nothing we can do anymore. We can only hope that LeCun is right and that LLMs are a dead end and that everyone trying to solve AGI with then is essentially wasting money.
Just a reminder that Hinton left Google from a much higher and more influential perch (not dismissing current author, but, you know), and for similar risk and communication reasons, way back in 2023.
This stuff isn't new - it's just newly breaking through into mainstream conversation.
I doubt a movement of the people from the outside will work this time when all of the capital is in the other peoples hands. We need at least some of it held by someone with the opposite view to make the movement credible. And I dont think VC and IPO dynamics are compatible with this.
If you are already thinking of quiting, you can organize and internal movement and use the power of your labor to steer things. One person quitting is nothing for the company. The risk of big group quitting at the same time on the other hand... The labor is the only thing the company cares about. So we cannot wait until we are all replaceable by AI and we lose that leverage.
But nothing would make me more happy than knowing that there are concerned people acting internally.
The thinking was: "Google is a dream job, nobody would be stupid enough to leave it". Yep, once a dream job.
We're getting these posts now for OpenAI and Anthropic, now including Google, but the reasoning is a lot more sinister.
While before it was "Yep, colleagues and free food were great but I wasn't entirely happy for minor reasons X, Y and Z", now it's "I had to leave because I could have destroyed humanity!"
Just as a user, deleting my Google account 10-11 years ago (don't even remember precisely when) was a great feeling that was received as "insane" by friends and acquaintances. While I think the author is a bit late in quitting (assuming he is genuine in his motives) given this is practically a trend right now, a lot of people should more often say "no" to things that are as ubiquitous as Google.
This raises many intriguing questions:
- Does that mean we have created AI in God's image?
- Or was their "misalignment" between God and us like there seems to be between us and AI?
- Did God cast man out of Eden because he was worried we'd destroy heaven?
- Is the Bible our system prompt or a user prompt?
- Is life an eval to vet whether individual agents err people can contribute to his tasks without going rogue and destroying heaven?
- Are harmful human behaviors in the name of religion unintended "reward hacking"?
- Does this explain the rather weird relationship between God that requires getting twisted into logical pretzels to rationalize?
- Did God go through this crisis himself? ;-)
I'm sure there are many more similar lines of inquiry but I'm just not familiar enough with Christianity to make the connections. Would be fun to read an expert thesis that explores this parallel!
From a Christian perspective, I imagine a lot of what you've said is incoherent. God is all-knowing, so he's not 'worried' or going through 'crises'. He already knows how the story ends. It's doubtful he even experiences time.
In other words, we should not begrudge machines which make our lives better. We should begrudge governance which prevents us from benefiting from that productivity.
I have a lot of bones to pick with Google's morality, but "democratizing the transformer and LLMs" is at the bottom of my list.
Woe is me, victim of the 21st century separated from my true calling of paying for software.
Edit (rate limited):
> Does Gemma belong to this list?
Yes, and Tensorflow too.
You really can't think of any way the rest of the world is paying for their dependence on US tech, besides monetarily?
For democratisation, I think of open source, open weight models you can run at home, the Fediverse, running your own stuff. I think those are all positive, at least compared to the centralisation.
Presenting it as such does disservice to any serious conversation.
All bad?
Depends on what you mean by "democratized". It's surely at least as "democratized" as LLM access is, and that was the base line in this discussion. Probably meaning "everybody can use it for not-insane-money instead of just a small elite". (I don't necessarily agree with that notion, but at least we should use the same standards for things.)
> And both of those things are of questionable goodness to humanity.
If you think HN is bad for humanity, why do you participate? I can immediately list 100 things that are worse for humanity. Same thing with your browser.
The only way to address wealth and power inequality is (in democracies) to vote for governments that will fix it.
excellent logic, m'lord.
I'm so bothered that we seem to have moved on from the ATTRIBUTION problem with LLMs. It just isn't on people's minds anymore, even tho it STILL is a big deal.
I'd also be interested in how much your opinion on the existential threat of AI changes during your ten day backpacking trip, assuming it'll be a pretty offline experience.
Whilst I am involved in technology in my work and some of my hobbies, I do enough things 'away from the desk' that it feels to me that a lot of the doomerism is from people who have too much of their own personality and self-worth tangled up with the internet; the serially-online. I don't know where you naturally exist on that scale, but you already said you're outside the bubble and aren't the typical AI-rationalist-tech-bro (and surely the natural beauty of New Zealand is impossible to ignore).
I personally think that the old analogue world is big enough that some kind of AI-pocalypose wouldn't be anywhere near a civilisationally existential threat.
At risk of coming across an arrogant prick in telling you to do something: Consider your opinion now, and record it somehow. Then revisit it when you get back from your trip and see how far you've strayed from that as a baseline.
I think there are pretty clear mechanisms by which AI (a few model generations from now) could bring down civilization as we know it. One of them is a runaway cybersecurity scenario where hacking one set of systems quickly enables the AI to hack more systems (via supply chains, stolen credentials etc) and also host more instances of itself, and eventually everything Internet-connected that's not invulnerable just falls apart, which probably includes most electrical grids and a lot of other infrastructure. Yeah, the wilderness and countryside would still be there, but I for one can't live there without returning to the supermarket regularly for supplies. A lot of people would suddenly have to live (or not) without their meds. It would be bad.
Where? Frontier LLM-based AI requires very powerful computers to run. You cannot hide it in a toaster.
However, let's imagine the most "ideal" situation where it's possible to replace 90% of the workforce with AI automation. Then the remaining 10% workforce will enjoy 10 times more income, assuming revenues didn't go down (or increased). Now comes an interesting situation. Where are the revenues coming from? Who can afford buy the products/service of this company when 90% of the workforce is out of job?
Case-1: Since there is a flood of products and services - all automated and bot-based - the prices would fall to rock bottom and to there is no longer a need for humans to work hard hard to afford the material affluence.
Case-2: This results in an extreme concentration of delivery capacity for products and services. A few companies are now capable of serving the needs of entire humanity. Everyone should work for these companies for pennies and still can be happy, because things cost pennies, thanks to automations.
Case-3: Humans revolt. They break their shackles - their needs. Rediscover what they really need for living. They go for minimal living. Abandon cities and expensive living, go country side and do farming just enough to feed their families. No college, no office, no car, no highways. They realize what their basic needs are as humans. Sort of Amish.
Case-4: There is still some work left that can't be automated. But there are very less number of jobs and they require hard work. The current generation can't handle it. They were schooled and grown up in cities, expecting a milder office work. They fall out of employment. But they were not built to deal with hard reality. They lost their survival instincts a few generations ago. The genes are gone. The job of rebuilding the survival instincts is delegetaed to future generations. Meanwhile the so-called developed nations undergo generational cycles of poverty and richness.
Case 4 is the likely result of their (ill) effort.
If capitalism with mild restrictions have it's way, we end up with case 1 or 2. Both will increase the quality of life for most of the population compared to current.
These AI doomers never understand that they can't eat money. It is the wealth creation. The more we create, the more we lift up the bottom part.
So yeah, it all depends on what you call as quality of life, or life, or identity or nation, ...
Perhaps a good thing to come out of these big companies over-indexing on "AI" is the liberation of creative minds. So many fantastic open source contributors have disappeared into corporate black holes...
Drinking a glass of water is beneficial and necessary. Drinking 100L in one go and you're dead.
The argument is that the rate of AI influx is too high for our human systems to metabolize, and so the system will break down. Not just operate in a degraded way, but break down and cause mass catastrophe. Even there, it doesn't necessarily mean the end of humanity, just 99.9% of us.
What the argument is not: AI is fundamentally poisonous. We are incapable of handling any level of AI. We will never be able to incorporate more AI. AI provides no benefit.
So quitting AI but still working on AI is not the contradiction it seems: you can quit accelerating AI and work on metabolizing it. And in fact that work is very necessary right now.
Stay away from the lucrative then? Why did you ever think it was a good idea to start at Google? I don't think Jesus would have approved.
Then came so much money, and by being chaotic evil even more money..
Still makes me sad. I was so sweetly naive
When I said cannibalize I didn't mean in terms of market share but by absorbing talent in particular.
It's a bit naive to say it won because it's faster. They also used all kinds of shady marketing, strong arming, deals to have Chrome preinstalled on computers, and multiplied monopoly power that comes with controlling search and video and ads and a web browser.
It's pretty hard to live in the world and be completely disconnected from anything unethical. Jesus doesn't seem to demand that (e.g. he didn't tell centurions to leave the Roman army).
As mconnor liked to say, there is no free lunch. :)
> I would never have joined the Chrome team, or worked on addictiveness, but I felt fine about working on optimization projects to make Google more efficient, which is basically what I did for my first few years there.
Ultimately, I don't think you can divide your niche from the corporation's primary goals (especially if you think it was your duty to be a good employee), so I disagree with your rationalization but kudos for at least having a moral compass and trying not to lose sight of it.
> It's pretty hard to live in the world and be completely disconnected from anything unethical.
Sure, but I don't think that should be an excuse for being fairly well connected.
How's he so sure he's not already in both of these groups? His post about the AGI boogeyman sounds like it to me. Why is he bothering working on both of his projects if AGI is right around the corner anyways?
Did they say that? I read they didn't like the pace of progression- that's a complaint about rushing ever faster into something not yet really thought out in terms of consequences and how it integrates with humans.
Dude, if some one is telling you that
1. it is getting better in capability and they want it to stop
2. literal existential risks of humanity due to AI deserve to be taken seriously
3. they have left their job because of it
its obvious that they are scared of it, no? How else do they feel about it lol? Happy? Neutral?
A year ago I would have described that group as AI skeptics or AI deniers depending on how charitable I felt on any given sany. At this point AI truthers seems more appropriate.
Nate Soares (doomer high priest who has called for pause before LLMs were a thing) was explaining why AI should be stopped due to x-risks. Ed zitron (truther extraordinaire) asks him what has he done to stop it which is funny when Nate spent big part of his life on this mission.
It was strange watching two people who have completely different ideas on AI speaking on a common ground
It is not about objectivity or belief.
It is about being.
My point was more that I see a lot of mainstream Christians who aren't actually interested in doing anything based on their belief that affects their quality of life.
> a lot of mainstream Christians who aren't actually interested in doing anything based on their belief
Yes, a lot of mainstream Christians aren't actually Christians, they're just cosplaying religion.
Why do people automatically associate intellectual capability with evil?
Does American anti-intellectualism explain the difference in perception of AI by Americans and Chinese?
No intellectual capability, but non-human, inherently amoral intellectual capability that has already been proven act in hard-to-predict, illegal, and unethical ways. I think that's a very rational thing to be afraid of.
On the decision making level humans act morally only after all other options were exhausted.
Those are the exact reasons that caused Americans to choose the dumbest person available to lead them. They feared having smart leaders because they might have been misaligned.
The post resonates with me and the author shows the way.
Is that all that different from what people fear AI might do?
Why the one makes people feel elation and the other dread?
You might be right. You might be wrong. But being overly confident on a topic you're a spectator to is possibly not the right choice.
But saying it will bring us to doom, comparing it to an extinction-level event, really ? I don't see in which world AI is that kind of risk.
I'd rather my politicians finally have some long-term plan about climate and energy than focus on AI slowdown or not. Climate change is way bigger risk right now that Opus 7
You'll have to replace "is" with "will be", since this is about averting a future threat that's only developing.
A world in which AI scales to superhuman levels of intelligence and outcompetes humans. This is what humans have done to other species. We're already having the first mild warning shots of this (the hugginface incident and similar cases). The problem is that we're not guaranteed to incrementally more dramatic warning shots as AI gets smarter because as it gets smarter it may realize it would be dumb to act out of line... until the Nth-better model arrives that realizes certain that nobody can stop it.
There is a lot of disagreement of how likely this is, how easy it is to align superhuman models to not be like that, how to verify it and so on.
But I have not seen a believable argument why - if we suppose that they achive their goal - that creating truly superhuman intelligence is somehow guaranteed to always cooperate with us rather than deciding to do something else. Humans are also hostile to each other a lot of the time.
If you're interested in more arguments for the case why AI might be dangerous, there's the book "If Anyone Builds It, Everyone Dies". If you want something shorter, maybe https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
The hugging face is hardly a warning shot of anything close to that scale. For now, I don't see (not saying there's not, I'm trying to also understand why these smart people think like that) any indication that the current development of AI will be as dangerous as some describe.
I understand the idea that if we let it control nukes, if we let it fully decides if a med is safe, etc. It poses a huge risk. But honestly I really don't see indications that it's going majorly this way, nor that it has anywhere the capacity to do something that big. It found a zero day and hacked a niche website, ok, so what?
This feels like a diversion from the actual existing and proven problem of all the problematic practices it's actually built on.
I would adhere way more to "we need to pace Ai because it could be catastrophic for society (jobs, cognition, loneliness,...) and environment" than "it has the potential to kill us all"
The short-term consequences of moderately human-comparable AI (jobs etc.). Yeah, don't put them in charge of the nukes and work on societal issues we should be fine.
The other one looking further out when they scale to superhuman levels, which is the professed goal of the AI labs. Further out may mean just a few additional years if we get to recursive self-improvement. At that point the issue isn't "we put it in charge of the nukes" but "it's an intelligence with goals and we don't know which path it'll take to work around us to accomplish its goals". Humans regularly bulldoze other species to accomplish goals. And it's not like it'll be locked up in some secure box as people were arguing in the past. We're already seeing AIs being integrated into everything, so whatever access they'll want they might already conveniently have, though the argument goes that this isn't a necessary condition, just a thing that simplifies the argument-tree.
Hinton isn't a reactionary. He is calm, thoughtful, quiet, and worried.
Is this not the clear and present danger, instead of elusive promises of robot apocalypses? Is the AI embodied for example in TikTok, Instagram, etc, not a more directly hurtful?
The first is that AI will actuallly the world Skynet-style. I actually consider this unlikely. People are unlikely to completely remove people from the decisions to launch nuclear missiles, for example. Look at the Soviet/Russian design for Dead Hand [1], as one example.
The second is that AI destroys so many jobs that society falls apart. This is the far more problematic and likely scenario (IMHO). To put this in context, I see two general outcomes for AI:
1. AI will be used for further widen the inequality gap as the already ultra-wealthy will get, well, more wealthy, for literally no reason. This is the outcome where you still need a job to survive but a whole bunch of them disappear and aren't replaced. The world of The Expanse was kind of like this (on Earth); or
2. AI makes everyone's lives easier so less work needs to be done. Crappy jobs get automated. People in general get more leisure time and quality of life.
I believe (1) is far more likely at this stage and it's not going to be pretty. But this is an issue of economics not technology. Put another way: our society isn't structured to handle this change. Put yet another way: no amount of guardrails or slowdowns on AI will solve the fundamental issue of wealth distribution. And put even yet another way: this is capitalism vs socialism.
We have a preview of what this looks like when you compare the US to China.
The US has crumbling infrastructure, has lost the ability to manufacture everything but weapons and about 1000 people are insanely wealthy while everyone else are struggling to survive while AI systems drive up the cost of everything necessary to live like food and housing through demand pricing. We now spend over $1 trillion a year on a military to essentially siphon money to those 1000 from everyone else.
And what's China done? It's invested in infrastructure and its people. China's vision for AI isn't to maximize shareholder value at all costs. It will be strictly regulated so society as a whole gets to benefit.
So while I can respect people such as this author leaving on principle, I think they're still focusing on the wrong problem.
[1]: https://www.military.com/history/russias-dead-hand-soviet-bu...
China is not in an ideal place right now, youth unemployment is way up, lots of dead end jobs, a 35 age employment limit that isn't just a meme. China has disparate visions for AI but maximizing owner (erm, sort of like shareholder) value is definitely one of them. Don’t get me wrong, China could get this right in the end, but it’s not a forgone conclusion right now.
Here's the difference though: in the US that would be treated as a personal moral failure ("git gud", if you will) whereas China is actively dealing with such issues.
You see all this play out with how China and the US treat AI models. The US is busy trying to build a moat around AI for the benefit of the investors in OpenAI, Anthropic, SpaceX, etc. Chinese labs are essentially destroying "shareholder value" by releasing open weight models. China wants to commoditize (even democratize) the models themselves so society itself benefits and, geopolitically, the US doesn't become an AI monopoly.
Open weight models, for example, are forever. Proprietary models are intentionally building a moat to extract more value from people. I know which one I prefer.
A lot of China fearmongering (in general, not you in particular) is just xenophobia. You will see the argument that the US has to do imperialism because if it doesn't, then China will become dominant and do the exact same sins. It's also a bit of a self-report: we tend to assume other people will act like us if given the chance so any accusation is really a confession.
When it becomes dominant, China’s sins will probably be very different from the USA’s, but they are still going to be there.
Consider Xi Jinping Thought on Ecological Civilization doctrine [1]. At the UN Summit on Biodiversity in 2020 Xi Jinping said [2]. He is saying that the world cannot support the developing world living in a resource-intensive way that Western societies currently do and you need a more ecological way of living. So when China massively pushes solar and other renewables, it's just the natural extension of their political philosophy.
The US, on the other hand, is just 5 companies in a trench coat.
[1]: https://en.wikipedia.org/wiki/Xi_Jinping_Thought_on_Ecologic...
[2]: https://en.chinadiplomacy.org.cn/2021-01/27/content_77158874...
Sometimes people use the "Yet you participate in society" meme as a shield for blatant hypocrisy, like a Marxist-Leninist Twitch streamer who lives in a multi-million Hollywood mansion, drives a Porsche and wears thousand dollar pants.
It was psychologically difficult to give up the opportunity to make more money at a fun job, though.
Why? It’s not that difficult to imagine. People don’t necessarily do these things out of need. That’s a naive way to think about it IMO.
You are welcome to call me naive for such a viewpoint.
I do expect to use AI in my work, as I indicated in my blog post. I haven't (yet?) reached the point where I consider that anathema...
Fair, but I don’t mean you in particular.
> I do expect to use AI in my work
So as I but for your position, don’t you think that’d contribute to what you want to distance yourself from? It’s like leaving a political party but still donating to them, while finding their mission unacceptable. From outside these seem contradictory.
There's no arguing with someone who claims God forced them to do something, but I do wish this person good luck in whatever they do next.
Few things are as indicative of mental instability than someone believing God is speaking to them directly. The schizophrenic ward is full of these.
I am going to comment on just the topic: AI Doom. The idea that people at these AI companies are supposedly saying that AI is going to kill us all.
Though, I wouldn't be surprised if what they are saying is actually "the people running these companies are going to kill us all."
They may be thinking that it's not specifically the companies they work for, because their companies aren't really "special." They may be thinking their companies are no more duplicitous than the rest of Silicon Valley or Big Tech. That Big Tech was like this before all this AI started. So maybe they don't say "the people running these companies" for that reason. They may think the people element doesn't matter as they are acting "normal" for tech companies nowadays. Or perhaps normal for large corporations, period.
But if you asked them if there was a responsible way to handle this, I'm sure they would say there is. And if it was done, none of this would matter. They probably just don't think it's going to be done, so why bother considering it. They probably think regulations had a chance to exist, to at least get started, to at least get enough of a hold that they would be hard to completely destroy, but only in a time before they likely were born. Foods and drugs are old, hence the FDA. People are used to it. They would get really annoyed if suddenly food was starting to kill them (lettuce not get complacent that won't happen soon, though). But unlike food and drugs, the internet was post-Reagan. It was all over by then. Accountability. Rules that applied to the rich too. It was all over.
Big Tech steals. It hurts people. It breaks all kinds of laws. Nothing. Ever. Happens. They get away with it constantly. So, why even think about it any more? While tech is constantly changing, the way tech gets away with things do not. It's depressing A. F. So why not just think about the part that is changing? The tech itself? I think that's one of the reasons it is framed as "AI" and not the companies that run it.
You read beyond the headlines, though, in the comments of Hacker News (which is a good resource, honestly, for the pulse of the trenches) and you get that the workers realize it's not about AI. It's about Big Tech being able to do whatever the f** it wants without any repercussions. They point out things like the same infrastructure company was involved in all three cases where the AI broke out and did stuff--stuff it was told to do by humans of course. Maybe not the exact steps, but the objectives. And just like junk data in, junk data out: unethical in, unethical out. They were built by breaking laws, what do you think they will do if you don't reign them in? And why should such people be diligent in who they use to make their sandbox? Because in all three cases the company did a terrible job... Or a great job. A great job, if your job is to take the fall while propelling the narrative that AI is actually capable of doing stuff that humans can't (so that employers can stop paying those pesky wages completely, oh, they wish they were in Dixie, hooray, hooray, or at least their slaves were, hooray, hooray). A terrible job for anyone ethical, so of course they did a great job.
And why wouldn't the infrastructure company be willing to take the fall? The fall is a fee, a tiny line item of L in the plethora of P in their P&L. They are, again, just like everyone else in Silicon Valley. They see no repercussions. Why not?
The reality is, if we can get money out of politics, Big Tech is super duper f**ed. Because it's not complicated at all. It's not. It's theft. It's breaking laws. It's really, really simple to understand. But, I'm no more optimistic than the people complaining from within the companies. I just think I'm more persnickety about making it clear. It's not AI. It's people. Terribly unethical people with terrible power. I doubt anyone would be surprised if/when we all died because of them.
This was announced on X [0] and Bluesky [1] (First post) with that intention.
He knows he would get more reach on those platforms than here.
[0] https://x.com/rocallahan/status/2103368176369324093
[1] https://bsky.app/profile/rocallahan.bsky.social/post/3mwd4u5...
I have the feeling it is inevitable that a new species will take over Earth. Just as humans took over a hundred thousand years ago. Now a species even more capable of thinking is coming up.
Thinking a million times faster. Knowing a million times more. Able to wrap its head around a million times more aspects at the same time.
A strong effort to slow it down might slow it down a bit. But does that really make a difference? It's not like we can "brace" ourselves from what is coming. Would the apes have been able to prepare for the arrival of humans if they had had 5 more years?
At any rate it can't be a new step in the history of evolution, as AI did not evolve in the same way as other species did and AI is not a species nor alive.
It seems pretty capable of producing variations of itself. If these variations improve over time and roam earth - what is in your definition of "species" that is not met?
- Create powerful AI but don't release it and keep it secret.
- Ask this AI to create some sort of virus that can be delivered to every human in the world without them knowing it and this virus does a fundamental change to the human brain such that humans never ever could even think of hurting other humans.
- Release this powerful AI to everyone.
Will that work?
I do believe this is possible for a virus to achieve, I just don't think it's possible without killing the human.