If I say "The founder of OpenAI is scanning people's eyeballs to build a database" that sets off real alarm bells.
You are supposed to preface it with how Altman is making AI services, selling them as AGI, and also painting AGI as a danger to the survival of humanity. Then, when they are baffled at why someone would sell something while describing it that way, you can walk them through regulatory capture and erecting barriers to competition.
> Creating safe AGI that benefits all of humanity
In big letters.
They are very clearly playing the game of positioning themselves as the AGI good guys, and at the same time Sam Altman has been encouraging the view that AGI is dangerous.
Regulatory capture.
OpenAI markets their product exactly this way, focusing on how ChatGPT can 'write code' (it cannot) or 'write poetry' (it cannot) or do any number of things that lean in hard on the 'stupid AGI' angle, never clarifying that their actual product is a spicy autocomplete. You'll never hear OpenAI say 'ChatGPT is a stupid AGI' out loud, but they'll never correct any mainstream misunderstandings that it is.
a lot of the dismissive takedown of LLM's intelligence uses their spurious knowledge of the world, but it is quite easy to have them 1. search the internet 2. search research articles 3. read books and have them work to iteratively improve something.
Right now they cannot do much reliably but calling them a parrot is miopic how little we understand how humans are intelligent and to how LLMs today are more intelligent than a non negligible chunk of the population.
ChatGPT can “write code” in the most literal sense that it will give you code when given a prompt, but (unlike how OpenAI markets-by-omission) it does not reason about code like an AGI could.
Here's a recent example: https://simonwillison.net/2023/Aug/6/annotated-presentations...
btw, i think i have tended to see these pessimistic takes by people who have only used 3.5, which is impressive but not reliable -- i strongly recommend using 4 in the course of your job to see how useful it is. every day in the course of interacting with it, it produces novel lines of thought which shock me.
it really changes the flavor of it from cambridge analytica to dr evil.
Protagonist 1: "I don't get it, why are they doing this? What do they have to gain?"
Protagonist 2: "I don't know, unless... oh my god, it's regulatory capture!"
Protagonist 1: "Shit, of course! What will we do?"
Audience: Laughs at this shit being played so straight it comes off as a farce.
The usual morality play is either the evil business hurting people who has to be stopped by the benevolent government regulators, or the evil government regulators hurting innocent businesses who are just trying to make a living and keep prices down and keep jobs from moving offshore.
To which the traditional solutions are either more regulations that businesses don't like or fewer regulations that businesses don't like.
Regulatory capture is when the villains of both sides are working together, and the solution to it is fewer regulations that big businesses do like. But nobody wants to see their hero as a villain so anything that accurately portrays this is unpopular.
The other problem is this isn't how regulatory capture happens. OpenAI was flexing its badass credentials in the halls of Congress while it wrapped up a multibillion-dollar fundraise. There was no path from Altman's testimony to any sort of regulatory apparatus that wouldn't immediately constrain him.
Actual capture involves industry coöpting existing regulators through revolving doors and burdensome rules that reduce the threat of new entrants. It happens quietly. The only times industry proposes new frameworks in public is when they're cornered.
The "regulatory capture" playbook OpenAI purported to follow has zero actual precedent. It is only rhetorically precedented in crypto, where the same nonsense about creating a new regulator that would simultaneously be less-burdensome than the status quo yet also cement incumbents' advantages was pitched to the naïve.
> that would simultaneously be less-burdensome than the status quo yet also cement incumbents' advantages
But that's classic regulatory capture. The industry gets the government to impose regulations that are wasteful and bureaucratic but can be navigated by a large organization with economies of scale, which allows regulators to appear as if they're doing something, but the rules don't prevent the incumbents from making a mint by consolidating the industry and charging monopoly rents or avoiding taxes paid by the middle class and smaller competitors etc.
AT&T wanted a regulator so it could scale like the railroads. The regulator wasn't to regulate AT&T, if I remember correctly, it was to regulate the states. (It backfired in the long run.)
Broadly speaking, if you’re weak enough to need a regulator to guard your moat, you’re weak enough to be overpowered by the regulator. Regulatory capture, successful examples, at least, begin from a position of strength. Altman pitched doom and gloom and then, when the EU began reacting, seemed surprised that they weren’t permissive.
It was to consolidate the regulation so they could focus on capturing the one federal regulator instead of 50 different state ones, sure. But you do that so that when the public starts clamoring for something to be done, you get to control what gets done, and do something that benefits the incumbents and not new entrants.
> (It backfired in the long run.)
It seems to me that it's still pretty hard to start a competing telephone company.
I might be able to get some open source VoIP software running on a VM in one day. Now I want to offer WiFi calling plans. There is some domain name in the public DNS where phone numbers are subdomains that resolve to the IP address of its associated phone company so I know where to route calls, and I just go to some free government website to have some numbers routed to me so I can assign them to customers, right? No?
> Broadly speaking, if you’re weak enough to need a regulator to guard your moat, you’re weak enough to be overpowered by the regulator.
But the strength comes from the moat protected by regulations. The law inhibits competition, so you extract lots of rents from customers, so you have lots of money to influence regulators with. The regulators are the source of your power, but you and the rents you extract are the source of campaign funding, and they don't want to cut that off. Or worse, have it directed to the other party, if they can't actually bankrupt you before the next election cycle.
That is literally what he is doing.
People walk up to an "orb," get their eyeballs scanned, and a unique key goes into a database.
What part of the parent's statement is untrue?