My north star for the future of AI
theatlantic.com
theatlantic.com
You're part of a machine making billions off of exploiting human psychology to push products and get people to click ads. She's maybe not wrong about the issues with AI.. but there are way bigger problems and lower hanging fruit than vague biases in machine models. It's like worrying about if the coffee beans in Google's coffee machines are ethically sourced
At the end of the day, as exemplified by ad companies, people are gunna go make money ethically or not. If you could legally turn babies into cheeseburgers someone will do it. Engineers are not the right people to deal with ethics. That's what laws and regulations are for. You don't just hope and pray the engineers are nice people. That's a naiive way to operate society
I think that engineers (and everyone in every field) are the right people to deal with ethics. Pretty much everything anyone does has an ethical component to it, and it's a mistake to ignore that. It's an even bigger mistake to farm thinking about ethics out to specialists.
Laws and regulations are critically important as well, but they don't define ethics. They are informed by ethics. That's why it's false to think that if something is legal, that means it's ethical and vice versa.
Because ultimately people will be driven by the same thing that drove previous generations: money, power, fame, success, ego etc.
And no amount of university lectures with enlightened scientists and their superior values will change that.
Until that can be achieved, alternative options that generate more value will always be prioritised. Such is only the logical turn of events.
Does the primary responsibility for that really fall on engineers in the first place?
It just feels a little victim blame-y to me. Shouldn't most of the culpability fall on those stakeholders who don't want to be convinced because it means lower feature-velocity or less investor-profit?
I mean, do we blame food chemists for "failing to convince stakeholders" that the candy-company should use ethically sourced fair-trade chocolate and less sugar? Or mechanical engineers for "failing to convince stakeholders" that the oil-company should avoid drilling in areas that are environmentally-sensitive or controlled by warlords of questionable legitimacy?
No, we recognize that a laborer having specialized-skills doesn't automatically mean the corporate-machine is interested in their other opinions. (Unless they're visibly documenting problems that could someday harm the company in a potential lawsuit, in which case they tend to get laid off.)
b) CEOs typically will do whatever it takes to maximise revenue, past the point of what is ethical/moral, up until they hit the limit of whether it's illegal.
And then most but not all will stop. I'm hopeful but doubtful that AI will be any different.
"The paperclip maximiser was great up until it turned the solar system into paperclips.
Thankfully, a few of us were aware of the risk and escaped to a nearby solar system. It's not as nice as the first one, but it'll do.
Now, who wants to buy some paperclips? :)"
If picking the ethical choice maximized corporate profits, the Earth would have become a utopia a hundreds of years ago.
They generally don't care about the opinions of engineers that much, except to estimate the cost and practicality of different ideas.
And we've already let those AGI align themselves predominantly on "profit"
In their worry about a new AGI that they may never actually discover, the apocolypts are being stymied by the one that's already there and that's already working to take over the world.
There's a lot of conversation around this, one term used is "Egregore"
That's the definition being used in AI research. Does this system succeed on this test of {reasoning, theory of mind, abstract thinking, ... etc}? Can this system outperform most human workers?
And yes, corporations and other collectives can fully satisfy both criteria for most economically valuable tasks. If you make the same degree of input/output adaptations for them that you do for LLM's, they'll write you articulate letters all day, perform inhuman computations and reasoning tasks, develop and execute grand strategy, control all your robots, ace all your tests, and then write all your code for a tenth of what it would cost with your Bay Area tech bro.
From OpenAI's charter:
> OpenAI’s mission is to ensure that artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work—benefits all of humanity.
( This is the punny name that OpenAI employees use to refer to what's officially known as AI Existential Safety, explained here https://news.ycombinator.com/item?id=38374739#38375199 )
It's interesting I think because all of the above have ethics in my experience. What they don't do is treat their ethics with the same sense of supremism as the vocal activists/advocates who frame everything unaligned with their view critically. And if you show even a little compromise that too gets treated as a lack of ethics in that people will at the very least view it as the money winning out over personal beliefs.
The idea that someone can be legitimately ethical without being an extremist about the matter is largely unacknowledged by the vocal ethicists. But in a way that also makes sense because from their view the silence is probably just more room to criticize in the conversation.
I understand that she only worked at Google during a sabbatical from Stanford (Jan '17 to Sept '18) and that she's back at Stanford now. [1]
It's a little bit confusing with her writing in the present tense about being at Google but I think that's because this article is composed of excerpts from her recent book, so they're excerpting a bit from that time period.
This is something I've seen as well, in contexts much more remote from the nerve centers like Google or even Stanford. It's deeply disturbing.
I have no doubt Sam Altman thought he was in control until he was told to go…by humans…
Machines think faster than humans.
Simple math…
For a real world example of automated systems gone wrong check out the flash crash of 2010: https://en.m.wikipedia.org/wiki/2010_flash_crash
To be clear I don’t think the risk is “AI gone rogue” it’s either corporations gone rogue or powerful automated systems glitching and causing chaos.
Granted, there's much more at play than _just_ neurons.
I've been thinking a lot about tools, recently. About how much things should be designed to be predictable, narrow in scope, and simple. Predictable in particular meaning not only determinative, but also meaning with no edge cases (so they can be simplified as a mutation rather than having to think about the why of how they work).
The thinking behind going with my submission title :
> Fei-Fei Li wants AI reimagined from the ground up as a human-centered practice
was that the article title was too vague. The 'North Star' she is referencing is explained in the last paragraph of the article, which is what I used to compose my submission title :
> This, collectively, is the next North Star: reimagining AI from the ground up as a human-centered practice. I don’t see it as a change in the journey’s direction so much as a broadening of its scope. AI must become as committed to humanity as it’s always been to science. It should remain collaborative and deferential in the best academic tradition, but unafraid to confront the real world. Starlight, after all, is manifold. Its white glow, once unraveled, reveals every color that can be seen.