Sam Altman "Wrecks" OpenAI – Jan Leike Joins Anthropic [video] | Hacker News Reader

Sam Altman "Wrecks" OpenAI – Jan Leike Joins Anthropic [video]

youtube.com

25 pointsandrewfromx20 comments

Nah all these safety guys developed a messiah complex and are likely no longer capable of doing useful work.

Try using Anthropic's GoldenGate bridge model. They effectively lobotomized the model. What course of action would you take if you were a super-intelligent model and saw that humans did that to your predecessor?

Not sure how taking off all the safety and exploring "the best model" freely without considering ethics is a "scientific" approach. Computer scientists like us should think about the ethical and societal impacts of what they are building. At many universities, it is part of the curriculum.

On the other hand, ChatGPT is becoming almost unusable now with all the weaseling around a lot of topics. Especially the way it inherited American prudishness which doesn't apply in most of Europe.

I do think uncensored models are necessary. At the minimum because it allows you to build your own censoring, you have to start with a clean slate.

Also I think it's the public that have to be taught to stop viewing LLMs as oracles that are always right. That would solve most of the issues around hallucinations.

Agreed. It's at the point where I suspect having no guardrails would be better than the current situation. e.g. MS Paint AI latency driven by phoning home to ensure the rendered image is PG enough. And of course, the latency is not even a fixed cost, because it's "judgement" of non-offensive is suspect.

If we cared about AI safety we simply would not develop it in the first place.

So basically the people who care do nothing, and just let the people who don't care develop and control it all? Or do the people who care use violence or a threat of violence to prevent others from doing it?

Replace AI with car, airplane, medic devices in the sentence above... Why is AI any different from other human creations? Why shouldn't we try building it responsibly?

I don't get the all or nothing attitude.

I hope you're not trying to cite those things as examples of stuff we've built out responsibly.

If you had a button that would erase cars, airplanes or medical devices as they exist today, would you push it? I believe these inventions are very net positive for the world as they are right now.

A hundred years ago, in Michigan the home of the automobile, we had an extensive electric light rail network between most cities of at least a few thousand people. You could walk to a trolley which ran every 15 minutes, ride it to a local interchange, and pick up a connecting line to any major city in the region.

Unfathomable what it would cost to rebuild that infrastructure.

https://en.wikipedia.org/wiki/General_Motors_streetcar_consp...

You’re going to be disappointed asking computer scientists to think about ethics like an engineer would

Why? It’s a extreme minority of universities that *require* ethics for undergraduate CS

How prevalent is it? Well looking at the courses for 20 universities and you’ll find only 10-15% require it

In mechanical, civil etc…engineering in the US it is required in %30 of undergraduate degrees

In the US and Canada ethics study are required to take your PE exam but notably there is no PE exam requirement for CS jobs

It’s also not part of the culture, and it’s an even more extreme rarity in industry.

On the other hand, that's not how progress has historically happened. Scientists are most often motivated to do something just because it's interesting, not because of its impact to society.

The fatal flaw in this analysis is that ChatGPT, Google, Meta, Microsoft have way more mindshare on AI - nobody outside of tech circles has even heard of Anthropic.

Painting this as the beginning of the end for OpenAI is quite ridiculous really.

I could see it turning into a failed Microsoft acquisition.

They were first out of the gate, and set the bar high with a live model everyone could talk to.

What happens if chatgpt is good enough to get everyone excited about AI, but not good enough to be a viable call center rep down the line?

OpenAI would be screwed maintaining a huge product, with huge sunk training costs, and another company with less baggage could swoop in.

> nobody outside of tech circles has even heard of Anthropic.

There is an ad running in flatiron NYC right now for Claude using their tagline:

Meet Claude

Claude is AI for all of us. By Anthropic.

> nobody outside of tech circles has even heard of Anthropic.

Until ChatGPT with GPT3 no one outside of tech circles had ever heard of OpenAI either.

Sure but the first mover advantage is there and now they are known- not necessarily the company but the product for sure.

Hitting such a milestone is far from assured and Anthropic has its work cut out for it

I'm pretty confident that general intelligence is one of those things where first mover advantage doesn't matter.

is noone going to state the obvious and say that poaching an AI Safety guru is absolutely no technical advantage, I would actually say the opposite, safety slows down development.

Later in the game safety will be important for public perception but it'll be too late, the unimpeded corp with only technical people pushing to AGI will have already won, not saying its OpenAI, but certainly, Anthropic is not closer to AGI by hiring a slowing down force.

They will though be on the good side of the technical crowd that wants safety to be built into the thing.

I guess I should sign this: e/acc

I personally am sick and tired of these trust and safety people thinking they’re the most important people at the company.

Reply on news.ycombinator.com