Hacker News Reader

Hacker News

Comment by blip54321 | Hacker News Reader

Parent Full thread

blip54321

On the ethics front:

* Yandex released everything as full open

* Facebook released open with restrictions

* OpenAI is completely non-transparent, and to add insult to injury, is trying to sell my own code back to me.

It seems like OpenAI has outlived its founding purpose, and is now a get-rich-quick scheme.

What I really want is a way to run these on a normal GPU, not one with 200GB of RAM. I'm okay with sloooow execution.

View on HN

Have you looked into HuggingFace Accelerate? People have supposedly been able to make the tradeoff with that. Although you still need to download the huge models.

Can confirm. HuggingFace Accelerate's big model feature[1] has some limits, but it does work. I used it to run a 40GB model on a system with just 20GB of free RAM and a 10GB GPU.

All I had to do was prepare the weights in the format Accelerate understands, then load the model with Accelerate. After that, all the rest of the model code worked without any changes.

But it is incredibly slow. A 20 billion parameter model took about a half hour to respond to a prompt and generate 100 tokens. A 175 billion parameter model like Facebook's would probably take hours.

1: https://huggingface.co/docs/accelerate/big_modeling

Thank you for the pointer. I've been poking at it with a fork for the past few hours, and realized I forgot to respond.

I don't understand why OpenAI has so many restrictions on its API. Isn't things like erotic writing, unlabelled marketing etc. good money for them with minimal chances of litigation? Is it for PR?

It's because it was genuinely founded as an organization worried about misaligned AI.

The critique is that the type of ethics they concern themselves with is borderline moral-panic/Victorian era. Not the Laws of Robotics kind of stuff.

Maybe it's my personality but I get the impression since AI is rather limited in 2022 that all the paid AI ethicists spending 90% of the time on bullshit problems because there aren't many real threats. And these gets amplified because the news is always looking for a FUD angle with every AI story.

The priority seems to be protecting random peoples feelings from hypothetical scenarios they invent, when IRL they are releasing research tools on a long-term R&D timeline... GPT-3 isn't a consumer product they are releasing. It's a baby step on a long road to something way bigger. Crippling that progress because of some hyper-sensitivty to people who get offended easily seems ridiculous to me.

> I get the impression since AI is rather limited in 2022 that all the paid AI ethicists spending 90% of the time on bullshit problems because there aren't many real threats. And these gets amplified because the news is always looking for a FUD angle with every AI story.

I think we’re about due for an AI-ethics winter.

Also, it's pointless. OpenAI might be a leader right now but it won't be forever. It can't control a technology. It's like restricting fire because it can burn down houses... yeah it can, but good look with that, all we need is some friction or flint. As time goes on that flint will become easier to find.

If OpenAI wants to concern itself with the ethics of machine learning, why not develop tools to fight misuse?

There are more than enough unaddressed ethics issues in ML/DS from racial bias in criminal sentencing to de-anonymization of weights to keep ethicists busy without needing Skynet.

Seems like that time would be better spent working for local justice orgs and ACLU than blocking OpenAI/Google from releasing chatbots or image generator because they fear someone might voluntarily type in some wrongthink words into input box and blame them for letting it happen.

That already exists depending on your definition of slow. Just get a big ssd, use it as swap and run the model on cpu.

A comment below said this model uses fp16 (half-precision). If so, it won't easily run on CPU because PyTorch doesn't have good support for fp16 on CPU.

Parent never claimed it was going to be fast.

It would probably just fail with an error "[some function] not implemented for 'Half'"

fp16 models inference just fine in fp32, though I was sorta joking in my original comment, it would potentially take weeks for this to run one input. You're better off trying to make something like huggingface accelerate work (like the comment above), which swaps layers of the model on and off the disk

On the ethics front Yandex should provide more details on the data they’ve used.

I don't see giving spammers, marketers and scammers more powerful tools as the ethical stance.

That’s an understandable view point. However, “Security through obscurity” just doesn’t work. Worse, trying to keep something from people really only punishes/limits the rule followers.

The bad guys get it anyway so this gives the good guys a chance.

I am curious what is the reasoning behind "giving "good guys" access to language models will {deus ex machina} and thus allow us to prevent the spam and abuse".

Automated tools to distinguish AI generated text from human writing and hide the AI spam.

This ^^ + many other mitigation/analytics use cases.

Can humans be trained en masse to output less distinguishable text from those of NN?

There's not much obscurity here. If you have tens of millions of dollars to throw at compute and a bunch of PhDs you could develop similar tech. I don't understand the idea that ethics somehow requires existing private models to be made available to everybody.

Yeah I was responding to a post asking why we should allow open access, given that some of those with access will do bad things.

I agree with you. Ethics doesn't demand that existing private tech be made available. Who's saying that??

OpenAI is just catching shade because their initial founding mission was to democratize access to AI tech and they've gone pretty far the other way.

Almost certainly they are getting it, OpenAI will just get paid for it.

Better take away the internet then

They wont, but the cat is out of the bag. It is data, and data gets leaked, shared in the open, shared in the dark. Researchers can be bribed.

It is like: you can not talk to your kids about drugs and pretend they don't exist ... or you can.

Reply on news.ycombinator.com