AI startup Scale lays off 20% of its staff | Hacker News Reader

AI startup Scale lays off 20% of its staff

scale.com

54 pointsbertdb18 comments

The letter frames recent advances like ChatGPT as positive indicators for Scale's future business. However, task specific bespoke human data annotation may fall away to datasets generated by massive self supervised multi-modal models. The other day, I saw that one of these image generation models can be prompted to create image-semantic mask pairs. And I wouldn't be surprised if ChatGPT could generate text along side named entity labels for specialized domains. The holy grail for AI has always been obviating the need for manual annotation and increased efficiency on the data we do have. Scale's data annotation business seems at odds with the core goals of AI researchers, and may dry up as those goals are increasingly realized.

Not only that, it's not clear to me that these models required the sort of annotation that Scale sells. He says:

>Each of the big AI breakthroughs (ChatGPT, GPT-3, DALL-E) have been made possible through extensive data collection and annotation infrastructure.

Maybe I'd count the RLHF stuff from ChatGPT as requiring a form of annotation. But like Dall-E is trained using existing captions/alt-text. No one is sitting in a room annotating images for Dall-E. Where would a company like Scale fit into this?

Yea… despite the massive amounts of data being fed into these models, the model quality is still data-bound. There is no way to produce models like GPT-3 with manually annotated data.

This is suddenly a highly debatable claim. Traditional ML was "data hungry" and required massive datasets for features and difficult to acquire "clean" labels.

You can ask ChatGPT to generate these datasets now. However it's unclear if future models will rely on such datasets if large models already have the answer. Here is an example of a ChatGPT generated dataset for a misinformation classifier. ChatGPT Can generate 10 examples every time I hit enter, I asked it to create this dataset using the format.

sentence, misinformation (1,0), notes

---

The earth is flat, 1, This claim is widely debunked by scientific evidence and research.

Vaccines are harmful and cause autism, 1, This claim has been disproven by multiple scientific studies and is not supported by medical evidence.

The Holocaust never happened, 1, This claim is a historical denial and disregards the overwhelming evidence of the genocide of Jewish people during World War II.

Humans and dinosaurs coexisted, 1, This claim is not supported by scientific evidence and is a popular conspiracy theory.

Water boils at 212 degrees Fahrenheit, 0, This is the standard boiling point of water at sea level.

The moon landing was faked, 1, This claim is a conspiracy theory and is not supported by any evidence.

Climate change is not caused by human activity, 1, This claim is not supported by the overwhelming majority of scientific evidence and research.

The sun revolves around the earth, 1, This claim was disproven by scientific evidence in the 16th century and is now considered a flat-earth theory.

HIV does not cause AIDS, 1, This claim is not supported by scientific evidence and has been disproven by multiple studies.

Vaccines are safe and effective, 0, This claim is supported by the majority of scientific evidence and research.

I was talking about ChatGPT itself. It could be made better with more data.

however that data may not come from human labels.

I interviewed with them a few months ago, got rejected, and then was contacted by the SAME RECRUITER to interview again a month and a half later. He'd forgotten who I was or that I interviewed. From what I could tell, they're a mess; this doesn't surprise me at all

This is the kind of things that can easily happen.I wouldn't judge them on it.

er… a month and a half later? Seriously?

That happened to me with Amazon and Microsoft. I asked Amazon to never email me again, and then I get a "got it, removing you from our list", and then the same recruiter emailed me saying I was a perfect fit.

Why do these so-called leaders break the news to everyone and in the same letter then start our imply that "the rest of us are going to do great". At a minimum, save that for a follow-up to just those who survived. And take the "we are hiring" taking off your LinkedIn profile and email footer!

It's usually the case when they abandon some high risk features, or they lay off low performers, and want to message that the company is doing well, just re-adjusting.

Pretty surprising news, I would have thought them to be positioned to do well from the current ML boom

I also am a bit surprised by this. AI is back to being the hotness now that crypto blew itself up. I suspect from reading this quote:

"Many of the industries we serve, such as e-commerce and consumer technology, have been buoyed by the pandemic and are now experiencing a painful market correction."

that they had an established segment of customers that were experimentally looking at ML to augment their core business, and how they could leverage with their data. Perhaps these companies are pulling back spend with whispers of a shaky 2023 economy.

When it seems like everybody else is doing layoffs, MBAs and wall street gets nervous if you don't do it

I interviewed with them recently, they've brought on lots of (expensive) new MLE type employees to provide ML SaaS services. Open-source might be putting pressure on them as these things become more commoditized

They are grifters taking advantage of boom and recruiting industry execs to head sales. They don't have any proprietary technology. The fact that private info was leaked in a fb channel by annotators in like south america shows that this company has little in the way of a meaningful product.

The irony is in their name "scale", laying off 20% would be ringing death bell for a company that actually operated leanly. They basically tried to copy mechanical turk

is.. ChatGPT writing all of these?

Reply on news.ycombinator.com