Hacker News Reader

Comment by lumost | Hacker News Reader

To add a voice of skepticism. The recent rush to open source these models may be indicative that the tens of millions that’s spent training these things has relatively poor roi. There may be a hope that someone else figures out how to make these commercially useful.

View on HN

There are tons of commercial uses for these models. I've been experimenting with an app targeted toward language learners [1]. We use large language models to:

- Generate vocabulary - e.g. for biking: handlebars, pedals, shifters, etc

- Generate translation exercises for given topic a learner wants to learn about - e.g. I raised the seat on my bike

- Generate questions for the user - e.g. What are the different types of biking?

- Provide more fluent ways to say things - I went on my bike to the store -> I rode my bike to the store

- Provide explanations of the difference in meaning between two words

And we have fine tuned smaller models to do other thing like grammar correction, exercise grading, and embedded search.

These models are going to completely change the field of education in my opinion.

1) https://squidgies.app - be kind it's still a bit alpha

I started to work on something similar but way behind your project. I really believe AI models can help us as humans learn better! Do you have a blog or any other writeups on how you approached these problems?

How does the vocabulary generation work?

We're using these at where I work (large retail site) to help make filler text on generated articles. Think the summary blurb no one reads at the top. As for why we're writing these articles (we have a paid team that writes them too), the answer is SEO. This is probably the only thing I've seen done with a text model in production usage. I'm not 100% sure what model they're using.

Content made for machines. Probably a billion dollar industry.

Made by machines, for machines. It’s poetic.

There's a line in one of Douglas Adams' books where he says something along the lines that things like VCRs were invented to watch TV programs so that you don't have to.

Who would have thought that one man's joke would become a reality?

Content made for machines serving humans made by machines pretending to be human

Sorry but every part of that sounds so terrible.

Yeah I'm not a huge fan of it. I'll never forget the look in our UX person's eyes when she realized that our team doesn't exist to make customer's experience better (there's a ton of other teams for that) but to make Googlebot's experience better. Right now we're in the process of getting publishers you've heard of to write blurbs for best lists, but we're supplying the products so it's not really a best list.

I can't say I'm a big fan but my teams is great and I don't have time to look for a job right now.

You just know that some Amazon listings are written by GANs.

I hate this so much. These tools are getting better, so often you realise only half way through that you are reading AI text. Then you have to flush your brain and take a mental note, to never visit that site again.

I'm not a big fan either. At least the pages are just like:

- useless ai generated intro text

- ten products that actually are the best reviewed per category by users

- brief ai blurb on product

- 3 actual user reviews of the product

So even with the ai text there's still some benefit to the page.

HuggingFace will soon release their BigScience model: https://twitter.com/BigScienceLLM/status/1539941348656168961

"a 176 billion parameter transformer model that will be trained on roughly 300 billion words in 46 languages"

So anything smaller than that will become worthless. May be a factor, companies have a last chance to make a PR splash before it happens.

Not necessarily, only ~30% of the database is in English, so it likely won't be as good as a smaller model trained solely or mostly on English words.

https://bigscience.huggingface.co/blog/building-a-tb-scale-m...

It kinda seems like a model trained on multiple languages would to some extent be better at English than a model trained only on English? I mean so much of English comes from other languages, and understanding language as a concept transcends any specific language. Of course there are limits and it needs good English vocabulary and understanding, but I feel the extra languages would help rather than hinder English performance.

"worthless" huh, not everyone can afford inference of a ~500gb models, depending on the the speed/rate you need you might definitely go for smaller model

But maybe your sentence was more about "after BigScience model, open-sourcing anything smaller than that will be useless" which isn't necessarily true either, because there is still room to improve parameter efficiency, i.e. smaller models with comparabale performances

My guess is they're mostly vanity projects for large tech companies. While the models have some value, they also serve as interesting research projects and help them attract ML talent to work on more profitable models like ad-targeting.

They did not publish benchmarks about quality of the models, which is very suspicious.

I personally squinted hard when they said removing dropout improves training speed (which is in iterations per second), but said nothing about how it affects the performance (rate of mistakes in inference) of the trained model.

I agree that the lack of benchmarks makes it hard to determine how valuable this model is. But on the topic of dropout, dropout has been dropped for the pretraining stage of several other large models. Off the top of my head: GPT-J-6B, GPT-NeoX-20B, and T5-1.1/LM.

An equally plausible frame is that once a technology becomes replicated across several companies, it makes sense to open source it since the marginal competitive advantage are the possible resultant external network effects.

I don't know if that's the right way to think about the open sourcing of large language models. I just think we really can't read too much into such releases regarding their motivation.

Yes, commoditize your complements.

From what I've seen, using these huge models for inference at any kind of scale is expensive enough that it's difficult to find a business case that justifies the compute cost.

Yandex uses it for search and voice assistant

Those models aren't trained with the objective of being deployed in production. They are trained to be used as teachers during distillation into smaller models that fit the cost/latency requirements for whatever scenario those big companies have. That's where the real value is.

Maybe training it is not that expensive?

I know from practice that it takes a really really long time to train even a small nn (thousands of params) , so you'll need a lot more hardware to train one with billions... But, it's expensive to buy the hardware, not necessarily to use it. If you, for some reason, have a few hundred GPU lying around, it might be "cheap" to do the necessary training.

Now, that's not your point - cost != price. But, still...

I can't think of anyone having a few hundred GPUs around unless:

- They were into Ethereum mining and quit.

- They've already built a cluster with them (e.g. in an academic setting).

- They live in a datacenter.

- They are a total psychopath.

But even assuming one magically has all those GPUs available and ready to train, I don't want to calculate the power cost of it anyway. Unless one has access to free or extremely cheap electricity it would still be very expensive.

> If you, for some reason, have a few hundred GPU lying around

Not to nitpick, but that is like saying that if you have a Lamborghini lying around, a Sunday trip in one is not so expensive.

Only half-joking use case: active communities like this one on HN make sites attractive to human visitors. A new site could use bots to fake activity. Not sure it would work in the long run though.

Reply on news.ycombinator.com