AWS will offer HF’s products to its customers and run its next LLM tool | Hacker News Reader

AWS will offer HF’s products to its customers and run its next LLM tool

bloomberg.com

148 pointsmmq86 comments

Open articleView on HN

HF = Hugging Face[1] they're the folks that are providing things like Stable Diffusion 2.1 Demo[2]

[1] https://huggingface.co [2] https://huggingface.co/spaces/stabilityai/stable-diffusion

thank you for this. maybe im nitpicky but wish titles would not use acronyms

I think it’s to do with the title character limits. I tried it out without the acronym and it’s a character or two long

oo thats a good, fair point. and common acronyms are whatever but these seemed very domain specific

conversly who else could AWS offer the products to?

if you know what HF and LLM meant in this context, i guess

Isn't AWS also an acronym? ;-)

No, it's an initialism.

No, it's a religion

No, it's a cult.

Yeah I had to check. For a minute I thought oh hum, Amazon will be selling Harbor Freight stuff online. But then went wait, that says AWS, not amazon.com. So I had to look.

Some things I'd like to see solved in this space:

    - Versioning. Don't change the LLM model behind my application's back. Always provide access to older versions.
    - Freedom. Allow me to take my business elsewhere, and run the same model at a different cloud provider.
    - Determinism. When called with the same random seed, always provide the same output.
    - Citation/attribution. Provide a list of sources on which the model was trained. I want to know what to expect, and I don't want to be part of an illegal operation.
    - Benchmarking. Show me what the model can and cannot do, and allow me to compare with other services.

IIUC versioning isn't really possible with a neural net, the training process influences the generation pathway

caveat: I have an incredibly superficial understanding of any of this.

LLMs are pretrained and then released for use.

So can have v1, v2 etc.

This is complete wrong.

A model is a binary artifact that can be versioned like any other binary asset.

It is definitely possible. At any point, you can just take a snapshot of the weights. Together with a description of the architecture, this is a complete description of a model.

All of these things are there right out of the box with the HuggingFace toolset.

(Determinism does depend more on the exact software running the model. In general it works now but there are occasional exceptions like PyTorch on M1 not being deterministic the first time you initialize it or something weird)

Determinism is largely impossible due to arbitrary ordering of GPU threads and non-associativity of floating point operations

Is this true for LLMs and not for at least Stable Diffusion? Stable Diffusion is largely deterministic, with the main issues mainly when switching between software or hardware versions of torch, GPU architectures, CUDA/CUDANN, etc.

Or perhaps I'm wrong about Stable Diffusion too?

I thought so too, but I run a stable diffusion service, and we see small differences between generations with the same seed and same hardware class on different machines with the same CUDA drivers running in parallel. It’s really close but there will be subtle differences, (that a downstream upscaler sometimes magnifies), and I haven’t had the time to debug/understand this.

Ah okay that makes sense. In my experience I've only noticed differences when the entire composition changes so I'm guessing it's near pixel level or something?

I assume they're the most noticeable with the ancestral samplers like euler a and the DPM2 a (and variants)?

I now think that "Hugging Face" refers to "the face one has when hugging" which, of course, is very nice.

But up until just now, today I thought it was an Alien(s) reference.

Honestly that's what I thought too.

Yeah unfortunately that was my impression when I saw the acronym expanded.

i as well.

i thought it was the principal behind the site. yiu can rip the face off of a model and retrain just the last bits.

that would make for a quite different branding.

Might think about it:)

for literally years I thought it was a tongue in cheek thing. The world to is now duller for the truth hath come and we found ourwelves wanting a grander fiction.

It's a reference to the emoji aka unicode character "HUGGING FACE" / U+1F917.

I also assumed it was a reference to "facehuggers" when I heard about the site mentioned, until I visited and saw the emoji displayed prominently on their webpage.

I'm not sure to congratulate or be scared for them but Clem, Julien, Thomas and team are the nicest and most helpful org in AI and should be given whatever they need to succeed!

Why scared for them?

Probably because Amazon

100% this.

I’m interested to see what strategy GCP will go with. Azure partnered with OpenAI and AWS partnered with HF. It seemed like from the Google blog post they are going to try to market their home grown AI tech through GCP but for many categories they have opted to partner or buy vendors to add more products to GCP.

Google has enough homegrown AI tech that they don't really need partnerships or acquisitions. Despite OpenAI getting outsized media coverage, Google still has a miles-long lead in the general area. Most of the advancements that made today's generative models possible came out of Google Research and Google Brain. Their real problem at the moment is productization and marketing.

I keep wondering if we're in the midst of Xerox PARC 2.0, but instead of Xerox, it's Googl. Amazing products and ahead of its time, but can't / won't execute.

Given Project Starline, this seems pretty accurate.

that... is just wild. stunning tech

It's also possible to be too early to market.

I'd say Google has played it just right. They clearly have a better idea of the readiness of the tech than Microsoft, who hastily released Bing's LLM to muted ridicule.

In terms of UI, ChatGPT is a glorified text box. The hard part is the basic research, which Google can grind away on in the background until the moment is right.

The problem with current LLMs is that they are inaccurate and untrustworthy. If LLMs were search engines then ChatGPT is the Altavista or Ask Jeeves. Google wants to be the, well, Google - come in later with tech that actually works.

> who hastily released Bing's LLM to muted ridicule.

Is it tho? I mean everyone is dying to try it. I’m beginning to feel like all this stuff was intentional to drum up free advertising.

Linus Tech Tips picked it up on the Wan show and were blown away to some extent.

Google self driving car : Tesla self driving :: Google AI : OpenAI AI.

Google is taking a very conservative approach, but their work is years ahead of groups getting far more press.

> who hastily released Bing's LLM to muted ridicule

…In the tech/hn echo chamber. People who have access that are not technical and that not try to trip it up seem to like it. Don’t think Bing has been used this much since its inception.

they wish to be seen as today's bell labs

Bullshit. None of it was ever released.

Until they actually release something and let the masses access their precious AI research there is no evidence they have anything at all.

Epic, complete blunder by Google.

Also, there's a huge difference between "it works for an ML benchmark" and "it works for real life use cases". OpenAI has done a phenomenal job with instruction tuned models, enabling fine tuning for any use case very very easily, and deploying it all at decent scale.

Pretty sure that in some cases, AI researchers in some companies are like: "sorry we don't give training set nor source-code (detailed instructions how to reproduce the experiment) but it works perfectly and is revolutionary, now give me my PhD / funding to our company"

Google translate was updated to uss transformers I believe. And snippits uses some kind of AI summarization.

Indeed, let's not forget that Google's BERT model was a very hot topic a few years ago, and their in-house researchers literally invented the basis of all modern language modeling, word2vec. Maybe they've been resting on their laurels, but with all the growing hype around GPT models (even before ChatGPT), I'd be surprised if nobody at Google was already working on this stuff.

They literally invented transformers. "Open" AI is just productising google's research.

Go take a look at the publication lists for any major AI conference and peep how many of the papers come from Google. For any year in the last 15.

That’s the point: no amount of academic papers can be a good ROI for Google’s investment.

It’s very nice of them to do foundational research for the industry at large, but that is not most people’s definition of business success.

Google is not releasing a product because there is no product to release*. Open AI isn't really releasing a product either, they are trying to win marketshare and gain funding.

*At the very least, not enough of a product to make a release sensible. If they launch Google GPT and it sucks, their product is dead. If Open AI doesn't release their "product," their company is dead and they lose funding and marketshare (that is, marketshare of a future market). Apple isn't releasing anything either; do you think they have nobody working on it? Microsoft is trying to use chatGPT in an existing product (still behind a waitlist) which is probably just to test the usefulness of thier $20 B investment in Open AI. I think Google's research speaks for itself, the lack of a product doesn't speak to anything.

I think you’re being too charitable to Google. This is the company that released Duplex, Orkut, and about 100 other not-a-products.

Google is quiet because they can’t figure out how to release something that is 1) useful, and 2) not fatal to their core business.

Google’s entire empire is built on users having to run multiple searches and click through multiple links to find anything. Any kind of summarization/knowledge system that reduces wasted user time is necessarily bad news for Google.

Google’s research here is admirable. Their lack of productization (remember, they also claim to be years ahead) is a symptom of business mistakes coming home to roost.

This isn’t true at all. Google built BERT which is an encoder widely used throughout industry.

> Their real problem at the moment is productization

Hasn't this always been a problem at Google?

Post-Gmail, what have they successfully taken from raw tech to successful product on their own? Hangouts/Duo/Meet? Chrome? (Although that latter they leveraged marketing pretty heavily)

They've had a helluva lot more success buying successful or nascent products, then developing the hell out of them into more successful products. E.g. YouTube, Android, Docs

None of these really contradict your point… but they have some ongoing shots, some near hits and some successful non-monetized products:

>Ongoing

It’s not a successful product yet, but there’s a good case that they’ve got a strong lead in self-driving cars.

Same with AI broadly, super recent sentiment notwithstanding… ongoing shot.

>Near hits

Stadia was a huge product marketing failure that proved the (very slowly) growing and succeeding cloud gaming model.

>Non-monetized

Go lang isn’t a “product” in the same sense, but it seems pretty successful.

Similar deal with Kubernetes.

Point is… productization + adoption from incubation is hard and they aren’t just totally flopping every time. Not sure any other big tech is doing way better?

The three best examples are a product that hasn't been proven yet, one of the biggest flops in recent gaming history (I would say the most notorious since the Dreamcast, probably), and a non-commercial project? If anything that reinforces the grandparent's point, I think.

Stadia didn't prove cloud gaming (there are many other efforts often predating Stadia), and its failure was much more a business strategy failure than a marketing failure.

Stadia failed because even in the launch thread people said they wouldn't buy because they assumed it would be shut down in a few years, which became true. Google is in a self-fulling prophecy situation for most of their stuff because nobody will build on their stuff because of all the stuff they kill and they kill stuff because nobody will use it

Chrome, Kubernetes, Google Home, Waymo

Although it's a good point for sure

There’s nothing anti-product about using marketing to increase product adoption. If Chrome was slower or crappier (at launch), marketing wasn’t going to save it.

I'd point to IE's market share while it was obviously technically inferior. Most people don't want to think about "Which browser?", sadly. Thus, marketing or default installs win.

Google’s Bard is worse than ChatGPT, though.

Why do you think that?

An internal google employee told me this

genuine question: isn't ChatGPT essentially applied huge markov models at google compute and dataset scale?

It’s not a Markov model, but it is autorrgressive.

Specify lead please. I have been using SD, HF, ControlNet, GPT3 for a long while now and have never encountered any "must have" Google Product.

They aren’t miles ahead. That’s a myth. That’s why their stock dropped so much.

> Google has enough homegrown AI tech that they don't really need partnerships or acquisitions.

I don't doubt this, but there was a HUGE marketing/news angle MS+OpenAI and big name recognition in the space for Hugging Face.

If it's about deep integration of LLM into products, I'm sure Google has been prioritizing that for awhile. If it's about making a splash with their own thing in their cloud offering, it lands a little softer than Microsoft's or AWS' news.

You can still use Huggingface in GCP. It’s an open source library with open source models. A lot of my Google Colab notebooks have been just messing around with HF models.

I believe GCP cut deals with Cohere.

Google invested over $400mm in Anthropic; not sure how people missed this. That's >10% stake in an OpenAI competitor and a clear bet on something.

Additionally, for those not in the know, Anthropic was founded by some pretty senior ex-OpenAI folks, presumably carrying over a lot of the same culture and technology. It's as close to a copy-cat investment as one could get.

Anthropic. https://www.ft.com/content/583ead66-467c-4bd5-84d0-ed5df7b5b...

I was hoping this was about Harbor Freight

imagine how one of their s3 in a box for copying digitized data from tapes would be if harbor freight got their hands on it. $100 and it makes that sound that failing hard drives did in the 90s

Don't forget to bring your 20% off coupon when buying ec2 instances.

I couldn’t decide between Harbor Freight and hydrofluoric acid.

I hope this means that Alexa and Siri will understand everyday speech better, since amazon tends to dog food their services and I would assume that Apple would want to keep up. As of right now, it is annoying asking anything beyond turning on / off lights, playing specific key words like news or music, or the weather. It’s like a command line but with voice.

Google is much better with questions pre-chat GPT, but their home integration has been broken post Nest debacle.

Will HuggingFace get paid or is it alongside other AWS "partnerships" where the authors will get their product taken for free, forked and maintained by AWS?

"run its next LLM tool"

Huggingface was already using S3 storage before.

I think it means GPU inference capabilities.

They were also already renting their GPUs on EC2 before.

Non-paywalled link: https://huggingface.co/blog/aws-partnership

They should create some next-gen VR goggles that use HF tech to create your wildest dreams. Call it "FaceHugger".

Great move by AWS. Given how they work I bet that there was a document presented to Andy within a couple days of the Microsoft announcement.

Google’s complete failure continues.

Reply on news.ycombinator.com