Hacker News Reader

Comment by capableweb | Hacker News Reader

Unless I missed something, it seems they didn't share which models by parameter count they're gonna be offering, nor anything at all about cost.

Also, this strikes me as really weird phrasing:

> Llama 2 now available for global usage on Cloudflare’s serverless platform, providing privacy-first, local inference to all

"privacy-first & local inference" would be to run it locally, on your own hardware, isn't that exactly what should be referred by when using "local"? Or has the definitions of words completely gone out the window as of late?

View on HN

This is just a press release, and not a great source of in-depth info. This blog post from yesterday goes into it: https://blog.cloudflare.com/workers-ai/

>Models you know and love

>We’re launching with a curated set of popular, open source models, that cover a wide range of inference tasks:

>Text generation (large language model): meta/llama-2-7b-chat-int8

>Automatic speech recognition (ASR): openai/whisper

>Translation: meta/m2m100-1.2

>Text classification: huggingface/distilbert-sst-2-int8

>Image classification: microsoft/resnet-50

>Embeddings: baai/bge-base-en-v1.5

I missed that blog post somehow, thanks for sharing that. Bit disappointing that it's just the 7B model, it's a good starting point for fine tuning another small model, but it really isn't useful on its own.

> This is just a press release, and not a great source of in-depth info

Not sure I'd call outright lying/getting the most basic points wrong "not a great source of in-depth info", like the "local-first" part.

> Or has the definitions of words completely gone out the window as of late?

I think it means local as in "nearby" - so if you're in the UK it isn't processed in us-east-1, for example. You can pick one local to you.

Yeah, I think so too, but I guess I'm more upset that they use "local" in a "remote but physically nearby" fashion when it usually refers to "this computer/device".

Yeah - I think it's "local" as in "localisation". It is a little overloaded.

At least you get privacy from Meta.

Considering this is a collaboration, don't you think the agreement between them has something about feeding data back to Meta?

I just think it's loading the models and revenue sharing.

It would be similar as their partnership with Huggin Face, I suppose.

What data would be useful to capture? Outside of a human feedback loop, I don't think there is an actual use-case for it.

Note: not certain

Without anyone of us actually knowing the details of the collaboration, all we can do is guess, I guess.

> What data would be useful to capture?

Pipe the prompts straight to Meta and I'm sure they'll be able to extract a ton of useful data.

What's the use-case without any human feedback, which I already mentioned?

I don't work at Facebook, so I'm not gonna spend my time doing their work for them or you.

But off the top of my head, classifying the data into various categories and mentions of Facebook/Meta could allow them to derive sentiment about the company based on geographical location, and know where to invest more in changing the sentiment.

Even if this would be a valid suggestion ( I don't think so).

This would be incredible hard to do without IP.

Cloudflare won't forward it and it can be setup as an API. Additionally, IP's on mobile are re-used across users.

That doesn't even sound remotely to a valid indicator for "investments". Definitely not considering the costs to create and update such a model.

100% marketing speak.

Reply on news.ycombinator.com