Google announces Firebase Genkit with Ollama support | Hacker News Reader

Google announces Firebase Genkit with Ollama support

ollama.com

102 pointstosh31 comments

The preference for Ollama over llama.cpp is almost certainly due to having a better user experience. Hacker News is in an endless cycle of not understanding how important user experience is, even often more important than the underlying technology itself. Setting up and using Ollama is much easier than setting up something like llama.cpp even for the more technically oriented.

Why ollama, why not llamacpp? The real effort is done by llamacpp, ollama is unnecessary layer on top of it and cashing in investor money.

This behavior is totally unacceptable and I am boycotting ollama and every ollama based stuff. I am not going to use ollama in my client projects anymore too.

What's funny is that ollama just runs the llama.cpp example http server on a random port under the hood and proxies requests to it, it doesn't even link against llama.cpp

Yeah they intrusively listen on a port. It is security redflag.

Yep, and the only acknowledgement to llama.cpp is that llama.cpp is a "supported backend" (see last page of their GitHub repo readme)

And the acknowledgement was only added after community outcry. If not for that they would have kept on as if llama.cpp didn't exist.

Implying ollama has other back ends which it does not.

The Genkit framework just launched and it has open plugin API. You can already find support for groq, openai, anthropic... I don't think the answer to your question ("Why ollama, why not llamacpp?") really matters (maybe only out of curiosity), but practically if you want llamacpp support someone will probably add it very shortly... It's open-source.

Waiting until ollama announces that they are raising money through VCs.

Just like Keybase, Bitwarden, Element, ggml also did.

We as a community need to rise up awareness that llamacpp can be used as easily as ollama. Another thing is ollama models are hosted on non-open infra service.

Funny to compare the signal to noise ratio on this blog vs the linked one to genkit https://firebase.google.com/docs/genkit/plugins/ollama A giant beta warning header and cookies banner on mobile. I’m seeing about one line of actual relevant info before scrolling

I haven't been following closely, but I had thought firebase was like a streaming database or something. But I guess not if it also runs AI models. Is "Firebase" more of a brand, then, nowadays that encompasses a family of products? Or am I misunderstanding something?

Yep, it's Google's brand of developer focused cloud mostly around backend for apps. It has a database (Firebase Realtime Database), functions (FaaS), hosting (static website hosting, now with dynamic node.js), auth, etc. etc.

Didn’t llama have a 700 million user clause that was widely interpreted to specifically exclude the FAANGs? How does that figure into this official Google release?

It’s not llama, it’s ollama, a way for you to run various LLMs locally

Thanks to Genkit's open plugin ecosystem, there's not only support for Ollama but also:

- OpenAI - Groq - Anthropic - Cohere - Mistral

Check it out: https://github.com/TheFireCo/genkit-plugins

Let's count how many days until Google announces that it will be shutdown?

so thaaats how ollama is going to make money

They’re not gonna make money. It’s just a llamacpp wrapper so functional zero moat

They definitely gonna get funding, at least getting popular due to FAANG using it

somewhat amused by the interchangeable use of “make money” and “get funding” like that. Very SV.

I am bias but even if I wasn't I would not touch Genkit. It is interesting but in the hands of Google I feel that I will be left in the cold after an year or two.

Besides, I am not sure what Genkit solves that cannot be done with an external provider. Can someone comment who is close to this technology?

(BTW, "bias" is a noun, and "biased" is the adjective. It's like the difference between "I am prejudice" and "I am prejudiced.")

Isn't this basically a gpu backed vm running ollama, only managed? Nothing some terraform/opentofu and a dockerfile can't replace.

No?

Genkit appears to be an open source framework. It can connect to various LLM backends. This appears to be a plugin for connecting via that framework to a local Ollama instance. Nowhere is there a suggestion that they're providing managed Ollama VMs.

yet

    > Besides, I am not sure what Genkit solves that cannot be done with an external provider. Can someone comment who is close to this technology?

I use Firebase extensively for my side projects and worked at one YC startup that initially built on top of Firebase.

To answer your question, what Firebase does particularly well is provide a really well-integrated full-stack development experience; one of the best ones, IMO. It's similar to Amplify on AWS which really just binds together multiple AWS backend services, but I find Firebase more pleasant to work with compared to Amplify. I would also make the case that it has some similarity to Azure Functions which skews more PaaS in its depth of integration than GCP Cloud Functions or AWS Lambda.

There's no reason any team couldn't just integrate with OpenAI or Llama3 on Together.ai or Fireworks.ai or run their own instance, but the DX on the Firebase platform is designed for teams to move fast and looking at the docs, I can appreciate the work the team put into making the experience feel seamless and reduce a lot of the repetitive code structures teams would otherwise have to build themselves to leverage the AI APIs.

Aside: I think Firebase is probably one of the best products out there for indie and weekend hackers building projects. The full-stack integration and local emulator are so well done and promote building with speed while still being scalable. The free credits are really generous and I run a ton of workloads in GCP for pennies. For some work, I'll drop into containers in Cloud Run and it still feels really easy to tie it all together. This is in contrast to working with Amplify, for example, which feels very "heavy" to begin with and then clunky as soon as you need to drop out into the underlying AWS infrastructure (ostensibly because CloudFormation is itself quite clunky).

I do a lot of solo building and Firebase lets me focus on just shipping stuff. In the last year alone, I've built at least 4 fully functioning products on Firebase (https://turas.app, https://coderev.app, https://usemeld.com, https://zeeq.ai); some with hundreds of users. It's a really productive platform.

Is Firebase at danger of being retired to the Google graveyard? I don't think so because fundamentally, Firebase is a set of wrappers around fundamental Google infrastructure (the same way Amplify wraps Lambda, DynamoDB, S3, etc.) so it wouldn't make sense to axe Firebase since it's just wrapping Cloud Functions, Firestore, Cloud Storage, etc.); it makes those services more accessible for application development. I could see them axing specific features of Firebase (which they have), but the core features that wrap infrastructure aren't at risk, IMO.

Reply on news.ycombinator.com