Hacker News Reader

Comment by mvdtnz | Hacker News Reader

Let's see the products. I'm so tired of hearing Google talk a big game about AI without delivering.

a) I think they have in a lot of ways, that are maybe not super flashy. Vertex AI offers more API/functionality/product than anyone else, by a mile. I think in a year or two this might be more important than the quality of model (not that the model won't have to be good, but all of them will be good enough to solve most peoples business problems)

b) Gemini 1.5 Pro public preview with 1 mio tokens is out now (including API access). Has anyone benchmarked it?

> not that the model won't have to be good, but all of them will be good enough to solve most peoples business problems

In a year or two? Mate I think you need to step back from the AI hype for a moment. We're not even close.

When I write that tooling will be more important in the future, I am not making any crazy claims about the future of AI. My claim is: It does not require much more AI than we currently have to solve most business problems. But it will take better tooling.

I don't think AGI is even remotely required to solve most peoples business problems. Better tooling is.

Maybe we also have different ideas about how mundane most peoples business problems are.

Having tried all 3 (Gemini, chatgpt and Claude) I'll agree with GP that 1-2 years seems conservative if anything.

I don't think AI will be won on the model but the tooling. Google is arguably ahead.

The models I'd say it's a toss. But it doesn't matter as before the year is up all of them will release new versions one-uping each other. It's a race to the bottom. They either collude on price or most will go bankrupt.

Edit:typo

#1) That's absolutely, harrowingly, shockingly, untrue. They're just rolling out system messages! I actively integrate with Mistral, OpenAI, Gemini AI API, Vertex AI API, and Anthropic's API and Google's are the weakest.

#2) It's Gemini 1.0 with a 1M context. It's not very good, it works like OpenAI long context for me (pulls out salient text, but it doesn't seem to be 'reasoning' over all of it)

It’s almost like they hire exclusively for your ability to leetcode rather than trying to hire people who can produce software

Maybe the ai teams were too busy keeping up to date with their tree reversals to read those textbooks on their nights off

Gemini is just as good if not better than GPT-4?

Extraordinary claims require extraordinary evidence... :-))

You can use the product yourself or check one of the many benchmarking papers

Benchmarking papers are borderline useless. Goodhart’s law is a thing when a measure becomes a target it ceases to be a good measure. That’s why I keep coming back to GPT4. It has the most “magic” and “mind reading ability”. Try to quantify that.

Reply on news.ycombinator.com