2023: A year of groundbreaking advances in AI and computing | Hacker News Reader

2023: A year of groundbreaking advances in AI and computing

blog.research.google

62 pointsjithinraj45 comments

This has been kind of a bad year for Google AI

Feb - Bard Launched with LaMDA -- Immediately replaced by Palm 2 backend within a month.

May - Launched Palm2 with 4 different sizes, 2 of which were unusable (poor Gecko) and 1 of which was unavailable (unicorn being invite-only/ask your google rep), which replaced Palm-540. Will be replaced by Gemini Ultra apparently next year as "Bard Advanced"

Launched Duet AI for both Workspace and Cloud, still compares unfavorably to GH Copilot.

SGE basically caught up to ChatGPT web search with Bing

June - Imagen released, results were ok, and immediately superseded by DallE3 within 3 months.

November - Youtube releases AI Music generation, but only as part of their Music AI Incubator.

Gemini launches with the infamous blue duck video.

Gemini Ultra will likely be gatekept just like Palm2-Unicorn despite performance only matching GPT4/Turbo (and despite delaying their official launch by a quarter).

They recently released Imagen 2, but now without any sort of benchmark or other academic details.

https://deepmind.google/technologies/imagen-2/

If those numbers were any good you know they would be all over the press with them.

Bard is a really stupid name. I get the origin of the name, and it's a short word, and in some sense it's a nice vibe with implications of a mystical poet or whatever. But it's a word that most people don't use.

As many have commented, ChatGPT is a pretty bad name too -- people get the order of the letters wrong all the time -- but at least it starts with "Chat", which explains what you're supposed to do with it.

Bard sounds way too close to "bad" or "beard" depending on accent. It's a terrible name.

I always associate bard with medieval mid looking old guy playing the lute

It's a stupid name like Google or iPad was a stupid name until it's worth a billion dollars and then it's not so stupid anymore. Success makes the name, not the other way around.

A lot of this stuff was actually cool...but OpenAI got there first.

By the time LaMDA came out, we already had ChatGPT. Palm2 was a serious ChatGPT competitor, but by that point, we had GPT4. Imagen was trained in mid 2022 (!) and could do text and everything, but they locked it away until SD and Midjourney had claimed the market. When it finally came out, nobody found it interesting. And now we have Gemini, which appears to be equal or slightly worse than GPT4, which itself was trained in mid 2022.

They just don't ship things fast enough.

Chickens, home, roost. After all the skuffed products, the destroyed ecosystems, and the general plague Google has become even in their core search product, I hope their gigantic carcass feeds a new generation of hungry companies eager to succeed by making the world better.

Google with some of the best people in the world. Unlimited resources in almost every way, manages to fall further behind and further into irrelevance in AI.

A bunch of closed models no one can run. A mediocre bard that is way behind everyone else. Pretty pathetic. Is anyone home at Google?

Google's AI budget is higher than the total funding for AI at all US universities combined. It's just mind boggling how they can drop the ball so badly.

What a change in fortunes. Just a few years ago it was the place to be and our PhD students were dying to work there.

Well, Google has a bunch of people who tell themselves they’re the best in the world. I’ve known perfectly reasonable humans start working there and by the time the left they felt they were smarter. By that I mean, in general, smarter than others. You can’t be smarter, you can be smart, but smarter isn’t a state of being. But they definitely feel they are, and assume you know that too. When discussing work I had done with one they insisted I must have based it off of a Google paper, which they weren’t sure if there was one on it, but surely I didn’t do it myself. That level.

One when confronted with the actual remit of their new job at a new company said “wow you hired an atom bomb to open a tin can.” They failed to achieve anything in two years and was fired.

Are you suggesting Google is a narcissist incubator? (only slightly tongue in cheek)

If you’ve met Eric Schmidt you will know that is precisely what it is. His narcissism is boundless and he apparently figured out how to make it infectious.

I'm afraid I'll have to call bullshit on your story, as firing an atom bomb is against the Geneva convention.

It's nice to see this. Hopefully small upstarts will make breakthroughs in different areas. Better than an AI monopoly.

This last decade, Google hasn't really made much of relevance outside AI either. The company is just milking its existing ad flows and not much else, it seems.

Google probably has at least 5x the management bloat as the competitors and it's killing them. Too many cooks in the kitchen vying to be the next AI thought leader. They're acting like IBM to be honest.

Gemini Pro, with arguably comparable quality to GPT4, competes with GPT4 with its superior pricing and speed.

If you actually use Gemini Pro, you will find that it gives nearly instantaneous full responses, which makes me use it more than even GPT4 Turbo for a lot of quick questions. And certainly more than Bing AI with GPT4, which is so slow and awkward to use and which I gave up on.

It's something that doesn't get discussed: speed and pricing, with near top quality. That's maybe how Google is planning to win for now.

Google's own benchmarking shows that Gemini Pro is just slightly better than GPT 3.5 and Gemini Ultra is comparable to GPT 4 (see their technical paper).

Google doesn’t really have much of a track record in real breakthroughs, does it?

It was built on its founders’ initial application of an eigenvector centrality algorithm to web search. The main innovation there was the application, not the algorithm - it was innovation more than invention.

Since then, perhaps the most technically impressive thing I’m aware of that Google has actually productized is their language translation system. But again, that was primarily innovation that built on existing technology and access to a lot of data, hardware, and distribution capability.

> Just a few years ago it was the place to be and our PhD students were dying to work there.

Wasn’t that mainly because of money and the collection of smart people you mention, bought by that money? Which is not the same thing as a record of achievement.

I admit I could be wrong, I’m open to being corrected.

I believe Google Maps invented AJAX. Google Docs I think was actually acquired, but it was definitely ahead of its time.

Microsoft invented Ajax/XMLHttpRequest for Outlook Web Access.

Microsoft invented XHR for use in Outlook Web Access. The Gmail/GMaps teams developed that technology into AJAX.

This makes it seem like they invented XHR and did nothing with it, but Outlook Web Access already used XHR for dynamic/asynchronous update of an email web interface context four years before Gmail.

If your claim is that Google popularized the ideas behind Ajax, then I'll happily agree: The term Ajax was invented outside Google [0] to describe Gmail front-end inner workings. My first contact with Ajax-like technologies was certainly Gmail in 2004 and not OWA in 2000, so I am not denying the merits of popularization. But I think even saying Google "developed the technology into" Ajax is unfair to OWA, given that Microsoft was pushing the other components of Ajax for a lot longer as DHTML (which led to standardizing DOM [1]; document.all anyone?) and OWA used DHTML+XHR, which was at the time essentially a non-cross-browser version of Ajax (props to Mozilla for implementing a compatible version of XHR and shipping it as part of Mozilla 1.0 in 2002).

I won't argue for Microsoft any further, though, because the "cross-browser" part is key. The fact that they invented and shipped these things in IE with no effort to standardize them (as part of their embrace, extend, extinguish approach) was a big headache for cross-browser compatibility. Google gets most of the credit for Ajax because web developers hated Microsoft's approach to the web at the time (for very good reason). It is interesting to compare Microsoft's behavior back then with Google's current approach of flooding standards bodies with (at times half-baked) proposals.

[0] https://web.archive.org/web/20150910072359/http://adaptivepa... [1] https://www.w3.org/DOM/faq.html#DHTML-DOM

The maps UX (zoomable infinite scrolling map) was implemented independently previously to maps, which was originally built by an australian start up (?) iirc before being acquired by google.

The term "AJAX" was coined around 2005 though. Before XMLHttpRequest there were iframe hacks. Most early stuff didn't really work due to browser incompatibilities and bad security.

Nope, I was using XmlHttpRequest back in 2003 (through ActiveX) for some intranet where clients used IE5. Might have popularised the concept but the technique predates GMaps for at least 3-4 years.

Aside from the fact that Google didn’t invent AJAX, it was trivial technology. It wasn’t some kind of amazing breakthrough, it was simply a workaround for explicitly designed limitations on prior technology. What AJAX did could trivially be done by any programmer with the ability to write networking code. It’s just that the browser environment placed artificial constraints on that.

But anyway, Google didn’t even invent AJAX.

Something something transformers

I’m not familiar with what you’re referring to, but I’d guess it has to do with some academic breakthrough in transformers. But that wasn’t something they productized, was it?

The comment I replied to seemed to be lamenting Google falling behind in the market, talking about closed models and Bard. My point is that Google doesn’t have much of a track record of homegrown invention that they’ve successfully brought to market.

That doesn’t necessarily mean they don’t have good R&D, but that can be a lot like a university: all sorts of cool ideas and interesting discoveries, but it doesn’t necessarily translate directly into something useful.

Next year is the year of model miniaturization, gpt4 performance on smart phones, likely for latest models

Every other lab is likely working very hard just so they can leave Google behind and build the next actually usable Search Engine & Portal (Browser / App Store).

I’m really hoping a new player displaces Alphabet and Meta (the ad giants).

Every year from now in AI will make the previous year seem inconsequential

And of course, 2024 is going to make 2023 look like 2022.

Are there any key indications for this? Why would it not just stop here (for a while)?

The key indications are capitalism, pace of current development (what was it, yesterday that mj6 dropped?), and active research.

In other words: 1) There's (a lot of) money to be made in making it better. 2) Releases are happening constantly, why would that stop? 3) Lots of recent research still that hasn't been productized and also that opens the door to future improvements.

OpenAI showed they can do capability extrapolate/predict (training loss) for the amount of compute they put in [1, see Predictable Scaling section]. This is also in line with earlier findings. Furthermore, there is no evidence at this point that there is an end in sight.

[1]: https://openai.com/research/gpt-4

> OpenAI showed they can do capability extrapolate/predict (training loss) for the amount of compute they put in [1, see Predictable Scaling section]. This is also in line with earlier findings.

That’s interesting if it’s actually true. I think it’s important to keep in mind that it’s also their own marketing materials…

We now have several organizations actively iterating on the same idea: OpenAI, Google, Apple, etc., most of which have more resources at their disposal than most countries do. They are also in fierce competition with each other. There's also a trend line if you look at the rate of progress over the last decade.

Then again, Bill Gates thinks it may have plateaued. He ought to be privy to OpenAI's going-ons. That, and Gemini was underwhelming.

> That, and Gemini was underwhelming.

Gemini Ultra wasn't yet fully released, so I'd like to correct your statement to: Gemini will be underwhelming.

Can you link to where Gates talks about this?

https://indianexpress.com/article/technology/artificial-inte...

> Then again, Bill Gates thinks it may have plateaued

He didn't say AI may have plateaued. He said the OpenAI may have reached the limit of the current design. Very different thing. From your link below:

He anticipates a stagnation in development initially. The billionaire said that, with GPT-4, the company has reached a limit, and he does not feel that GPT-5 will be better than its predecessor.

I don't understand the difference, since why would the company restrict GPT-5 to the current design of GPT-4?

Can I extrapolate that 2025 is going to make 2023 look like 2021?

https://xkcd.com/605/

Reply on news.ycombinator.com