Apple is the big laggard in terms of big tech and complex neural network models.
v. You will not use the Llama Materials or any output or results of the Llama Materials to improve any other large language model (excluding Llama 2 or derivative works thereof).
https://github.com/facebookresearch/llama/blob/main/LICENSE
Just like Google scrapes the internet to improve their models, it might make sense to ingest outputs from other models to improve their models. This licensing prevents them from doing that. Using Llama to improve other LLMs is specifically forbidden, but Google will also be forbidden from using Llama to improve any other AI products they might be building.
I understand trade-secrets are not free-speech but if the goal is to build better AI to serve the humanity the different bots should learn from each other. They should also criticize each other to find flaws in their thinking and biases.
It’s not.
Whose goal is that?
And they want to be very careful about labeling outputs as derivative works, because the moment they do that then they have no defense against the model being a derivative work of every single input.
> I wouldn't be surprised if Amazon does as well.
I would - they are not a very major player in this space.
TikTok also meets this definition and probably doesn't have LLM.
Google's publically available model isn't as capable. But they certainly have models that are far better already in house.
Meta is definitely ahead of Google in terms of NLP expertise and has been for a while. I suspect that Google released their best model at the time with Bard.
https://imagen.research.google/
https://google-research.github.io/seanet/soundstorm/examples...
https://google-research.github.io/seanet/musiclm/examples/
Why would it be surprising that they have better models for resarch that they don't want to give out yet?
The other things you are describing are just standard for research paper releases.
Yes I would agree with you if Google wasn't set on to full on panic mode by their investors about releasing something vs Open AI due to Chat GPT's buzz.
Bard was just a "hey we can do this too" thing, it was released half assed, had next to no marketing or hype.
Vertex AI is their real proper offering, and I want to see how PaLM 2 does in comparison.
Not going to keep replying, believe what you want about Google's capabilities
> Llama 2 70B results are on par or better than PaLM (540B) (Chowdhery et al., 2022) on almost all benchmarks. There is still a large gap in performance between Llama 2 70B and GPT-4 and PaLM-2-L.
https://scontent.fsyd7-1.fna.fbcdn.net/v/t39.2365-6/10000000...
If Google's publically available model is better Llama 2 already then why is it so inconceivable that they'd have private models that are better than their public ones which are better than LLama already.
Palm-2 isn't better than GPT-4 but the convo was about better than Llama models no?
You seem to be under the mistaken belief that: 1. Google has competent high-level organization that effectively sets and pursues long term goals. 2. There is some advantage to developing a highly capable LLM but not releasing it.
(2) could be the case if Google had built an extremely large model which was too expensive to deploy. Having been privy to what they had been working on up until mid-2022 and knowing how much work, compute and planning goes into extremely large models, this would very much surprise me.
Note: I did not have much visibility into what deepmind was up to. Maybe they had something.
Some among us work with it, or have friends or family who work with it. I imagine it is one of those.
Gotta put products in the market, or it didn't happen...
It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google.
If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despite being the first to publish on RLHF (just right before OpenAI) and bring the idea to the academic sphere, they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc). However, they're mostly academics, so they aren't set on making products, doing the janitorial work of fancy UIs and web marketing, and making things easy to use, like much of the rest of the field.
> they mostly work in areas tangential to 'just chatbots' (e.g. how to improve science with novel GNNs, etc)
Yes, Alphabet has poured tons of money into exotic ML research whereas Meta just kept pouring more money into more & deeper NLP research.
All the AlphaGo/AlphaFold stuff is very cool, but since no one has seen their LLMs this is about as convincing as my claiming I've donated billions to charity.
It was probably a challenge to integrate it into search, but they did that.
So your assertion has been refuted based on your use of "all", at the very least.
To assume that Google doesn't have anything competitive with Meta is to say that their papers just so happen to contain recipes for Meta's models but they've arrived at those not through training and benchmarking but by divination and bullshitting. This, let us say, does not sound plausible.
Then again, Microsoft uses LLaMA for research, and they should theoretically have some ability to get stuff from OpenAI. Evidently this isn't how any of this works, huh.
Google is getting the asses handed to them, badly. I figured that the code red would whip them into shape but the rot runs deep.
Google hasn't made their best models public because they're too expensive to run for free.
> Google is getting the asses handed to them, badly.
Bard has 30M active users and isn't even available in large parts of the world. They're in 2nd place - when they were pretty late to the game - that's an odd way to say someone is getting their ass handed to them.
?
It's the same issue with paid models.
I am paying per each request sent to Google Generative AI and this is what I get: https://i.ibb.co/4KCmz55/bard1.png
...
And then, given that, why is it worse than the competition?
If you actually have worked in the area of NLP for about 10 years, you would recognize how the work from Deepmind is much more novel and innovative than other groups. OpenAI certainly has great public facing services, and Meta should be congratulated for releasing these models (although I would still prefer the Galactica training data), but academically Deepmind is one of the best groups around.
See https://www.tomsguide.com/news/googles-new-gemini-ai-could-b...
Surely Google can find another team of code monkeys to whip out a frontend if there is money to be made.
I don't think Google is going to pull back from making some more money.
I think the most likely option is that they have a bunch of talented academics who get paid on time to work on what interest them - but they're the stereotypical large inefficient company and they can't coordinate the effort of productionizing some cool models before the competition.
I think your argument is basically that Google has the potential to create the best models because of superiority in the theory of LLMs, even though we hear of no signs from the board, the ceo, or beta releases or product showcases.
But let’s say you’re right. When do you think we would experience the supremacy of DeepMind in our daily lives?
If they work out how to deal with, say, New York weather conditions, there's potential, but they don't seem to be any closer.
Or perhaps my threshhold for “scalable” takes different parameters and weigh these inputs differentfly from you.
I'm pretty sure if google had something much better, the board and C-suite execs would have at least ensured we saw previews of it by now...
It seems it is not aware of the notion of historic development, perhaps its world-model is "static"?
Temporal reasoning is interesting , if you google for "news" do you get what was news last year because a website updated last year had a page claiming to contain "Latest News".
REF: https://www.stefanjudis.com/today-i-learned/property-order-i...
I agree that there is an incentive to put AI models on your OS. I just don't think Apple can own the whole stack if they want to play ball right now.
They will soon be able to train an LLM because it simply has become commoditized, but they just are not a major player in this space at all.
I thought the ml work they do in photos for text selection and facial recognition is pretty neat.
I don’t think we will “hear” about Apple using LLMs either way because they will no doubt call it something different like they always have.
That said, they seem to prefer catchup waiting till others explore new tech they swoop in an (claim) to perfect it from a usability pov. I have no reason to suspect they won’t do the same here.