There is a network effect forming around its models. The strengths of its kit speak for themselves. (It also cannot be understated how making ChatGPT public, something its competitors were too feeble, incompetent and behind the curve to do, dealt OpenAI a massive first-mover advantage.)
But as others note, other models are in the ballpark. Where OpenAI is different is in the ecosystem of marketing literature, contracts, code and e.g. prompt engineers being written and trained with GPT in mind. That introduces a subtle switching cost, and not-so-subtle platform advantage, that–barring a Google-scale bout of incompetence–OpenAI is set to retain for some time.
How true is this? From playing around with Bard and Claude, GPT-4 seems to be significantly better, especially around code generation / understanding.
Maybe PaLM is near there (it's not evaluated on that page) but nothing else even comes close at all
GPT-3.5 will get the gist of what the code is doing, and then provide what looks like a direct translation but differs in numerous details whilst having a bunch of other problems.
GPT-4 does a correct translation, almost every time.
It kills me that there's a waiting list for the API. I have put together some tools to integrate 3.5 into my workflow and it helps for my current task a lot (for others it's useless). But to really shine it needs to have API access to 4.
Qualitatively, it's wildly different from gpt-3.5-turbo for discussions. 3.5 feels a little formulaic after a while with some kinds of questions. 4 is much more like talking to an intelligent person. It's not perfect, but I'm flipping between discussing a sporting thing, then medical malpractice, legal issues, technical specifications and it's doing extremely well.
If it's affordable for you, I'd really recommend trying it.
Anything involving reasoning, code, complex logic, GPT-4 is a breakthrough. GPT-3.5 turbo is more than good enough for poetry and the other text generation stuff.
And yes, it does indeed make an amazing rubber duck for brainstorming.
At the very least, it's a massive productivity booster.
I have at most moderate confidence in this hypothesis.
> I am suspicious that Google is incompetent.
Google has put a lot of effort and investment into AI. With Bard I get the feeling they're not showing us what they really have - it's like for some reason they're holding back the good stuff, at least that's my suspicion.
They have the dominant product that makes them billions and billions of dollars at 'relatively' low cost.
The new dominant product is on its way, but it costs far more to operate and will net them far less money, so... um no one wants to kill the goose that is still laying golden eggs, even though its days are numbered already.
If I was Google I'd be worried. Very worried indeed. They either need to dramatically change their entire company within 18 months, or accept they are going to loose substantial amount of market -- and once its gone, it's gone in a first mover, winner takes all environment like what we have right now. Just ask Google themselves what it felt like back in the early 2000's when they completely destroyed the other search engines.
Now it is. Google used to be good too, until ads started looking like search results, and then the first page became entirely ads.
In the future, when you ask ChatGPT to help you write your resume, it will try to upsell you a premium account in linked in. It will withhold its best resume advice only for LinkedIn premium users after all.
You think Clips was bad? You’ve seen nothing yet.
The cost of computing these ads would be a lot more than today's keyword-based approach, that's certainly a problem. But think of hyper-relevant ads, based on the chat itself. There's a lot of information there, that beats tracking people's behavior online all day.
I’ve been infuriated with DuckDuckGo on occasion because it refused to exclude certain results.
In fact when you add an exclusion clause it simply boosts those results further instead of removing them.
I’ve been told this is because the underlying search providers refuse to exclude paying customer even when you explicitly don’t want to hear from them.
I could definitely see this happening in LLM answers too and I don’t expect it to be particularly subtle.
That depends on ad publishers, right? If they want to sell A, B and C and I am interested in D, then Google's still showing one of A, B or C to me. D doesn't make profit if there is nobody paying for ads.
Google is advertising things we don't need, that's why ad clicks are so abysmal. LLMs won't change that.
https://en.wiktionary.org/wiki/Kodak_moment
Etymology
(moment worth photographing): From an Eastman Kodak Company advertising campaign.
(business's failure to foresee): In reference to the Eastman Kodak Company's decline when cameras and film were overtaken by smartphones and digital technologies.
Noun Kodak moment (plural Kodak moments)
(informal) A sentimental or charming moment worthy of capturing in a photograph.
(informal) The situation in which a business fails to foresee changes within its industry and drops from a market-dominant position to being a minor player or declares bankruptcy.
Kodak Film Commercial - These are the Moments - Baby (1993):
I highly doubt this. If they had it they would show it because if they don't react swiftly and decisively their brand will be in 'catch up' mode rather than out front where they are used to being.
Google is run by smart people whose mission is to maximize clicks on ads. If a user finds what they’re looking for quickly, that’s lost revenue.
Google’s profit motives are not aligned with useful AI. The better AI is, the less people need to click through to lots of web pages and ads, the less revenue for Google.
I don’t think they can catch up without a major pivot in business model. It’s very hard to be deeply invested in providing more value if it means reducing your revenue.
https://seekingalpha.com/article/4469984-how-does-google-mak...
GPT3.5 turbo is much more interesting probably, because they seem to have found out how to make it much more efficient (some kind of distillation?).
GPT4 if I had to make a very rough guess, probably flash attention, 100% of the (useful) internet/books for it's dataset, and highly optimized hyperparameters.
I'd say with GPT4 they probably reached the limit of how big the dataset can be, because they are already using all the data that exists. Thus for GPT5 they'll have to scale in other ways.
I’m curious about this too; not just on the dataset size, but also the model size. My hunch is that the rapid improvements of the underlying model by making it bigger/giving it more data will slow, and there’ll be more focus on shrinking the models/other optimisations.
If anything, I wonder if the actual limit that'll be hit first will be the global manufacturing capacity for relevant hardware. Check out the stock price of NVDA since last October.
1. They would already be using everything they can get 2. They would easily be able to explain what they're not using, without giving away sensitive secrets.
They have not, which makes me curious about which company gp works for because the "F" and "G" in FAANG are publicly known to already have LLMs. Not sure about Amazon, but I'm guessing they do too.
As an outsider, the amazing thing about ML/AI research is that you get a revolutionary discovery of a technique or refinement that changes everything, and a few months later another seminal paper is published[0]. My bet is ChatGPT is not the last word in AI, and OpenAI will not have a monopoly on upcoming discoveries that will improve the state of the art. They will have to contend with the fact that Google, Meta & Amazon own their datacenters and can likely train models for cheaper[1] than what Microsoft is paying itself via their investment in OpenAI.
0. In no particular order: Deep learning, GANs, Transformers, transfer learning, Style Transfer, auto-encoders, BERT, LLMs. Betting the farm on LLMs doesn't sound like a reasonable thing to do - not saying that's what OpenAI is doing, but there are a lot of folk on HN who are treating LLMs as the holy grail.
1. OpenAI may get a discount, but my prediction when they burn through Microsoft, they'll end up being "owned" by Microsoft for all intents and purposes.
Discord comes to mind.
I'm guessing that this is the #1 fear for people inside OpenAI have right now.
[0] For the record, I have zero problem with this.
2. You won't be able to get the hundreds of millions or more interactions that OAI will have (both due to cost of API as well as it being not easy to figure out a good way to generate that many queries for a good multiturn conversaton). Maybe you can make up for it by querying smartly. We don't know if we can right now.
Google has been collecting user interactions since 2007 via GOOG-411, which was a precursor to the Google Assistant - I suspect Google has billions of user interactions on hand through the latter. Facebook has posts and comment, Amazon has products pages, reviews and product Q&As and all of them have billions of dollars to draw upon if they choose to buy high-quality data, or spin-up / increase teams that create and/or categorize training data.
They also have deep roster of AI researchers[1] to potentially obsolete LLMs or make fine-tuning work without access to of ChatGPT records.
1. I suspect Google alone has more AI researchers that OpenAI has employees
OpenAI has released a ton more easy-to-use-for-everyone stuff that has really leapfrogged what a lot of "applied" folks everywhere else were trying to build themselves, despite being on-the-face-of-it more "general."
What? Huh? Yes the human genome encodes all human level thought.[1] Clearly it does because the only difference between humans that have abstract thought as well as language capabilities and primates that don't is slightly different DNA.
In other words: those slight differences matter.
To anyone who has used GPT since ChatGPT's public release in November and who pays to use GPT 4 now, it is clear that GPT 4 is a lot smarter than 3 was.
However, to the select few who see an ocean in a drop of water, the November release already showed glimmers of abstract thought, many other people dismiss it as an illusion.
To a select few, it is apparent that OpenAI have found the magic parameters. Everything after that is just fine tuning.
Is it any surprise that without OpenAI releasing their weights, models, or training data, Google can't just come up with its own? Why should they when without turning it into weights and models, the human neural network architecture itself is still unmatched (even by OpenAI) despite being digitized twenty years ago?
No, it's no surprise. OpenAI performed what amounts to a miracle, ten years ahead of schedule, and didn't tell anyone how they did it.
If you work for another company, such as Google, don't be surprised that you are ten years behind. After all, the magic formula had been gathering dust on a CD-ROM for 20 years (human DNA which encodes the human neural network architecture), and nobody made the slightest tangible progress toward it until OpenAI brute forced a solution using $1 billion of Azure GPU's that Microsoft poured into OpenAI in 2019.
Is your team using $1 billion of GPU's for 3 years? If not, don't expect to catch up with OpenAI's November miracle.
p.s. two months after the November miracle, Microsoft closed a $10 billion follow-on investment in OpenAI.
OpenAI is enjoying first mover advantage around the platformication and product-ification of LLMs.
For instance, why has G not yet exposed some next-level capabilities in mail, in docs, and many of their other properties?
Why do Google Assistant and Amazon Alexa and Apple Siri still suck?
Bard is overtly a reduced-resources model compared to the best version of the same technology (which, if true, is probably a boneheadedly bad choice for a public demo when everyone is already wowed by the people who got theirs out first, but easily explains that disparity. Though so does “guy who wanted public attention made stuff up well-calibrated to that goal.”)
There's a scaling problem. ChatGPT/LLM systems cost far more to run per query than the Google search engine. Google can't afford to make those the first line query reply.
A big business model question is whether Google will insist you be logged in to get to the large language model.
At Google scale, these things are going to have to be a hierarchy. Not everything needs to go to a full LLM system. Most Google queries by volume can be answered from a a cache.
And given how aggressively they limit the number of search results (in spite of listing some ridiculous number of results on page #1) that percentage may well be very large.
Bard is way behind ChatGPT with GPT-3.5, much less GPT-4. Haven’t tried the others, though.
OTOH, that’s way behind qualitatively, not in terms of time-of-progress. So I don’t think it is at all an insurmountable lead, as much as it is a big utility gap.
GPT4>ChatGPT>Claude>Character AI> Bard
Claude and Character AI are great at holding a conversation but they lack the ability to do anything specialized that really makes these LLM’s useful in my day to day life. I ask GPT-4 and ChatGPT questions I would ask in stackoverflow, I can’t do that with Claude or Character AI. Bard actually seems behind even conversationally to the rest