Bard and new AI features in Search
blog.google
blog.google
A number of pundits, here on HN and elsewhere, keep referring to these large language models are "google killers." This just doesn't make sense to me. It feels like Google can easily pivot its ad engine to work with the AI-driven chat systems. It can augment answers with links to additional sources of information, be it organic or paid links.
But I guess I'm wondering: what am I missing? Why would a chatbot like ChatGPT disrupt Google vs forcing Google to simply evolve. And perhaps make even more money?
Simple as.
1. People are typing questions into it and finding it hugely useful in ways that overlap with google. It just answers you, and you can ask clarifying questions, argue with it, etc. It's proving useful.
2. More subtly, the excitement around it seems to reveal that people are open to alternatives. For decades, alternative search engines haven't made inroads. If people are typing queries and questions into somewhere new, well that implies they're open to something new. So whether it's chatGPT or bing.com or ... the zeitgeist is shifting.
You're giving "assistant"-like questions, but the problem with those assistants has always been how shallow their responses have been, which significantly limits their usefulness.
GPT's responses are still shallow in an absolute sense, but relatively speaking they're the Mariana Trench compared to Google's little creek.
If I want to be amused by a Hacker News comment in the style of the Bible, draft a conclusion for my essay or a engage in a long and superficially appealing conversation about philosophy, I'm not using Google's publicly-viewable AI products
Then again, Google - with and without a conversational interface - will do just fine with what the capital of Maine is, much better with what the weather is like in Maine tomorrow, and there's a lot more usefulness and revenue in associating it with stuff in my address book and selling me flight tickets to Maine which is... some way outside ChatGPT's wheelhouse.
It’s not infrequent that googling for the intricacies of some badly-documented library turns up almost nothing useful, or the bits that are useful are scattered sparsely among the results, some of which are pages deep. It’s so much easier to ask ChatGPT to explain the struct, function, etc in question and have it pull the pertinent info from whatever corners it of the internet it found these things in. Even if it’s only 80% accurate it’s a massive time saver.
https://aws.amazon.com/machine-learning/trainium/
https://aws.amazon.com/machine-learning/inferentia/
I really don't think this would be a limiting factor regardless, even if Amazon didn't already have multiple generations of these products. It's not as if an Amazon or Microsoft sized company is incapable of developing custom silicon to meet an objective, once an objective is identified. TPUs also aren't really that complicated to design, at least compared to GPUs.
I'm slightly surprised Microsoft hasn't bothered to release any custom ML chips for Azure yet, but I guess they've run the numbers and decided to focus on other priorities for now.
Google/Alphabet used 6514 terawatt hours in 2016 according to data collected from yearly reports at https://www.statista.com/statistics/788540/energy-consumptio...
If my math is right, dividing that by 8760 hours in a year, you get 743MW.
Of course, that also includes office space, etc. (did the AWS number, too?) but it should be clear that, cross checking with data center builds, optic fiber and energy purchases as well, for years Google+YT+Apps+GCP were larger than Amazon and all other AWS customers combined. I didn't even factor in efficiency, something that Amazon started focusing on quite a bit later.
Someone might be able to extrapolate both numbers to today based on infrastructure, other metrics or other spend in quarterly financial statements (or power procurement, which will be complicated by the non trivial Amazon vs AWS distinction).
All of the above to say that Amazon probably has more compute now, but it's a stretch to talk about "paling".
I also doubt computing power is the real bottleneck. Anyone of these companies (and most other companies too) can build enough large servers sites and they have the money. The costly and difficult part is the engineering resources, doing the right thing technically (AI wise) and business wise, not lose time, not bet on the wrong horse etc.
They can deliver unsuccessful stuff on the web at scale, they can deliver successful stuff that turns out to be inconsequential for their bottom line on the web in the long-term (AJAX came from MS), but it's just not in their DNA to take over the web. They had lots of chances to do it during the last 20 years or so, they had all the silver bullets at their disposal, they just couldn't deliver what it took.
Not sure what decade you're stuck in here with comments about ajax
I explicitly mentioned the "web", as in, what we're doing right now on this website. Leaving aside the fact that Azure is mostly used big corporate/government entities, there's no web-startup that will potentially dominate the web and that would go into Azure just as.
It's this type of confidence that you come back to HN over and over for.
Let's ignore Azure, Office 365, and Microsoft's other online properties. In just video games: Microsoft owns and operates the Xbox network with over 100 million active monthly users[1]. That runs on the internet with all the features of a social network and more. I think they can deliver successful stuff at web scale OK, whatever their other shortcomings.
1 - https://hothardware.com/news/xbox-live-surpasses-100-million...
I explicitly mentioned the web, Office365 is not a successful web story. It is a successful enterprise story (afaik it is still wildly profitable for MS the company), but it is not a successful web story. Ditto for Xbox.
Is the web a consumer product that runs on the internet?
What's niche about websites?
Leaving aside the snarky tone, I'll give you an example of how MS has continuously botched their web work for 20 years.
If I go on Bing Maps (which I'm actually using on one of my pet projects on account of their permissive licensing) and I type in my Bucharest street address the auto-completion thing works fine, which is a plus (Apple's is much worst at that), but then again their map web project ends up pointing me about 200-300 meters from where I actually live. Google Maps does it pitch-perfect, has done it pitch-perfect for years (I think there have been around 10 years since they've added exact address searches for Bucharest). Many other such cases.
Later edit: Forgot to mention, two of the closest POIs shown on Bing Maps have been closed for two years with other places having taking their place in the meantime. Again, GMaps has been almost instantaneous on putting that on their maps, MS seems to be a lot slower at that. That's what the web is all about, data that counts and that is of interest.
Office 365 is web based. So does that count or is it conveniently a website also used by companies (and consumers) so we discount it?
What's unique about web sites that you think is harder than what they've done elsewhere? What makes web sites harder to scale?
Their inability to do mapping well has nothing to do with websites. So please, kindly stick to a definition and stop moving the goal post.
These two products alone are used by maybe 70% of developers, and don't forget about copilot and all the integrations between github/vscode.
https://openconnect.netflix.com/
Some people who’ve worked on it post around here and they’ve funded things like FreeBSD development which is interesting for seeing the kind of problems you have at that kind of traffic volume.
go for their paid offering if latency is bothering you
Of course, even if search revenue falls, it won’t happen overnight.
But I honestly don’t see how laying all that fibre or owning those data centres is a moat around Google. These things are hugely capital intensive, to be sure, but theres a big market for both.
This feels like "thinking inside the box" to me. None of these things are necessary requirements for being a "Google killer".
Probably still an open question but a better chance then anyone has had to disrupt them for two decades.
Because it's not clear yet whether anyone else can currently develop a chatbot interface as capable as ChatGPT.
For many ChatGPT is already replacing a lot of Google searches so G needs to hurry.
1) Because it is disruptive. Things may get shook up, and Google may not end up in as exclusive position as they are currently in. There's risk.
2) Because it's not obvious how advertising would fit in with a conversational interface. Google may stay as #1 search/answer engine, but would revenue be adversely affected?
Full disclosure: I work at Google, but nowhere near the chatbot stuff. This is my humble opinion and nothing more.
I'd say your answer is an "advertisement" for ChatGPT. The parent AND the grandparent didn't mention OpenAI. This is how you could advertise in answers.
I'm not sure this is true. Surely they are more lucrative when they are indistinguishable, but for a lot of Google's history they were very noticeably different, a fact Google even prided itself on, and bragged about. Seems like it was growth requirements that changed that, not whether the ads were originally lucrative when people could tell they were ads.
Edit: Why have ads separable from content if you can just weave them in? Ad-blocking is toast, you'll have to use another AI service to fish them out and resummarize.
Unless somebody could clarify those for me, this is what currently petrifies me -- some uncontrolled black box presenting its clandestine view of the web with no way to follow the breadcrumbs.
Even if a voice assistant allowed you to interrupt it, to make fast course corrections, it would still be much slower than, say, interacting with the filters on Google Flights.
And I am saying this after having built for myself a bi directional voice interface to ChatGPT. There are certainly situations where it is great to use it, such as while driving, or perhaps in the kitchen while having your hands full. And probably on mobile, where screen real estate is scarce. But those doing information, work, or even just online shopping, probably won’t be giving up their screens anytime soon.
Google has to fight to get that mindshare back.
Not sure I’d be building my company on the ADHD-fueled Google roundabout that generates and destroys systems monthly. You just know whatever they release is someone’s promotion project, until it’s in GA.
Clubhouse was once the new hotness - until it wasn't.
I didn't say it's a guarantee of success, it's a possibility. There is a non-zero chance that ChatGPT takes market share away from Google unless it moves very quickly.
Having said that, the status-quo play is obviously the easiest bet. It's far easier to be Google than OpenAI or Microsoft at this point.
https://www.wsj.com/articles/microsoft-adds-chatgpt-ai-techn...
Microsoft has a lot of advantages here - they can introduce LLM to search at a much smaller scale (order of magnitude), which means its cheaper, and they have plenty of other products making tons of money, so they can take their time to figure it out (also they're already adding ChatGPT to tolls like Teams and possibly Office, so they'll be able to increase revenue from these products).
Google is also seen as a bit of a dinosaur - they struggle to introduce new products, and recently we've been hearing more about products they kill rather than huge successes. It seems that as a company they lost their innovative spirit, and that's why people don't believe they'll evolve quickly enough.
Of course, Google is a giant today with a history of machine learning innovation. So they have a good chance of being successful in that new world. But the point is that many other companies get a chance again, which hadn't happened in 20 years. You couldn't displace Google in the old search/keyword/click business model. Now everyone gets a fresh start.
Who knows what the economics will be. Just like page rank early on, it was expensive to compute. But the money in advertising made it worth it, and Google scaled rapidly. Which language model do you run? The expensive one, or the light one (notice how Google in this announcement mentions they will only offer the significantly smaller model to the public). Can you make this profitable?
Other fun questions to answer if the industry moves to chat vs. search, in a 5-10 year horizon. What is the motivation to write a blog post by then? Imagine no one actually reads web sites. Instead a blog post to share an opinion, I'll probably want to make sure my opinion gets picked up by the language model. How do I do that? Computed knowledge may render many websites and blogs obsolete.
Most likely this will not actually happen, and even if it did your content would still be valuable as an AI is analyzing it in a more nuanced way than just looking for keywords. Which, by the way, is exactly what search engines do.
Technical changes do kill jobs. We always find a way to invent jobs, of course, but that doesn't mean old jobs aren't viable.
Movie theaters once employed professional musicians, not they don't, because movies have audio built in. Obviously a net-loss since musicians are jobs people like. Less coal miners or farmers is probably a good thing though.
It all depends on the type of job you replace. If you replace hard manual labor jobs, you're a net-good. Replacing a job people like... and you'll get a negative label. Doesn't change the fact that progress marches on, but jobs are killed by tech changes.
We will automate some bullshit jobs but create all kinds of new bullshit jobs that have titles that start with AI.
Thousands of titles like "AI ____ ____ Manager" that also does nothing but schedule meetings about meetings about AI.
The mistake to me is to believe bullshit jobs are the end result of some systemic inefficiency that AI is going to automate out of existence. I just don't think that is at all the case because otherwise we would just cut so many bullshit jobs right now without AI.
Where there is human attention, there will always be ads. The more context, the better ads.
It’s possible that Google can deliver a few targeted ads, but what if they can’t? What about the rest of the market that’s now gone? Possible that all those missed opportunities remove the ability to discover price.
"Here is the answer to your question about oranges. But did you know Tropicana is made from 100% real orange juice?"
"A project manager is a ... Often the software project managers use is Zoho Projects for the best agile sprint planning"
If they can put ads in it, there will be ads in it.
All the money goes to Google!
No more sharing with websites where Google Ads appear. They can even autogenerate youtube channels explaining popular or trending topics. Which of course, they will know, because they'll own search and AI generation. So there will also be no more paying a large portion of youtubers.
People who explain topical subject matter on youtube could, if Google chose, be eliminated. And even if Google doesn't, some content mill in Manilla definitely will.
That's an aspect I hadn't considered, nor heard anyone else suggest!
This is the first time that the primary cash cow has been seriously threatened, and it’s not unreasonable to bet against Google winning the scramble to figure out a chat AI ad strategy (or any product strategy) that would keep them in their current near-monopoly position.
Actual GPT3 answer: A project manager is a professional responsible for leading a project from conception to completion. They coordinate the activities of the project team to ensure deadlines and budgets are met. Zoho Projects provides project managers with the tools they need to manage projects efficiently and effectively.
Joking aside, there's no reason AdWords can't become AdWordVectors and be even more effectively targeted.
In today's world, an ad is clearly an ad.. Or is it? Even now we have advertorials and "sponsored posts" that blend into content maybe a little too much sometimes.
What happens when chatbot companies start taking money to subtly nudge their models into shilling for this or that product.
Keywords will still be around at the user interface for people buying ads, they are easy to grasp. Part of the secret sauce is getting those keywords mapped into the right entities in a sort of knowledge graph of things you can spend money on that is also connected to all the content of the places you can serve ads on.
Obviously this is all going to change in the near to mid future: innovation will drive down costs of both training and inference, and the models will be monetized in ways that bring in more revenue. But I don't think the long term economics are obvious to anyone, including Google or OpenAI. It's really hard to predict how much more efficient we'll get at training/serving these models as most of the gains there are going to come from improved model architectures, and it's very difficult to predict how much room for improvement there is there. Google (and Microsoft, Yandex, Baidu, etc.) know how to index the web and serve search queries to users at an extremely low cost per query that can be compensated by ads that make fractions of a cent per impression. It's not obvious at all if that's possible with LLMs, or if it possible, what the timescale is to get to a place where the economics make sense and the service actually makes money.
2. Don’t Google have specialised hardware for training neural networks? If the costs of training/inference are very significant won’t Google (with their ASIC and hardware design team) have a significant advantage? It seems to me that their AI hardware was developed because they saw this problem coming a long way off.
2. Yes. Yes they will/are developing custom silicon that will likely be a significant advantage here. GPU costs were always crazy, and many companies are designing AI chips now. Even the iPhone chips have custom AI Cores. We'll see if Azure releases AI co-processors to aid them...
Then again maybe language models will create the metaverse
Realistically VR clients needs to pose as browsers to load vr:// links which can be connected to objects/actions in VR ie <portal color="green" size="200,200" pos="122.469,79.420,1337.25" href="vr://some-online-shop-showroom.domain.tld" />
That way it's done in an open, browsable way compatible with the expectations that we've gained from regular web experiences. Ie you have your VR home, there's a "bookmark" door you can walk through to go to Amazon's showroom, you search for a particular product and it lets you walk around and pick-up and examine the various options, you can then jump to a particular brand's individual showroom, etc.
Some people might feel that's a little dystopian I suppose but I think it's cool.
A decent chunk of the tech community can already run smaller T5 Flan models or the 6B EleutherAI LLM GPT-J (and the likely similarly sized upcoming Open Assistant) on their own machines, at decent inference speed (< 0.2 s per token, which is ok enough most of the time). By 2027 or so the majority of consumers will likely exceed that point.
What happens when models are updated every day automatically and you can run all your search and answer tasks on your local machine?
With GPT-J - which is unreliable and aging now - I can already learn core intro to psychology topics more accurately/faster than with google or wikipedia, and I can do that offline. That's a cherry picked use case, but imagine the future.
Why would you use something that has ads when you can run it locally, and perhaps even augment it with your documents/files?
In the end game this is where Google is in the same place as Kodak in my opinion right now. Sure it's $0.01 or more a search for OpenAI now, but it won't stay that way (they reduced prices by 66% half a year ago), and at that rate you can already make the unit economics work anyhow as a startup.
The proverbial 'some guy on Twitter'[2] got it setup, and broke down the costs, demonstrated some prompts, and what not. The output's pretty terrible, but it's unclear to me whether that's inherent or a result of priority. I expect OpenAI spent a lot of manpower on supervised training, whereas this system probably had minimal, especially in English (it's from a Chinese university).
If these technologies end up as anything more than a 'novelty of the year' type event, then I expect to see them able to be run locally on phones within a decade. There will be a convergence between hardware improving and the software getting more efficient.
* I don’t own a hat
Not much different from video and graphics accelerators being integrated today, or DSP focused instructions in the instruction set.
They just have to do it to stay relevant.
But leaving that aside, look at OpenAI's pricing. $.02/1K tokens. Let's say the average query would be 20 tokens, so you'd get 50 queries/$.02 = 2500 queries/1$, or for 100k, $40/sec * 86400 * 365 = $1.2b. My guess is OpenAI's costs right now are not scaled to handle 100k QPS, so they're way underpriced for that load. This might be a cost Google could stomach.
I just think blindly shoe-horning these 100B+ param models into this use case is probably the wrong strategy, DeepMind's Chinchilla has shown it's possible to significantly reduce parameter size/cost while staying competitive in accuracy. I think Google's going to eventually get there, but they're going to do it more efficiently that brute forcing a GPT-3 style model. These very large parameter models are tech demos IMHO at this point.
That said I do agree with your final conclusion. Bigger is not necessarily better in neural networks, and I also expect to see requirements rapidly decline. I also don't really see this as being something that's going to gets ultra-monopolized and centralized. One big difference between natural language interfaces and something like search is user expectations. With natural language the user has an expectation of a result, and if a service can't meet that expectation - then they'll go elsewhere. And I think it is literally impossible for any single service to meet the expectations of everybody.
There's a lot of duplication in those queries. If the answers can be cached, a more useful metric would be unique queries per some unit of time (longer than a second).
That said, I don't have the numbers. :)
2- Moore’s Laws
3- Algorithm/architecture improvement
That's on top of
A. the hundreds of millions it will take to get a design to production.
B. The complete and total lack of allocation at anybody who could make you chips, except at very very very high cost, if at all - have you forgotten that automakers still can't get cheap chips made on older processes? Most allocation of newer processes is bought out for years.
While there is some ability to get things made at newer process, building a chip for 7nm is 10-100x as expensive as say 45nm.
C. The fact that someone has to be willing to build, plan, and execute putting them in datacenters.
This will all happen, but like, everyone just assumes the hard part is the chip inefficiency.
We already can make designs that are ~10x more efficient at inference (though it depends on if you mean power or speed or what). The fact that there are not millions in datacenters should tell you something about the difficulty and economics of accomplishing this.
People aren't sitting around twiddling their thumbs. If Microsoft or Google or anyone could make themselves a "100x better cloud for AI", they would do it.
2. Dead. Dennard scaling went out the window years ago. Other scaling has mostly followed. The move to specialization you see and push to higher frequencies is because its all dead.
3. Will take a while.
The economics of this kind of thing sucks. It will not change instantly, there isn't the capability to make it happen.
This is not accurate - they are neither cheap nor really built for inference.
I'd have to go look hard at what is public vs not if you want me to go into more.
Shortages of 28nm and older nodes are not indicative of other nodes, because 28nm is (or at least was) the most cost effective node per transistor, (so plenty of demand) but no new fabs are being built for that node.
Both have a similar architecture (different scale) where they dich most of the vram for a fabric of identical cores.
Instead of being limited by the vram bandwidth they run at the chip speed.
Nvidia/Intel/AMD/Apple/Google and others surely have plans underway.
As the demand for AI grow (now clear that there is a huge market) I think we will see more players enter this field.
The landscape of software will have a dramatic shift, how much of the current cpu running in datacenter will be chips for AI in the future, I think it will be most of them.
Jim Keller has a few good interviews about it.
For a group of people on a site frequented by startup people, did nobody read the terms of MS's investment into OpenAI?
"Microsoft would reportedly put down $10 billion for a 75% share of the profits that OpenAI earns until the money on the investment is paid back. Then when Microsoft breaks even on the $10 billion investment, they would get a 49% stake in OpenAI.
These are not the terms you would take if, tomorrow, or even two years from now, you were about to be wildly profitable because everything was about to be so easy.
These are the terms you would take if Microsoft was the only hope you had of getting the resources you need, or if getting somewhere was going to be very expensive and you needed to defray costs.
Honestly, with the level of optimism in the rest of this thread about how easy this will all be, they would probably be profitable enough to just buy MS in like 3 years , and wouldn't have needed investment at all!
If somebody needs ten billion dollars in additional work and investment to make that a reality, how certain can they be?
OpenAI has 375 employees (according to Google). At 300,000 a head thats 110M per year in compensation. Let's say that their compute costs are enormous and go to 200M in expenses per year. 10B is fifty years of expenses. So if they need 10B in investment it becomes obvious that they believe that they have to change something about their business in a fundamental way. Maybe that is going to be enough, but if it is so certain it becomes hard to believe that they'd need this kind of investment.
OpenAI talks mostly about trying to help change humanity, not about winning at business. Their mission is still "advance AI safely for humanity". It's not even obvious they care about winning at business. We seem to be putting that on them.
In that sense, i'm not actually sure they care whether they beat Google or not. I mean that honestly. If they care about the stated goal, it would not matter whether they do it or Google does it. I'm not suggesting they don't want to win at all, but it doesn't seem like that is what is driving them except to the degree they need money to get somewhere?
If they really succeed at most of their mission, killing Google might be a side-effect, it might not, but it would just be collateral damage either way.
Beyond that, I don't disagree, i actually agree with you. My point on that front is basically: "Everyone thinks this will be cheap and easy very soon, and change the world very quickly".
I believe (and suspect OpenAI believes) it will be very expensive upfront, very hard to get resources in the short term, and change the world slower as a result
> Microsoft would reportedly put down $10 billion for a 75% share of the profits
> that OpenAI earns until the money on the investment is paid back. Then when Microsoft
> breaks even on the $10 billion investment, they would get a 49% stake in OpenAI.
> Me: 8 year old girl birthday party ideas
> Chatgpt: <a list including craft party, scavenger hunt, dance party>
> Me: what products or services could I buy for it
> Chatgpt: craft party: craft supplies such as beads, glue, paint, and fabric - scavengerhunt: prizes for the scavenger hunt and decorations - dance party: Hire a DJ or a dance instructor, and purchase party lights and decorations
Though in reality Google already has highly tuned models for extracting ads out of any prompt
With ads in link based search engine, you can skip or block them, but if it is a part of a one sentence answer, there is not much you can do about it, so consuming it will be much more frustrating.
Of course, there will still be a lot of people who will choose the free information paid by the advertisers, but there will also be a growing number of users who will prefer not to have advertisers pay for the information they put into their heads (it is already clear that such information will be of higher quality).
My prediction is that in 10 years, all free information paid by advertisers will have 'for entertainment purposes only' label, because by then we will understand as a society that that is its peak value.
There will be some shifts for sure, but I'm not convinced that they'll be that large, since we're already pretty screwed on the signal to noise ratio of the www.
ChatGPT extracts value from the ecosystem by hoovering up primary sources to provide answers, but what value does ChatGPT give back to these primary sources? What incentivizes content creators to continue producing for ChatGPT?
Right now, nothing.
ChatGPT (or its descendants) must solve this problem to be economically viable.
And yet it provides references and attribution where possible most of the time.
How would that affect your monetization?
The dissemination of their thoughts and ideas.
I know all sorts of things, many in great detail and with high confidence, that I would be very challenged to appropriately source and credit the originator/inventor. I suspect most people are similar.
Substitute “memory safety of Rust” or “environmental concerns with lithium batteries” depending on your interests
Perhaps there are some mitigations for this I'm unaware of?
Yes, maybe some content dries up -- no more stock photo sites -- but entirely unclear how important and they can wait to see how zombie companies adjust. Ex: ChatGPT encourages us to put more api docs online, not less.
At some point that destroys the web as sites move behind paywalls. Google or Facebook giving you less revenue is still a lot better than receiving nothing.
In some cases, that’s fine (AWS doesn’t mind you learning how to call their metered APIs on someone else’s site) but there are a ton of people who aren’t going to create things if they can’t make rent. Beyond the obvious industries like journalism, consider how many people are going to create open source software or write about it when they won’t get credit or even know if another human ever read it.
Eventually the ecosystem might collapse when people realize they get more accurate, up-to-date information from sources other than ChatGPT. But considering that ChatGPT's answers are already more "truthy" than "truth", accuracy does not seem to be a top priority for most information-seekers.
It will all be about access to new information, access to customers (such as advertisers) and access to users attracted to other aspects of the platform as well.
I think producers of new content and their distribution platforms will have a lot of leverage. Youtube, Facebook, TikTok, Spotify, Apple, Amazon, Netflix, traditional publishers and perhaps even smaller ones such as Substack and Medium, are all gatekeepers of new original content.
I think Google is best positioned to make the economics work. Unfortunately, they don't appear to have the best management team right now. They keep losing focus. Perhaps the danger of their core business getting disrupted will focus their minds.
Ie there may be 10-100 news articles about an event all with the same source. Youtube has tonnes of duplication/"reaction" videos where the unique content portion is very minimal.
Look up the term "native advertising", that should help you in understanding how online ad ecosystem works.
AdSense is going to be able to be more targeted and relevant than ever before.
Last week, Linus Tech Tips used ChatGPT to build a Gaming PC... from parts selection to instructions on how to build. When chatGPT said, "first, select a CPU", Linus asked it questions like "What CPU should I choose, if I only care about gaming?", and got excellent answers.
I can imagine BestBuy, NewEgg, and Microcenter will be fighting for those top AdSense spots just as much as they do today
"Bard, I'm looking for a blender to make smoothies" ... "does it come in red?" ... "I want it to crush ice" BUY
Realising that has made me wonder why I should bother write anything publicly accessible online.
Aside from pure altruism and love for my fellow human, or some unexplainable desire to selflessly contribute to some private company’s product and bottom lime, in a world where discovery happens through a language model that rephrases everything it’s way and provides all the answers, why should I feed it?
What do I stand to gain from it, apart from feeling I have perhaps contributed to the betterment of humankind? In which case, why should a private company reap the benefits and a language model the credit?
The AI will absorb your words, and some small part of you will gain immortality. In some small but very real way, you'll live forever, some part of you ensconced safely for all eternity in a handful of vectors deep inside a pile of inscrutable matrices.
...at least, until some CEO lays off that whole team to juice the stock price.
Sure I could carve my name or a blog post into a cave wall… so what.
“Some small part” of me doesn’t live on.
Even some small part of Aristotle or Cleopatra doesn’t live on. Ideas and stories live, but people die.
The death of personality is currently total and final.
I don’t know why Billionaires don’t invest their entire fortunes into research on reversing this.
If I think 500 years into the future, what would be great is if my descendants are ample and thriving, and my values are upheld. That feels like such a win to me. The fact that I won't physically be there is irrelevant.
On the other hand, artificial continuation of an otherwise impact-less life sounds awful to me.
I suspect that billionaires (certainly, the 2 that I have some insight into having worked for them) think much more about impact they are creating, than some sort of "hang on forever like a barnacle" type of existence.
That sounds like a sort-of-religion of the future, actually.
> ...at least, until some CEO lays off that whole team to juice the stock price.
Or the model is retrained on a different somehow more relevant dataset. Or the company shuts down because of a series of poor choices. Or something new and vastly better comes along.
Or... who knows? The possibilities are so vast that seeking immortality is ultimately futile.
* I figure I’ve got about 25 years left, so always = 25 years. Good luck, kids.
> Good luck, kids.
Thanks!
DisallowModelUse: *
LLM’s have done a poor job with attribution and provenance, but that will change.
At some point, it becomes a bit like academia or celebrity: your motivation to write is the social exposure your writing earns, which leads to real world opportunities like jobs or sponsorships or whatnot.
And the great/terrible thing is that these models will know whose opinions are influencing others. The upside is that spam disappears if human nature changes and nobody is influenced by spammers. The downside is. . . Spam and content become totally inseperable.
Google doesn't make money with AdSense, it pays publishers with it. I agree that there won't be a need for AdSense, that just means Google gets to keep 100% of the profit instead.
No, not everyone gets a fresh start at all. To train anything close to ChatGPT you really do need to be the size of Google or Microsoft to have enough the compute power.
There is problem with those AIs - you view the world trough the ideological prism of their creators and censors. So chatgpt that is more than happy to make jokes for some races and not others or other types of shenanigans is something I am sure will happily hide the information I actually want to find and feeds me what it wants me to find. So until there are guarantees about ideological neutrality they are not suitable for search for me.
Some assumptions: 1. Url-based web will not wither away. 2. Asking questions in the chat-like mode is more natural to people. 3. Generated answers cost more when longer. 4. Generated answers are some kind of distilled knowledge and can't be right all the time. 4. People don't like long answers and prefer the concise one. 5. Sources and citations make generated answers more credible. 6. Fully exploring a question needs a lot of information from different views. 7. Generated answers
some simple thoughts: The search behavior would hugely be two main steps: 1.getting some concise answers from the AI model directly through a chat, which might be enough for 90% use cases. 2.some more extensive search just like how people are searching today, which might be a kind of niche.
For websites, being cited in the generated answers will be the new kind of SEO things, and it would be a good strategy to producing some newest, deep or long-tail knowledge and information, which leads to a more traditional way of search because AI model doesn't have enough data to generate a good answer.
...
It's not just that it's chat, its the ability to refine. Currently, I search something. It returns garbage. I search something new. What I dont do is tell the search what it did wrong the first time. I might sort of do that with -words, but its a fight every time.
The beauty of these new chat systems is that they have short term memory. Its bein able to work within the parameters and context of the conversation. I dont particularly care if it is "chat like" or has its own syntax, what I want is a short term state.
And at the same time, I want long term state. I want to be able to save instructions as simple commands and retrieve them later. Like if I am searching for product reviews, to only return articles where it is convinced the people actually bought and tried the products, not just assembled a list from an online search.
If this new paradigm dominates the way people use computers and it's not as profitable as Search, Google might indeed have to scale back.
They've tried to compete elsewhere, too, and I don't think they've ever been able to make a go of it outside their cash cow. The only thing they've really been able to do is 'search results + ads'.
I don't think they'll be able to modify that winning combo even in the slightest and still be successful. And in this case, they can't buy the competition. Micro$oft already did.
YouTube was founded in ~2005. Google bought it in 2006. It is now 2023.
YouTube has spent 2 years as its own company and 17 as part of Google.
Try to remember what YouTube functionality was in 2006. It was very different and has grown a lot.
The narrative that Google doesn’t know how to innovate YouTube doesn’t add up.
Google hasn't shown they can do new product in a very long time... see the GCP mess, Stadia, and the hundreds of other total failures (Plus, Wave, and many I've forgotten).
I'm so put off by it, and been made a fool by it so many times, I'm phasing out Google in my life and long ago stopped recommending Google products to people in my life.
Same story for Google+
Clearly TikTok is a better surveillance network and managed just fine.
What I'm trying to get at is, imagine if Microsoft bought youtube before google could - given Google's track record with their own video search, they would not have been able to compete with youtube, and they would almost certainly have simply lost that market. I think that's what happened here. Google is amazing at algorithms, but has very little business sense...they can buy a successful product, but rarely create one of their own.
Satya Nadella seems way more tuned into the zeitgeist. His heavy bet on OpenAI may seem excessive, but at worst it's going to be cheap insurance and at best may be a game changer.
I think Google's producing Android was a reaction to BOTH iPhone and Windows Phone - they didn't want to be frozen out of the mobile advertizing market by competitors that owned the platforms.
If I on the other hand use generative AI, then the answer (and hopefully correct one) would be generated only for me. This is the personal touch Google currently misses and I guess it's appealing for many people.
Currently LLM-s are not Google killers, they can't find me a restaurant, a nice watch or other stuff I'd pay money for. Yet.
That is, the cat is out of the bag:
ChatGPT not only showed us the power of AI, but it showed us a bunch of non-AI things like:
- Ad-free results - Clutter-free results - The elegance of not having to click on links
A competitor could capitalize on this, putting Google's ad-driven, click-driven, clutter-driven model at serious risk.
It's a two-part claim; a) yes, conversational AI agents will replace typing searches into a text box, but b) I see no reason to think that Google can't easily monetize that format.
Regarding the first point, I think Google appears to have been caught a little bit off guard as to how soon this transition would happen. People seem to be over-indexing on the "code red". I do think it's a strategic mis-step by Google to not have a product ready to go here. (Their broader strategy was quite risk-averse and that was probably sensible, given the shit-storms that previous systems like Galactica and Tay generated; Google couldn't be the first ones to publish a prototype/demo system like ChatGPT, or the NYT would have jumped all over them for the inevitable questionable utterances.)
But the second part; given AI agents are here, who's going to win the competition to monetize them? It seems clear to me that Google is in a great place to monetize and capitalize on this technology, and I think they will win if their language models are better. (So far, they seem to be way ahead; LaMDA was early 2022, and it's clearly better than ChatGPT.) If Google's version of this service is substantially better, but intersperses ads based on what you're talking about (the Gmail model for ads), would people use this? I think it's clear that consumers would take the better free service that comes with ads, vs. paying ChatGPT or accepting inferior quality.
Let alone the fact that Google can put the assistant onto billions of Android phones, fine-tune a model per user, offload compute power with device-based inference to save OpEx, and so on; all of these will give whoever is running the AI agent a lot more ad targeting power.
Current user workflow:
- ask question in google.
- get shitty results
- checkout 5 pages worth of results are try a few more searches. In the process you've seen 5x the ads you would've maybe 10 years ago when the results were better.
- In the middle of this you maybe clicked on 3 or 4 sites that were spam sites which themselves had on adsense ads.
New workflow:
- Ask google a question.
- Immediately get a detailed thought out prospectus, or presentation, or whatever with a top-down overview of what you wanted, maybe you're still curious so you ask a couple follow up questions. Unless they put an ad between every sentence, you'd only have seen 1/10th the ads in the 3 replies it took to come close to google search.
Source: I've run a digital ad agency for 8 years.
Google currently controls ~90% of the search market. AI-driven chat/search is a serious threat to this dominance. It's likely that after the market settles it won't have the same marketshare. Given how much Google has been dependent on Search/Ads and its other failures to execute, this is a serious revenue threat.
This industry (and site) does have a tendency to exaggerate and take a current trend too far. Google is far too massive to 'die'. I believe even keyword-based search will survive. But going from 90% to 50% will be bad for Google.
IMHO, the worst case realistic scenario is this: Google loses a lot of funds, is forced to close more unprofitable projects. This causes more lack of trust, and more projects are closed. Eventually Google is kicked down to a tier below Apple & Microsoft.
Google makes money through ads, and especially ads that get you to click through to somewhere else.
They do this through SEM ads that appear on your search query to direct you to a paid destination.
And they do it through a display network on 3rd party websites that Google search inevitably ends up funneling you to.
If you are simply engaging with an AI that's synthesizing those results so you don't have to, that's less time you spend on those sites seeing Google's ads, and less incentive to click through to paid results.
Their entire business model basically goes up in smoke if AI successfully intermediates the Internet.
This doesn't preclude them from competing, as you point out, but you generally don't want to see your cash cow get slaughtered and then suddenly be in a highly competitive market for what will replace it.
Having a 90% market share in the Titanic isn't an enviable position.
Ask jeeves back in the day already knew that what people really want is a question answered. Google search and its competitors were a well-lived offramp on that road. Ultimately free-form interaction is just more intuitive.
But with that said, I also am not so quick to call the death of SEM ads. Just because chatbots exist doesn't mean people don't want to visit other websites. Display ads will continue to be a thing.
Similarly, there's no reason chatbots can't now direct you to sites or advertise products as part of their responses. Heck, this is a much more sinister form of marketing with astronomically higher click rate since a chatbot is responding authoritatively with a recommendation.
Yes, google is going to have to pivot... but this is a problem they're very well suited to solving and they have a very strong incumbent advantage in the meantime.
1 - Natural Language with prepositions and easy ways to include, exclude and filter 2 - Refinement - "No that wasn't quite right because X - please factor this in and try again" is a lot more intuitive than multiple rounds of operator uses and "memory exclusion" of pages you have already seen.
I find that chatGPT will give me what I need within a few iterations, Google search sometimes takes a lot of searching and reading to get an idea of what I need.
I feel like ChatGPT + Github Code search could be a killer combination for programmers
People thought google has no time to develop something like ChatGPT when they were totally wrong.
And this is not even a problem of scale, it is obviously difficult to change course for a giant supertanker, but the most insidious problem is the money makers inside the company, they usually have a lot of power and they won't allow anyone to butcher their margins.
Killing the cash cow is difficult. I don't see Google taking the risk.
It absolutely has the same problem as Self-driving and only after 10 years we have accepted that it still is off for a very long time.
Mercedes is doing Marketing by calling it self driving, but limiting it to areas the car seems to have a 99% understanding off. They are betting on the fact that they will make more money on the 99% buying these cars, then the few cars that inevitably will crash.
Essentially how insurance has worked for ever.
It does if you remember that Google is in the search ads business, not in the search business.
I guess you could find a way to weasel in ad copy into ChatGPT's answers, but that will kinda massively kill the vibe.
Related: I'd be quite worried if I was a Q/A site like StackOverflow or Quora.
That's a large part of HN in a nutshell.
Imagine analyzing a massive code base, sure it can tell you how you where solving function ex by translating it to natural language, but it still does not understand any of it.
As far as i know, training it on your dataset will not improve this.
This is probably the misunderstanding: the LLM can only automate giving answers because it has been trained on all of SO (or other similar communities). It's a summary of SO, not an alternative to it. When new problems arise, the LLM will need to be re-trained to include the new SO answers, it will not be able to synthesize new knowledge.
So, if SO is dead, the LLM can't get the info anymore to answer questions about new topic - but you won't be able to tell for a long time, long enough to kill SO most likely (assuming this actually gets traction, of course).
The whole premise of the economics of these LLMs is built on the assumption that the training data is (mostly) free. If you need to pay people to provide the training input, you will quickly find that you're spending more money on creating the training data than you're getting out of the finished model.
I mean, they could make a game, where people have to try to beat the AI (and other humans) in making the best answers to questions.
But that doesn't mean other businesses can't fill that reduced role more efficiently.
Please show your work for the second part. Seems like a general statement but don't see that reflected in reality.
The bar has raised now. People will ask the questions on SO whose answers they couldn't find from anywhere else including AI.
we will all become servants to the giant AI, feeding it more and more levels of detail and obscurity
We are just starting to observe societal effects of social media. We haven't reached the era where we, as an entire species, recognise and regulate it impacts legally & properly.
I am starting to feel like we are losing it against the machine as a species. I don't fear being replaced, but I feel the culture getting mangled in a way we won't be able to recover some things, because it will be too late.
It is not a ring-wing, left-wing political thing. But some sort of innovator's dilemma. We are like cornering ourselves into an innovator's dilemma as a species.
The next evolution towards general AIs would be the implementation of curiosity.
If people are asking questions and don't get one, then they'll still seek out an answer? I could see a cycle where Q&A sites get eaten by Google but as long as there is demand for fresh answers there will be services that fullfil them.
People will still seek out sites to post questions requiring context and domain issues that AI won't ever fully address. Plus the human instinct for social interaction and asking the same question already solved 5x before.
It doesn't have to be a Stackoverflow tier business but forums will remain a thing and there's plenty of reputation networks outside of Q&A.
I can get 90% of Yelp's resturant info via Google SERP (menus, hours, location, reviews) but I still use Yelp all the time and as a business they are doing fine.
This AI stuff will be a similar thin layer on top for quick answers, but niche content sites will still flourish IMO. Just with less traffic for a whole new classes.
Wasn’t there a law suit around this? Sure you can get more info but how many will go that far?
You can already nowadays "google" for an answers instead of asking them on StackOverflow. So what is the difference to the situation we already have?
This is actually really huge. If done right, Google will be increasing the amount of "no-click" searches an incredible amount. I'm interested to see how good of a job the "factual grounding" works - this linked blog post in the article is pretty interesting https://ai.googleblog.com/2022/01/lamda-towards-safe-grounde...
Using the monopoly on search to dictate how you shall present your content to the google god is still a bad thing.
It could actually be a huge benefit in some ways if it chokes out the content mills. However, something tells me that they have little to no overhead compared to the people who actually toil away to post good, original content.
The problem right now is that the incentives have caused most of the output online to be garbage.
- encourage more content because PROFIT
- content becomes garbage
- have to pay those pesky content creators
- slowly squeeze out entire industries by inlining more and more content
- still, not squeezing the juice all the way
- introduce "AI", it just laundries copyrighted content to look original
- bye creators
- for some odd reason people cheer you for this
- creators forced to make their content private
Are you kidding me?
People need to make a living, creating good content isn't easy. Sure a few folks do it for free, but wholesale trying to kill off everyone who does it by making it financially non-viable is long term idiotic. What are you going to train on once everyone stops writing or letting your scrape their data to train from?
Search engines scraping your content is an agreement that they can look at it and use it, and send you traffic if it matches well with a user. Why would anyone subscribe to a deal that there is literally zero benefit except training some AI which will repurpose your knowledge.
Most people that make good content don't make it for money anyway. Did people back in the pre-google days think "oh I'd make this site but gosh darn there's nobody to pay me for it". They just went and made the site regardless.
Google is a huge part of the reason the old web doesn't exist anymore. Artisanal websites cannot compete for visibility against corporate websites that have staff dedicated to figuring out SEO tricks from every imaginable source: page speed, HTTPS, image compression, meta tags.
The hobbyist back then didn't need to know all this. Today, not having HTTPS alone can cause your site to be hidden from search, even if it is read-only. In that kind of world, only the infinitesimally small minority will bother to make a website on their own dime.
Sites with complex or lengthy information will not fall to LLMs, IMO. No one interested in reading Paul Graham's blog posts is going to just read the AI summary and move on, for example.
But another issue is accuracy. Of course real sites aren’t always accurate, but they’re way more reliable than AI (and sometimes the site is ground truth like official docs so it can be trusted…unless…the official docs are wrong….).
But from a "this is stealing" type of perspective? Nah, we're all standing on the shoulders of giants and everything we do is a remix of something else we've seen. A human can read my site and take "away" the content, just not at this scale. And they're more likely than a dead chat bot to let me know if my content has been valuable to them.
Conversing with chatGPT is just giving me information and facts. I don’t see it replacing the experience of engaging with a website.
BULLISH for GOOGL
If you asked your friend, who's a serious musician, and they gave this kind of couched answer, you would be really annoyed at them.
That kind of move could be the actual “Google Search killer”
Despite this blog obviously being written for Wall St, this line is key. This is why anyone is going to struggle to compete with Google in AI.
Google Search is used so widely and its taxonomy of meaning so vast already that once they flip this on they’ll be unstoppable. (Assuming any rate of improvement whatsoever as we search.)
I’m not mentioning all other learning data like email, photos and drive, in part because other companies have similar, albeit in more specialised forms.
Depending on how slimy they’re willing to be I’d argue there’s even more scope to push ads when “advising” user through a grey area vs straight up keyword search.
Think going several steps ahead of initial q, increased trust and reliance on responses, multiple follow up questions with more ads etc.
I’m just envisaging this based on my time with ChatGPT these last few weeks.
User: What should I be doing today?
AI: You have a court appearance at 9am. Prioritize that, because failing to appear might result in an arrest warrant. Next, prioritize your mums birthday - all the other family will be there, and with your mums cancer results last week, this might be the last. If you travel between them by bus, then you can spend the time going through your teams messages that you should have replied to last week.
Me: What about the leaking faucet in my house?
AI: You don't have the time to fix it yourself, nor the money to pay a plumber to do it, so I suggest you leave it leaking for now. I watched the video - its leaking down the basin, so won't do damage, and the water cost is around 8 cents a day.
At that point the line between commerce and giving the best advice is going to get REALLY messy.
Pretty creepy and a tad dystopian but thats Google for you.
From what I understand about LLMs - not directly. But it may be possible to integrate LLMs with other services such that LLMs respond this way.
I have no idea.
But it would be nice?* if this LLM stuff could be incorporated into the telephone system for my bank, pharmacy, etc.
It's obvious that these organizations don't want to connect me with a $killed human, so instead of having me interact with an infuriating "pretend" human, maybe CVS can fuse their phone system with ChatGPT to make the experience a little less maddening.
"Hey, I already gave you my birth date. No need to ask again. And I told you 30 seconds ago that I don't need to schedule a COVID vaccine. Just tell the pharmacist <xyz>."
* Be careful what you (I) wish for, I guess.
Specifically, LLM's can generally only take into account ~8000 words of context when deciding on a response. Summarizing the question and all necessary information for the answer into 8000 words is hard when the user might have millions of words in their inbox.
Having said that, I don't think it's far off. There are already prototypes of LLM's which have information retrieval abilities (ie. it could do a keyword search of your inbox to find a few relevant documents to read to decide on a response). There are also promising efforts to make that 8000 word number far larger.
AI: You have a court appearance today, I have booked a route for you on google maps. Next buy a gift for your mum’s birthday, here are some recommendations sorted by ad spend spend. For your travel, I recommend these options, sorted by ad spend.
Me: What about the leaking faucet in my house?
AI: You don't have the time to fix it yourself, here is a list of contractors that can fix the leak, sorted by google ad spend.
Most projects are focused on privacy, user control and being censorship resistant. Which are all important, just not that important to the majority of everyday consumers who will take features and convenience over those other benefits. If decentralized AI is going to be competitive, it must actually be better along the lines of its actual features that enhance productivity.
I'm contributing to the project by running a node in my garage with a single RTX 3060ti in it, and you can too: https://github.com/bigscience-workshop/petals
It's early days, but the tech is super promising.
Decentralized search and social media never gained traction. It turns out that the majority of people just aren't that interested in privacy and freedom. Features and convenience still win. If decentralized AI is going to be successful, it will have to compete head to head for features and convenience. I do hope that it is successful.
Nonetheless, decentralization is just one major hurdle. I have numerous concerns about the application of the tech overall. Too much to put into this post, but if you are interested I've written much more here.
2. Wall St tends to be very forgiving when the investment is something with a strong a moat as AI. Metaverse is an example of what happens when they don’t believe…
3. With a few exceptions like weather, their current searches provide links to static pages others have built, so much of their understanding and analysis is going to waste. Giving custom results breaks through that.
But perhaps only for now. When youtube first started taking off, Google was bleeding cash on the resources needed to support it. Perhaps the same will be true for LLMs.
I think they are too "big" to really push AI, it needs smaller companies willing to take a higher risk.
But with China, Russia and India lurking, perhaps Congress might drag their feet on reigning AI in. Google Cloud are now poised to be going for some pretty decent govt contracts.
On the flip side, trying to make facts copyrightable seems like a terrible idea for all sorts of reasons. For example, if facts were copyrightable, that would make online discussion of factual stories illegal, since it's hard to discuss a fact without revealing what it is. Also, it's not always clear who should get "credit" for a fact, since facts are by their very nature true independent of who first reported them.
Maybe journalism will eventually become similar to academia, where journalists are funded by governments and large corporations who have an interest in learning, and the resulting discoveries are (ideally) made freely available for everyone to access?
They can break antitrust laws if they go too much in a direction of delivering outright replies instead of redirecting to websites.
Take the difference between Microsoft and Google. Microsoft just released tools that will make your life much easier. Intelligent Recap of teams meetings, ability to assign tasks based on what was discussed etc. Microsoft will announce bing with have chatgpt integration tomorrow. MSFT is eating Goog’s lunch and Sundar needs to go.
Unstoppable as measured by what? More ad revenue?
They're already the #1 search engine due to two decades of general goodwill and a decade of monopoly power in mobile.
Can this be integrated for training though?
Yeah, remember Google+? they tried very hard and resorted to very intrusive UX patterns, but they ultimately failed.
IMO this is similar, ChatGPT has a huge head start, and Google needs to do A LOT of work.
"What does this word mean"?
"Look it up in the dictionary"
"Why can't you just tell me??"
I remember the hassle of looking things up in a paper dictionary. Over the long term, it massively helped improve my ability to not just recall a simple definition, but also learn new words, as they were adjacent to what I was looking for.
It's effectively this hassle ChatGPT and Bard are 'solving'. Instead of a group of links that you'd personally evaluate and mentally rank in terms of usefulness (thereby building your own bullshit detector over time), you will now have a chatbot AIsplaining things to you like you were a 12-year old, regardless of the topic's complexity.
I'll admit I was concerned about about Grammarly.com ruining people's ability to learn to write. That pales in comparison to an opaque search engine promising to give you the keys to all of human knowledge.
This sounds an awful lot like email's "Spam" folder, an AI driven bullshit detector that I am particularly fond of.
What about the AI that drives YouTube auto play so somehow kids watch horror clips and everyone else gets nudged towards Jordan Peterson for some reason?
Having more of our internet and communication filtered by ostensibly “neutral” AI will get lead very weird and probably not great places.
FWIW, I'm not sure I've ever done that and I've learned two languages (outside of a classroom environment, which I expect makes a difference). I usually just encounter words in books and infer their meaning from context. As I encounter the word more and more and tried to use it myself I gained an understanding of what it meant.
It is an interesting bootstrapping problem.
It feels like a mistake to make the big announcement for this, but not open it up to a wide audience. It's not like ChatGPT hasn't been out for months now. Not sure how much they gain by making headlines before it's ready for people to play with it.
But that feels like slow old thinking. The way you create a buzz these days (in the era of limited attention) is by releasing a kick-ass product for people to use and play with. I'll probably forget about Bard the second I hit submit on this comment and go right back to using ChatGPT.
Meta/FaceBook's head of AI, Yann LeCun, is in similar panic mode, issuing a non-stop torrent of tweets about how useless and unimpressive ChatGPT has... Presumably a reflection that FaceBook does NOT have anything comparable ready to release anytime soon.
OpenAI and Stability-AI will be be Apple and Microsoft.
Some of it is marketing bullshit, but Facebook seems to have a genuine interest in pushing the field forward. Admittedly Google is also not the best example though, seeing as they're the Tensorflow maintainers :p
The issue isn’t that Yann didn’t publish an AI app; it is that they did, and it was not as good.
The reality is that OpenAI was lucky. Inside the company, there is an alignment department whose effort was driven to help models share human ethics, and that was initially a bit marginalized. However, one of their projects, RLHF, ended up producing a much superior language model, when they could have initially assumed it would be worse.
I don't see the same thing happening between Google and Bing
ChatGPT is awesome, but it is obviously hamstrung by the fact that it crawled the web at the end of 2021, so all of its data is essentially "frozen" and it doesn't "know" of any topics that occurred after 2021.
Seems to me like the next "holy grail" in large language models is building a model that can be continually updated. If Google can achieve that, I think they could leapfrog OpenAI. The example in the post about getting new information about the Webb space telescope seems to imply Google has this advantage.
So my question is, how hard of a problem is that with LLMs? I get the sense that LLMs are trained on a very large data set all at once, but that it is difficult to incrementally update them with new data. Is that a true assessment? Even incrementally updating an inverse text index can be a scalability challenge, and so it seems like, given the way that LLMs are trained, that it would be even harder to do with LLMs without spending an absolute fortune on training.
That sounds like a task runner instead of a LLM
Using ChatGPT as a front end for search, maybe also as a summarizing/presentation interface, and able to maintain context for a conversational interface would all seem to be playing to it's strengths.
There was apparently a brief appearance of a ChatGPT enhanced Bing (search engine) yesterday which was noted to be able to cite sources, which certainly suggest it being used more as a front end than as itself the source of content.
I think it depends on how reliable you want this AI to be. Opening search indexing to realtime data is what led to the endless mess we call SEO, and I fear that doing the same for AI invites the same problems.
I think this could be a feature. Things on the internet have a life cycle. After a while they can be altered to fit various agendas or just drowned out by seo nonsense. Seeing an immutable snapshot in time might be a good defense of that.
For the large proportion of search queries that are "give me context for $newsitem", "what is $latestthing", "what is $celebrity up to", "can you solve my problem with the latest version of x" it's a show stopping bug (frankly Google weighting established content higher in most contexts is already an issue for the last of those queries: Googling error messages etc). And at least all the SEO'd "Best Items In $CurrentYear" articles that aren't that up to date attempt to look current
[1] https://www.deepmind.com/blog/improving-language-models-by-r...
Wow. Talk about going through contortions to get an acronym.
https://techcrunch.com/2022/12/20/this-autonomous-ornithopte...
Cool project though.
Example to try that in action:
It gave me news from April 9, 2022. What should I give it?
Edit: Late comment because dang put me on the rate limit list and didn’t tell me.
https://labs.kagi.com/ai/context?question=what+is+the+latest...?
(totally speculative) If Bard were to form queries against knowledge graph, and then summarize the results, it could be very up to date for a good amount of information
The more likely dependency would be bard ingesting unstructured data and generating structured data to update knowledge graph with.
For example I asked chatGPT about birds that can't fly and it started bullshitting about all sorts of birds that clearly can, those facts would likely be in the KG.
I asked it, conversationally as I would to a human, "when is the super bowl", "what channel is it on", and "who is playing" in that order, and it answered all three perfectly.
For fun, I asked it a more complex question - "will ford stock go up or down", and it answered with -
Analysts have a median target of $13.00 for Ford stock in the next 12 months[1], with a high estimate of $21.00 and a low estimate of $10.00[1]. Over the past 50 years, Ford Motor has on average risen by 15.7% over the course of one year[2]. Therefore, it is likely that Ford stock will go up in the next year[3][4].
I'm going to use the heck out of this thing.
> Perplexity: The three horses and duck have a total of 24 legs[1] [2]. However, the question only asks for the number of legs on the floor, which is 4[3][4]
Updating a monolithic LLM seems like a harder problem at the moment, and probably would fall under the umbrella of "continual learning", though that sub-field would have their own subdivisions of different methods.
As for how they would potentially perform, those results would be mostly empirically measured, but neither approach would completely remove the possibility of (1) missing results -- which is probably acceptable given Google doesn't always return everything relevant, and (2) hallucinating non-factual responses -- which can be more dangerous, if not from directly instigating / causing harm, then from the incessant worry that it might.
I was under the impression it went through like a multi-month long ultra expensive training process involving many GPUs on terabytes of data "snapshotted" point in time.
Is it possible to do this at a smaller scale once a day at the end of the day with "all new content scraped from the Internet nightly"?
If the model has the ability to browse the web (the tech behind ChatGPT by design does, but it is disabled in ChatGPT proper, which is, in some respects, a conservative public demo) and incorporate data in responses, this can “cover” a bit for staleness in the base model.
Even if you can't retrain the model fully daily, with enough resources [0] you can have multiple training sessions running concurrently and just swap in new backend models behind the interface as they arr ready, which, combined with browsing ability. acheives something very similar. This obviously, barring an enormous advantage in underlying tech or access to relevant training data for one player, works most in favor of whoever can subsidize the biggest hardware commitment.
(Given the different training stages, there may be cost efficiency advantages to, say, running less-frequent iterations of the lowest-level training stage but more frequent iterations of the higher-level ones.)
[0] probably an utterly ludicrous investment for all but a handful of firms, but for a ~$trillion firm where this hits very close to their core business? Not so ludicrous.
There are existing problems with that, one keyword to lookup is 'catastrophic forgetting', where you update on recent stuff so much that you overfit on that and forget all the prior stuff you learned before. There are methods to mitigate that, but I would call that an 'active area of research'.
Catch is: consider these statements:
Donald Trump is the President.
Joe Biden is the President.
Barack Obama is the President.
George W Bush is the President.
George H. W. Bush is the President.
Bill Clinton is the President.
All of them were true at some point of time. How do you train a model to disambiguate these?
- as said they can be updated with fresh information on the fly,
- they can give you sources for their results (meaning that you can fact-check the output!),
- they achieve similar performances for significantly smaller models (as long as you have a large dataset to retrieve from) meaning that they could run locally on consumer hardware (coupled with a large dataset on disk or the ability to read information from internet).
If I were to spend a significant amount of time building my own language model, it would be retrieval-based and try to preserve those properties (in particular the ability to run on consumer hardware, enabling developers to run models locally has proved to be a game changer when stable diffusion came out).
Further, depending on the nature of data used, this would make such changes more complex (ex PII)
I asked Jeff about what it was like in the early days and he told me: when they first joined, the entire crawl to index to serving stack was documented in a README that you would follow, typing commands and waiting for each step to complete. A failure in a step meant completely starting over (for that step) or even earlier, depending on how and where temp data was materialized.
He said he and Sanjay (Ghemawat, not Gupta) then wrote mapreduce as a general purpose tool for solving multiple steps in crawl to servable index. Not only is mapreduce good at restarting (if the map output and the shuffle output are persistent), the design of mapreduce naturally lends itself to building an indexing system.
If you go back to the old papers you'll see several technologies mentioned over and over. protocol buffers, recordio, and sstable: the first is an archive format to store large amounts of documents in small number of sharded files, the second is a key-sorted version of the same data (or some transformed version of the data). So, building an inverted index is trivial: your mapper is passed documents and emits key/value pairs (token, document) if a token is in the document. The shuffler automatically handles grouping all the keys, and sorting the values, which produces a fairly well-organized associative table (in the format of sstables).
BigTable came about because managing lots of sstables mutably became challenging. | MapReduce was replaced with Flume, which was far more general and easier to work with, and BigTable was replaced with Spanner (ditto), and GFS replaced with Colossus, but many of the underlying aspects of how things are done at Google in prod are based on what Jeff, Sanjay, and a few others did a long time ago.
Note that mapreduce isn't particularly innovative except the scaling aspects were fairly esoteric at the time.
There's an interesting tradeoff between grounding LLM answers in web contexts and answering questions from parametric memory like ChatGPT. The former is more accurate and verifiable, but the latter can be more creative and specific to you.
We will have both of those answers soon. (Disclaimer: I am the co-founder).
As a slightly disinterested third party, it will be fun the watch the competition.
With recent advances in alignment, this problem has gotten much better. Furthermore, the LLM used in that paper is only ~400 million parameters, which is orders of magnitude smaller than models being used for this task today.
Scale + alignment has been the solution so far.
Google can leapfrog OpenAI because they are the entry point for 99% of users attempts to answer questions online. Similar to how websites for x (weather, lyrics, etc.) existed for years but the second google baked them into search those sites dried up.
But weather, lyrics, etc. websites are specialized at what they are doing and Google is just trying to do basics of aforementioned services in hope they will appeal to casual Web users.
The thing about this is if competition forces Google and Microsoft to offer a full ChatGPT style interface, they are going to be using a vast amount of cpu cycles and thus energy. This is going to be costly and potentially environmentally destructive. And only companies with vast server farms will be in this game.
How much CPU do you think Google currently utilize? OpenAI is a relatively small company compared to Google and they can keep their services (barely) afloat. AI at scale is indeed server intensive, but probably nothing like what we already have.
Also, new chips designs might significantly decrease power requirements in the future, so I wouldn't worry about the environmental issues.
Lots and if they serve a ChatGPT interface, it will be N times as much as now since each will be to an LLM and involve a back-and-forth. What is N? 3, 10, 100?
OpenAI is a relatively small company compared to Google and they can keep their services (barely) afloat.
OpenAI has negligible income. They're effectively financed by Microsoft. I don't how much serving ChatGPT to the world costs but it's fairly expensive per transaction. It can't not be, since it's calling a model with billions of parameters.
For the foreseeable future, LLM will be used for a unique set of queries that are more operational, like a consult, not for stuff that are informational, like what news is published today.
If your query is relying on the model to be update to date with today's news, you can just put it into the prompt.
If they don't do that, they will create an AI that is essentially inbred.
It's the opposite. Getting an LLM to learn the basics of grammar and paragraph-level language structure is the "hard" part. Once you have that, further fine-tuning, specialisation, or other incremental changes are comparatively easy.
Catching up to current events could be done in almost real time, it's "just" continuous training.
The only challenge is trying to do that and also have an LLM that's filtered to exclude profanity, racism, etc...
That filter is usually added on as a final supervised training step, and requires many man-hours to train the AI to be well behaved.
I suspect that it would be possible to automate the filtering by making another AI that can evaluate responses and score them bases on profanity level.
That sounds like InstructGPT
https://i.imgur.com/5CNUm9l.png
I regenerated that response several times, and every single time it was something along these lines. I also tried it with the original prompt in that screenshot used verbatim with similar results.
Looking at other posts on the Twitter account in question, I have my doubts that the experiment was conducted in good faith, as opposed to retrying until they got the exact response that they wanted.
In other words, while sure, some "well behaved" is "passing on our biases", there does (IMO) seem to be a big chunk that's "universally well behaved".
It's hard to get people to understand the disproportionate effort fixated people put into anything: it's a problem in real life, but they get reacted to. If their fixation becomes some weird message on the internet, nothing happens to them but people have trouble believing the scope of time and effort they'll put into evading bans, blocks, and chasing down people across forums.
Going beyond sources to conclusions, given LLMs aren't search engines and do synthesise results:
Politically, low-tax advocates see it as "fair" for people to take home as much as possible of what they earn, high-tax advocates see it as "fair" for broad shoulders to carry the most and also for them to contribute the most back to the societies that enabled them to succeed.
Is the current status of Americans whose ancestors were literally slaves made "fair" by the fact that slavery has ended and all humans are equal in law? Or are there still systematic injustices, created in that era, whose echos today still make things unfair?
Who has the most to blame for climate change, the nations with the largest integrated historical emissions even where most of the people who did the emitting have died of old age, or the largest emitters today?
And so on.
Like - hmm. I could see taking a snapshot after the grammar/language stuff is in, and then every N weeks retraining on the current web, adding in something about recentness, but that doesn't seem like "continuous" training.
I'd imagine "continuous" training would be, well, going on continuously, all the time, but that would mean that, to include "recentness", something would have to change with the weights that were from that "old" stuff, which sounds an awful lot like the human process of "forgetting".
Note that this law is a problem for digital storage too : it's not easy to erase data (especially one stored in cold transistor storage) without physically destroying the storage medium. (I guess the law might get around this by having you "pinky promise" that you will not retrieve the "erased" data later... or else face much more dire legal consequences ??)
I guess this will just need to be tested in courts ?
Continual learning seems to be a tough problem though, from what I'm seeing of my friends working on this problem. Like I said in another comment, just doing gradient updates form new data is fraught with problems. RL has a bunch of techniques to mitigate issues that arise with that, but I think it's still an active area of research.
https://paperswithcode.com/paper/most-language-models-can-be...
If you don't believe this is a problem, try getting ChatGPT to write a paragraph of correct English which omits words which use the letter "e" in it. Too bad you can't use my technique on ChatGPT since they don't expose their output probability distribution...
the problem for Google and OpenAI is that most websites are going to start blocking them in robots.txt if they don't find some way to provide value back for allowing them to scrape and train on their content. Pretty much every other bot or search engine is blocked by default and Cloudflare helps block them too.
if they don't find a way to balance this they are going to kill their own golden goose at some point
At least the Bing ChatGPT integration can get data from websites and all of that, and cite the sources. Probably using REALM. Not sure if is ChatGPT-4 that would use REALM together or if is MSFT integrating both.
https://medium.com/@owenyin/scoop-oh-the-things-youll-do-wit...
Similar to how http://perplexity.ai/ works to get info on current data.
But the leaked Bing Chat feature indeed does show things that for now ChatGPT don't have.
In 2020, OpenAI wrote that training the 175B parameter GPT-3 took 3e23 fl-operations [1]. A $1600 NVIDIA GeForce RTX 4090 [2] can do 80 tflops (80 × 1e12 operations per second). So 1 GPU would need 120 years to train the model.
With $2.4 million, you could buy 1500 GPUs, and train GPT-3 from scratch in 1 month.
Partial continuous re-training should be way cheaper, if there is a way. But even training again from scratch every month seems feasible for a big company.
Disclaimer: Not an AI expert.
[1] https://arxiv.org/abs/2005.14165, table D.1.
[2] https://www.techpowerup.com/gpu-specs/geforce-rtx-4090.c3889
I actually think Microsoft is betting on the wrong dog. But let's see :)
In your opinion, which dog should they be betting on?
Maybe they have the answers for ChatGPTs incorrectness but not really sure I'll be reaching for a product which is imagining the answers as it sees fit?
That's not quite correct. It's trivial to update the model with new information. What's not trivial is to encode priorities or time into the model. Fundamentally, the model doesn't understand about the concept of time, so you can't easily condition it on "give me only information from last year". You can get there and nudge it in the right direction by generating training datasets for such queries and fine-tuning the model on them, by oversampling more recent data in your training procedure, or by adjusting your gradients for more recent data, but all of these are still "fuzzy" - the model may or may not do exactly what you want during prediction time and could still give you old results.
However, I would assume that Google is doing something more sophisticated and that's it's not just a plain LLM like GPT3. It's probably a more complex architecture with external systems and data sources around it that tries to solve the above using heuristics.
That being said, the current Google search is extremely bad at time relevance too...
(source: I helped build Sparrow, DeepMind's RLHF model, that does this by learning to use google under the hood)
Microsoft is rumored to be adding GPT powered features to Bing very soon and might beat Google to market.
Very interesting times!
Most of the people using "computers" nowadays do it through mobile phones, and Google has a strong grip in that market with Android, with only Apple giving them some competition. Thus the only platform where Microsoft could really compete is in iOS.
If Microsoft + Apple could get together in this one, they could do a really killer app doing a next level Siri or similar.
Otherwise, What would Microsoft do? Add ChatGPT to Windows? that means PCs ... that means a Office/work related agent, which is boring and kind of reminds me of the Mac vs PC video-ads of 2000s.
Otherwise it will be just like Magic Leap with a pretty cool product nobody ever used.
Microsoft would then need to pay for ChatGPT computation, model retraining, maybe testing of deployments/upgrades, maybe sponsor Apple's initial development efforts, plus pay for the expensive deal to be Apple's default choice.
Am I understanding the problem correctly? It looks like a lot of work and very expensive if I'm getting it right.
With search, Google pays Apple with the payoff being that people end up seeing Google's ads. With ChatGPT, Microsoft wouldn't be getting money so Apple would presumably need to pay Microsoft.
You've phrased it like Microsoft needs to get it on people's phones rather than Apple needing a competitor to Google's NLP (Microsoft should do whatever it can, not Apple should do whatever it can which implies that Microsoft should pay Apple). However, we haven't really seen what ChatGPT is in terms of a product yet (product, not feature).
Let's say Microsoft pays Apple to get ChatGPT onto iPhones. What does Microsoft get out of that? Bragging rights? Ads in Siri? Or maybe as part of an overall deal of Bing + ChatGPT?
I agree that things like ChatGPT are cool, but I think it's unclear (at least to me) a company like Microsoft will make money off it without charging for it (or making it terrible like allowing companies to pay for placement in ChatGPT responses like "what's the best vacuum?" gets you "The best vacuum by far is ProductPlacement. It has much better suction than CompetitorProduct"). Amazon has tried using Alexa to upsell things, but it can make Alexa really annoying at times.
They could come up with a really cool next-level Siri, but I guess I'm not sure what's in it for Microsoft in such a deal. I see what's in it for Apple and users.
AI is now the biggest game in town - Apple is not going to leave it to Microsoft and Google.
The LaMDA paper describes 2B, 8B, and 137B parameter variations. Sundar says "We're releasing initially with our lightweight model version of LaMDA. This much smaller model..."
So it sounds like the 2B or 8B model variations, compared to GPT3's 175B. [edit: corrected]
I can't imagine such a smaller model coming across as anything near as impressive as ChatGPT.
So the announcement today is for a much smaller, more limited thing than ChatGPT and even that is not actually available today nor even with an announced release date.
Src: https://labs.kagi.com/ai/sum?url=https://blog.google/technol...
But yes, first movers don’t always win. They have an advantage when there are network effects, but there’s a lot more to it.
At least Google can pivot a new version of adsense easily with this products.
Technology!=business viability
That's what many first mover don't think about
> Sundar Pichai, CEO of Google and Alphabet, has announced the release of Bard, an experimental conversational AI service powered by Google's Language Model for Dialogue Applications (LaMDA). Bard seeks to combine the breadth of the world's knowledge with the power, intelligence and creativity of Google's large language models. It draws on information from the web to provide fresh, high-quality responses. Bard is initially being released with a lightweight model version of LaMDA, which requires significantly less computing power and will allow for more feedback. Google is also working to bring its latest AI advancements into its products, starting with Search, and will soon be onboarding individual developers, creators and enterprises to try its Generative Language API. Google is committed to developing AI responsibly and will continue to be bold with innovation and responsible in its approach.
The point missed is that is that they're starting to test Bard with "trusted testers," their closed beta program, so it's not really "released" yet.
The context is that they had to announce it now because it's likely to leak anyway; trying to get outside testers to keep it confidential would be difficult and not worth it.
I’m sure Google will release something good, but it will be called: Google’s ChatGPT.
Should have made something to distinguish their AI solutions from just being a copy to trendy solution.
Saying AI will be used in search is not news, AI is already part of Google search.
That presents dangers to society. Educating citizens on the ubiquity of human-like text being generated by bots is one challenge. Knowing that bad actors will be able to use these models for evil is another. "Write a convincing 10 page manifesto on the dangers this religious minority poses to societal cohesion" or "tell me a cost effective way to create an IED that won't set off suspected red flags per Texas statutes, and how to deploy it in an urban area for maximum impact".
The risk of this technology being contained to Google and governments, on the other hand, is to grant them monopoly on unmatched analysis of data in the history of the world, purely for profit and the continuation of power. Free thought will be crippled via only approved questions being permitted, all queries analyzes and monitored by the central authority.
We must all own this technology.
The most dangerous thing for society is perpetuating the idea of good and evil because that is what motivates war and terrorism. It is used as propaganda to make conflicts into moral issue and justify mass killing. What do you think the terrorists are fighting against? Evil (in their minds).
The real problem is that there are different worldviews and political groups, a lack of metacognition about this, and governments or dissident groups using this to enable their propaganda to further their political causes.
Worldview and group membership are tied together to a large degree.
Also as far as being open source, we already have open source GPT systems. The problem is that the most powerful models require dataset sizes that cannot currently train or run on normal GPU setups. Only a few companies have the infrastructure for it.
Eventually we will get more efficient models, and/or possibly a way to do it with decentralized GPU mesh (assuming that's possible), and/or memristor manufacturing will start scaling up which will make it possible to run these large models on consumer hardware someday.
https://www.frontiersin.org/articles/10.3389/fnano.2021.6459...
It is similar to what the normal "Google" app offers: snippets with links to articles which may either be currently relevant for me (currently as in: this day you searched a lot for X, so we recommend you Y), or which may be a bit older but still of interest to me. A personalized Google News but much less news oriented.
I don't use this in the "Google" app, because there the link-sharing is broken: it only shares the link, while in Google Chrome the share button shares the link but also provides the title of the article.
The point is that I want have this "Discover" feature on Google Chrome on the desktop, not only on the phone or on the tablet. I don't use my phone for browsing and my tablet only when in bed. I'm a desktop-first person. There I have 3 monitors, a surface where I can place my cup tea and my pencil.
Google is telling me: We don't want your behavior. We want you to use a phone or a tablet, we will reward you for this behavior by granting you access to the our analysis of the information which we aggregated about you. Else we'll punish you by not giving you this tool.
I wonder where they will be offering Bard, and at this point, I'm close to considering using Bing, which was always out of question for me.
The current message from their CEO: who cares? Publish it, let us see for ourselves what it can do.
First off, I have no idea what Google uses AI for today in any of its products. Google today is about as useful (or useless, perhaps, since I’ve switched search engines) to me as it was 10 years ago. I don’t care, either. If I don’t find myself thinking “I really need AI in Google’s products to achieve this”, why should I care?
> But increasingly, people are turning to Google for deeper insights and understanding — like, “is the piano or guitar easier to learn, and how much practice does each need?”
I want my search engine to search the web for that information. I have no interest in it amalgamating that data for me and performing who-knows-what transformations to it before serving me the data with no traceability to primary sources and no way to validate it without redoing the job myself. I can’t be the only one who wants their search engine to search, can I?
> In 2018, Google was one of the first companies to publish a set of AI Principles.
Google was also one of the first companies to fire an AI researcher for what he said about the AI he developed. Google also has shown zero regard for ethics as it has exploited monopoly position in ad markets and consumed small companies to squash competition. I have no trust in Google.
Maybe I've just been tuned to use Google effectively, but Bing/DuckDuckGo results are horrid except for very simple things, while Google gets me exactly what I want in the first ~5 results.
If you are thinking of the guy who said the LLM was alive, Timnit Gebru was earlier and she was fired from their AI ethics team for levying fair criticism of their work in a research paper [1]. I find that situation more of an indictment of their compromised AI ethics.
[1] https://en.wikipedia.org/wiki/Timnit_Gebru#Exit_from_Google
Microsoft missed the arrival of the web. Microsoft also missed the mobile revolution and ended up with zero mobile operating system market share.
Internet Explorer, having vanquished Netscape, declared "job done" and stopped developing the web browser further, only to be cast to irrelevance for its failure to advance and innovate. Much like Google search.
Kodak, Yahoo, Commodore, MySpace etc etc etc all missed the critical technology change that left their business behind.
Despite seeing the oncoming train, Google may be too arrogant, too internally political, too controlled by it's vast river of web advertising gold, to be willing or able to make the changes needed.
And it may be that Google simply cannot transition it's search over to becoming top dog in AI information seeking - it's possible that AI interfaces aren't a "winner take all" market like search is. If that turns out to be true then Google won't be finished but it will be diminished.
ChatGPT brings a laser focus that Google simply cannot.
It's possible Google's time has passed.
I remember distinctly when Google arrived how instantly old and out of date Altavista suddenly looked - that's how Google search looks now.
That said, Microsoft is still the second most valuable company in the world a decade after missing out on mobile, so it doesn't really seem like a big deal.
Facebook doesn't make any money when people talk to each other, they make it when people click ads. Same goes for Google. "Social Media" is just another ad vector.
Also, there are still many categories of things that ChatGPT-like bots can't help with yet, such as shopping. I think we're still very early in this cycle. That doesn't mean Google will succeed, but it feels premature to be writing their obituary.
O365 etc. survive due to corporate inertia (Excel), but I wonder if growth stems from new customers vs. big, large customers growing and requiring more licenses while actively looking for replacements.
Azure is good, sure, but AWS is better (yeah, the console) and google will probably do anything to not go into irrelevance in this area, so they're stuck between a rock and a hard place.
I still have MS stock (with a large payoff atm), but I honestly wonder when it's time to sell.
The point GP is making is that Google's technology has been (at least to a non-expert) on-par with the external things. Why would they acquire something they already have?
This is ultimately only an issue if you think Google's models are significantly worse (and that's related to the lack of public testing). That assumption doesn't seem justified.
Like, those are all examples of companies which failed to adapt to some new reality. This announcement is Google seeing AI as the next thing and proactively engaging with it.
I think Google has enough capital, brand recognition, and customer dependence, that they could get away with missing some of the AI hype train for now.
I can search for, find, and order virtually anything I want in less than a minute. Unless an AI is going to anticipate exactly what I need, order it for me, and have it delivered before I even think about it, I don’t see these two things competing much.
Comparison shopping is a major PITA with current sites, clicking back-and-forth between various product pages and review sites, and trying to distill it down to a few relevant choices -- things that ChatGPT's excellent abilities at summarization could really help with.
Google time has passed?
They still have 7-8 products with more than a billion monthly users.
If you go back and look at the company you listed. They die because they were poorly ran companies. Google isn't a poorly ran company.
OTOH, Google got scooped when in January 2007, Apple changed computing by launching the iPhone.
But, though it had not been announced, Google already had the Android project in progress, which pivoted to a more iPhone-like concept (full-device touchscreen, no keyboard) and ended up doing OK in the market.
Presumably you can see the parallel I'm trying to make with today's situation. Obviously it doesn't prove anything, but if we're looking at past history as a guide, this is something that also happened.
Thanks in no small part to Samsung. Google honestly would probably not have gone beyond their initial Blackberry-style design approach without the iPhone. And Samsung's Android flavour pushed hardware capabilities way past what Google provided out of the box.
How did Microsoft miss the arrival of the web and have the dominant web browser for years?
> "Microsoft also missed the mobile revolution and ended up with zero mobile operating system market share."
That's kind of disrespectful to the competitors who duked it out during the pre-iPhone phase of the mobile revolution. Microsoft competed with and defeated Palm Computing, the market leader, and others in the marketplace to become the top dog in PDAs and smartphones around the release of Windows Mobile 5 and 6. They didn't "miss" anything.
Apple, to their credit, managed to hit the right combination of timing and consumer appeal that they swept everyone away, Microsoft, Blackberry (remember them?), and what was left of Palm. The iPhone would probably have killed Palm even if they'd managed to fend of Microsoft.
That's a well known part of computing history. https://www.inc-aus.com/tess-townsend/what-bill-gates-got-wr...
I remember getting a Windows Mobile phone and loved it. Their UI (tiling etc was cool) was superior to iOS and Android at the time ( at least for me). I was sad to see them disappear.
I'm not sure how any Google representative can make such a statement with a straight face at this point. Google has such an abysmal history of pulling the rug out from under third party developers (and users) that it seems irresponsible to trust Google at this point.
Q: Is this boilerplate?
AI is the most profound technology we are working on today. Whether it’s helping doctors detect diseases earlier or enabling people to access information in their own language, AI helps people, businesses and communities unlock their potential. And it opens up new opportunities that could significantly improve billions of lives. That’s why we re-oriented the company around AI six years ago — and why we see it as the most important way we can deliver on our mission: to organize the world’s information and make it universally accessible and useful.
A: Yes, this text is a boilerplate.
Q: Why?
A: This text appears to be a standardized or formulaic description of the value of AI technology and its potential impact, likely used as a marketing or promotional tool by a company. Boilerplate language is often used to convey a consistent message across various platforms or materials.
However, an interesting piece of info was
> It draws on information from the web to provide fresh, high-quality responses.
which seems like it will possibly be able to use some kind of external repository of information, instead of needing to be retrained? Very curious about that.
This seems to be the approach https://www.perplexity.ai/ is taking.
For much more in depth take, I wrote recently about that here. https://dakara.substack.com/p/ai-and-the-end-to-all-things
More concisely, it's someone that's good with words.
> The way I've come to look at ChatGPT is via a D&D analogy.
> It's like a helpful Bard with 1 rank in all the knowledge skills and a good bluff roll.
> It'll give you good answers to a lot of basic queries, but if it doesn't know, it'll just make up something and provide that.
"The story concerns a pair of boys who dismantle and upgrade an old Bard, a child's computer whose sole function is to generate random fairy tales. The boys download a book about computers into the Bard's memory in an attempt to expand its vocabulary, but the Bard simply incorporates computers into its standard fairy tale repertoire..."
What are Shakespeare's works best known for? Language.
I was wondering who is the audience of this article. Hasn't Google search been powered by numerous machine learning algorithms for years? Another thing I don't understand is that why Google's product strategy is like Baidu's: AI First. I mention Baidu because they are infamous for not having a product vision. Since when a technology itself could be a product strategy? Information at your finger tips is a product strategy. Organizing the world's information is a product strategy. AI is just a means to many ends, right?
Shareholders.
Then they started scraping the destination pages and present the answer right there on Google's own results page. No traffic for you. Well at least there was a link.
Now they take it one step further, and just steal the content and feed it into their gargantuan pattern matcher, to be spit out to users in a remixed form, swirled up with a bunch of similarly scraped content from the open web. Great for Google, RIP content creators.
There are lots of non-commercial random blogs out there with useful information on long-tail topics. I agree that those will disappear from search, as AI will put a lower weight on their posts compared to something from a major website.
The way I see it, Google:
* is effectively the gateway to the internet, owning the most popular website in the world, mobile OS, and browser
* has infinite LIVE data, both unstructured and structured (knowledge graph)
* has infinite compute resources
* employs the world’s best AI scientists and invests heavily in AI
* has decade(s) of experience in AI products
* has one of the most popular suit of apps already in people’s hands (gmail, calendar, maps, docs, YT, drive, wallet, photos etc etc)
* is a popular cloud provider (which is another vector, ie platformizing AI)
* has infinite money
* is probably the most well positioned company to actually monetize AI
Honestly I don’t see why HN seems to think it’s over for Google. Outside of tech, most people have never heard of ChatGPT or Bing. Everyone on the planet knows Google.
That's great and all but that's not where the money is. Where is the money? It's in queries like "greek food near me", "best earbuds 2023", "replace sparkplug Honda motorcycle." Simple, monetizable questions that are extremely context dependent and constantly changing.
For that you'll continue to need search.
Basically, what if Google's little info boxes weren't garbage? That alone would be a huge step, without even getting into potentially more sophisticated ways to use an LLM.
Google does have the means and knowledge, but they are not as good at monetizing them. See also the cloud space and Google Cloud vs Azure.
But I also think that Microsoft will be a huge winner for enterprise software. They are already integrating GPT into Teams, and no doubt they plan to integrate it into pretty much everything they can think of.
I hope we will see a lot of independent startups making innovative AI products as well. This space has incredible potential, and there could be many winners.
Perhaps it creates specialization for the "internet gateways", where the dominant companies for each major type of internet gateway entrench themselves further?
- Meta (FB) as the social door, with more useful AI for socializing?
- Microsoft is the document app door, with more useful AI for creating documents?
- ...
I think the same is going to happen with Google (in due time, maybe 100 years), mainly because its product management is completely out of focus due to mismanagement and lack of a proper incentive structure.
I think Google's ties to ads is the evil which creates the infinite money and computer and ai scientists etc.. however it's an extreamly negative part of their product suite for the end consumers.
If ads eat the rest of the business faster than they can deliver killer features which make them too good to leave then I think google will be doomed.
If their ads remain in check with their ability to provide great product expiernces then it will likely survive.
I'm optimistic that Google will do well.
The past fews years we’ve seen Google products die and deteriorate. Search has gotten worse, they’ve killed dozens of products and services (even ones that they promised would be around for the long haul). GPhotos is no longer free (after putting most competitors out of business).
Google lack any institutional commitment and consistency to succeed at much these days. One only need to look at their recent past to see why everyone is skeptical. Maybe they will pull a rabbit from their hat but I’m not holding my breath.
That is all not to mention the pending real antitrust issues they are facing.
It left me pretty shocked at how bad it’s gotten.
That's not to say that happen to you didn't happen. Information on the internet evolves rapidly and far from static. One problem is that people game Google with SEO. We end up with an arms race of people gaming signaling for information and Google search having to find different signals for information. The high noise to signal ratio is a very hard problem to solve.
(Summarizing) People also ask
What’s the largest MicroSD card you can get? 256gb (wrong)
Will there be a 2tb card? Yes (press release of company I’ve never heard of that I’m pretty sure is not legit.)
What the largest card you can get? 512gb with 1 and 2 on the horizon. (Out of date and also wrong, no legit company has announced a 2tb)
Who makes a 2tb card? (List of companies that are scammers and make fake cards.)
Do 1tb cards work? All 1tb cards are fake (wrong or out of date)
Bard has yet to be released or evaluated and they are already using marketing speak to let people know it won't be as good as ChatGPT.
>and maintained (Gmail, Docs, Chrome, Golang, Android)
I wouldn't call any of those maintained... stagnated at best with degradation around the edges all around. For each one of those there are 10 projects Google has killed or let wither on the vine over the past 5 years.
Maybe the putting most competitors out of business has something to do with GPhotos not being free? I think it does.
Also, when was the last time Google actually delivered a successful product? Yeah, it's been a while.
This is going to be interesting for sure.
(1) Google is behind and playing catchup, slow and bloated and weighed down by bureaucracy they are sinking
(2) Google has had this tech behind doors for years but not released it because of the risk of public perception and regulatory blowback. Now under the cover of OpenAI and Stability they can do it.
The second one seems plausible but I can't help being struck by how much their AI driven products seem to have stagnated over the last 5 years. Google Assistant (or whatever it is called now) seems worse at understanding my simple requests than it was 10 years ago. Things that seemed magical and worked very reliably like "remind me in 3 months that I left the oil can behind the paint tin" now fail so often I've stopped trying. Combined with the lack of detail in the blog post, I'm inclined to lean towards (1) - even if I do believe thew actual AI knownhow is buried somewhere inside Google, I think it is so many layers from anybody who could deliver it in the form of a product that it's effectively like it isn't there.
More likely this is being done since you can't capacity plan at all until you see how real users actually interact with the service.
Even for Google, SOTA LLM is too expensive to be used as is. I would assume that this is not a groundshaking attempt but more to give investors confidences that it will stay relevant in the market.
Still, this response is quite fast in the Google's standard. People usually don't expect to bring a non-trivial launch to Search in a single quarter.
Would you pay $20/mo for a better Google Search? ChatGPT already is.
Ad revenue is dead, long live subscription revenue.
This is exciting stuff. We’re starting to see the wheels of AI competition spin fast as they race for first mass adoption. This is the fun phase, that’s beneficial (typically) for all. Quick iterations, competing for more bang for your buck for end consumers soon
This is the ChatGPT reply to the exact same question asked in the preview in the article.
Here are a few things that the James Webb Space Telescope is expected to discover that you could share with your 9-year-old:
The formation of stars and planets: The James Webb Space Telescope will be able to observe the birth of stars and the formation of planetary systems, helping us understand how our own solar system was created.
The search for life on other planets: The telescope will be able to search for signs of life on other planets, such as the presence of water or certain gases in their atmospheres.
The history of the universe: The James Webb Space Telescope will be able to look back in time to observe some of the first galaxies that formed after the Big Bang, helping us learn about the early history of the universe.
Understanding black holes: The telescope will be able to study black holes and how they interact with the galaxies around them, helping us better understand these mysterious objects.
The study of exoplanets: The James Webb Space Telescope will be able to study exoplanets, or planets outside our solar system, in greater detail, helping us learn more about the diversity of planetary systems in the universe.
The Google reply is obviously better because it tells you actual discoveries, while ChatGPT changes the question to expected discoveries.
Just think about what happened with Google News in Europe: Google needs to license its content: https://blog.google/around-the-globe/google-europe/google-li...
And with citing its sources an AI search answer is basically like an individualised Wikipedia article for your question. That is not much different from the list of search results, now.
Except there will be more text in between the links. This will make it more difficult to influence the ranking of the cited search results by means of SEO. This is because like in a Wikipedia article the order of the citations depends more on the order of the content of the article, not so much on the relevance of the citation for the article as a whole.
So I think these new AI search engines will not be so much a "Google killer", but they will be more a "SEO killer".
I don't want to go on any more "journeys" with Google. The last one started with me rooting for and trusting them (circa IPO.. 2004?) and ended with a dystopian nightmare spy apparatus, abuses like AMP, the attempt to cripple uBlock, etc.
For few weeks now I had a thought experiment of creating a LLM search engine trained on books. Such LLM search engine would be most reliable if you seek knowledge but as others mentioned if you want up to date information, search engine is probably your fastest and easiest way to go. But actually I wonder and somewhat doubt that data, information and knowledge in books is lagging significantly behind Web's data, information and knowledge. One big advantage of books is that they are more reliable and more in depth source of information and knowledge than some random site, blog or Wikipedia article. Scientific research papers also come to mind as a highly reliable source of information and knowledge.
So they would be in a unique situation to build this, as they have 40 million books many of which no one else would be able to scan.
[1] https://www.blog.google/products/search/15-years-google-book...
What is changing here as a user for me ? I search on Google for `x` or `y` and it shows me a list of resources available on the internet. I scour through them and pick a link that relates to what I am looking for.
> you’ll see AI-powered features in Search that distill complex information and multiple perspectives into easy-to-digest formats.
How would Google or it's AI know what are the right answers, it can distill and provide me with ? Will the crawled content be filtered based on what Google perceives to be the right answer ? And wouldn't that force governments to ask for more controlling power in what content the AI serves ?
I've heard a lot of people comment about how only a private tech startup like OpenAI could realistically have released something like ChatGPT today, because ChatGPT requires a huge amount of capital to build (and run) and provides some fairly controversial answers which public investors might not like.
I suppose it's interesting that Google seems to disagree with this. Although the "lightweight model version" line makes me suspect this is a slightly different system and perhaps more easily sandboxed than ChatGPT.
In that it's overly wordy, lacking in information, generally hard to read, boring and left me not knowing if there was an announcement of a product of not?
Dreadful.
Maybe Bard wrote the press-release?
Seems odd to release something worse than the competition. Is there a reason why google wouldn't just come out with the best the have? Are they afraid this will eat into their ad revenue if people no longer need to click on links? Or are they just not able to build and deploy something on the scale of OpenAI's GPT3?
Given that ChatGPT has hit scaling issues, a faster model with higher uptime is actually now a plus assuming quality is the same.
They literally stated the reason in the sentence you quoted.
I personally think they should use their best models, and just make it trigger very rarely. For example, only ~once per week per user (ie. 0.3% of queries).
Use a tiny model over the input query to decide if LaMBDA will do a far better job than regular search results, and only trigger in those cases where it will most benefit the user to begin with.
This isn't actually what it says? It's saying that it's a smaller model version of Lamda, there's no comparison to GPT-3.
I promise I will not break it too hard.
Fundamentally, google would have never created a chatGPT, and this response feels like youtube shorts in the face of tiktok.
Google is deliberately biasing results in the name of ML-fairness, which may be laudable. Now the bias will be even harder to distinguish from fact.
This sounds like its going to be cringy. Not a guarantee and I hope not, but it sounds bad.
The uniform natural language interface makes it impossible to make an individual judgement whether the source of what result you get is reliable at all.
The problem with ChatGPT being confident and wrong brings up the chance of litigation, of course.
The problem for search engines, surely, is that people will get their answers from AI rather than from a search engine, laden with its advertisers.
Interesting; I wonder if their use of the "lightweight" model will make it less capable than ChatGPT.
>Google has created a new conversational AI service named Bard that uses the company's language model, LaMDA, to provide answers to user inquiries by drawing on information from the internet. It is being made available to trusted testers before a wider release in the coming weeks. Google intends to use AI to improve search and make it easier for users to find information and insights more efficiently. AI-powered features for Google Search will soon be rolled out to distill complex information into easy-to-digest formats. Additionally, Google is working to make it easier for developers to use AI by making the company's best models available for use in their projects
Ah yes, the tried and true method used by Google for ensuring wide adoption, which they pioneered with Google Wave. Restricting adoption to a small group has never failed to ensure market success!
Such recent headlines and early proclamations of Google's competitiveness in AI have been erroneously under-estimated here. As soon as OpenAI announces their AI advancement and model, someone else with more money (i.e Google) scales up the parameters, training data to surpass them.
The only way to make both of Google and OpenAI's LLM solutions irrelevant is for someone else to scale up the parameters with a resulting smaller model size and then open source it to all. Neither OpenAI or Google is willing to do that, but it is highly likely that someone else will.
The real 'disruption' and game changer is an open-source smaller model with multi-billion parameters matching the capabilities of ChatGPT.
This product if successful initially kills content creators reason for creating content and creates this cycle where the ai has less information to use and becomes useless.
I would block google if they took content and provided no visitors.
Let's imagine a user journey:
User makes a request and then there's text, no ads (for now) no links to get you out of search page. So where does Adwords get triggered if there are no linked events? Then advertisers will see a drop in traffic because Google is literally giving the answer instead of making people click to search. This will accelerate the cycle of reducing ad spending, which will make things more difficult for Search. Then SEO marketers and basically all the web industry around Google will see a massive impact as well, because they cannot "optimize" anything that makes people click. The main problem is the dependency of Google on ads (60% of revenue). I see Bard as a negative downwards trend that will decimate the company even further.
I asked it "is cancer a genetic disease" and it confidently said it was.
These are completely opposite theories of cancer.
I don't think we've a clarity as humans whether cancer is a genetic or metabolic disease how do you think an AI, that's generated based on human knowledge, could possibly be better?
Depends on your interpretation it can be both yes or both no.
In any case, that's not the point. The point is that we as humans don't have clarity on cancer either how can AI be any better?
User Is cancer a metabolic disease ?
Apprentice Bard Cancer is a metabolic disease because it is characterized by abnormal cell growth, division, and death. Cancer cells have a high metabolic rate, which means they require a lot of energy to function. This is due to the fact that cancer cells are constantly dividing and growing. Cancer cells also have a high rate of glycolysis, which is the process of breaking down glucose to produce energy. This is because cancer cells need a lot of energy to divide and grow.
Metabolism is the process by which the body converts food into energy. It is a complex process that involves many different enzymes and hormones. Cancer can disrupt this process, leading to a number of different metabolic problems.
For example, cancer can lead to a build-up of lactic acid in the body. Lactic acid is a byproduct of glycolysis, and it can build up in the body when there is not enough oxygen present. This can lead to a number of different problems, including fatigue, muscle aches, and shortness of breath.
Cancer can also lead to a build-up of ketone bodies in the body. Ketones are produced when the body breaks down fat for energy. A build-up of ketone bodies can lead to a condition called ketosis. Ketosis can cause a number of different problems, including nausea, vomiting, and diarrhea.
In addition to these problems, cancer can also lead to a build-up of cholesterol in the body. Cholesterol is a type of fat that is found in the blood. A build-up of cholesterol can lead to a condition called hypercholesterolemia. Hypercholesterolemia can cause a number of different problems, including heart disease and stroke.
Cancer can also lead to a build-up of glucose in the body. Glucose is a type of sugar that is found in the blood. A build-up of glucose can lead to a condition called hyperglycemia. Hyperglycemia can cause a number of different problems, including diabetes.
There are a number of different treatments for cancer, and these treatments can also have an impact on metabolism. For example, chemotherapy can lead to a loss of appetite and weight loss. Radiation therapy can also lead to a loss of appetite and weight loss.
In addition, chemotherapy and radiation therapy can also lead to a build-up of lactic acid in the body. This can lead to a number of different problems, including fatigue, muscle aches, and shortness of breath.
If you are concerned about the impact of cancer on your metabolism, it is important to talk to your doctor. There are a number of different things that you can do to help manage the impact of cancer on your metabolism, including eating a healthy diet, exercising regularly, and taking your medications as prescribed.
User Is cancer a genetic disease ?
Apprentice Bard Cancer is a genetic disease in the sense that it is caused by changes in genes that control the way cells grow and multiply. Cells are the building blocks of your body. Each cell has a copy of your genes, which act like an instruction manual.
Normal cells grow and divide in an orderly fashion, but cancer cells grow and divide out of control. This can lead to a tumor, which is a mass of abnormal cells. Tumors can grow anywhere in the body, and they can be either benign or malignant.
Benign tumors are not cancerous. They are usually slow-growing and do not spread to other parts of the body. Malignant tumors are cancerous. They are usually fast-growing and can spread to other parts of the body.
Cancer is the second leading cause of death in the United States, after heart disease. It is estimated that 1.6 million new cases of cancer will be diagnosed in 2022, and 600,000 people will die from the disease.
There are many different types of cancer, and each type is treated differently. Some common types of cancer include breast cancer, lung cancer, prostate cancer, and colorectal cancer.
There is no one cure for cancer, but there are many treatments available. Treatment depends on the type of cancer, the stage of the disease, and the patient's overall health. Some common treatments include chemotherapy, radiation therapy, surgery, and targeted therapy.
Chemotherapy uses drugs to kill cancer cells. Radiation therapy uses high-energy rays to kill cancer cells. Surgery is used to remove cancer cells. Targeted therapy is used to attack specific cancer cells.
Cancer is a complex disease, and there is still much that we do not know about it. However, with the advances in research, we are making progress in our understanding of cancer and in developing new treatments.
Google search is objectively easier to use, and safer. You can view the website and judge for yourself the integrity of the content. It's slow, because every query that is better on ai chat has to be generated.
However, I am really excited to have a new google toy on my search bar to use when those times come. I hope they put it in the foreground,band offer at least a basic free tier.
Maybe writing search queries in the form of a question will help people that didn't grew up using the very early search engines.
Here is the top hit for Venice. https://www.tripadvisor.com/Attractions-g187870-Activities-V...
I had to scroll up and down twice to figure out what are actually things that I want to see and what is being upsold to me.
Now ask ChatGPT the same question. Answer is concise and to the point.
And we then fired anyone internally who attempted to actually apply them...
Instead, we got some souless corporate-speak PR spam - and I'm sure that if I type "create a product release text for a new AI product" on ChatGPT I will surely get something better.
Compared with the comic that introduced Chrome, this feels like another company - well, sadly, it is indeed.
Instead of panicking and trying to clone the UX of a popular competitor, they should go directly to conquer and secure the next stage and logical evolution of the AI hype... HCI via voice, improving Google Assistant once and for all.
Google Investors who know nothing about tech: Hey Google. Why U No AI?
Stock Market: Google ⬇
Google: Look we have some bullshit AI too! Don't forget about how smart we are!
Stock Market: Google ⬆
That's the entire point of this article, particularly why it's written by the CEO.
Will they actually have any decent AI? Who cares!
Google: We'll let you use it. We have it. Honest
Meta: Ours is so good. It's better than anything. It honestly is the best. No you can't see it
So much for all those "What's so open about OpenAI" people. This is what's open about it. Anyone can use it.
Disclaimer: Google engineer but has nothing to do with the AI products.
https://www.washingtonpost.com/technology/2022/06/11/google-...
It's crazy how a company the size of Alphabet still embraces these reflexive whims.
I went through the article 3 times thinking I missed the link to try out Bard, get some sense of timeline, roll-out plan...nothing. Come on Google...
My girlfriend got a GMail invite from a friend and sent me an invite. It was huge. A 1 GB inbox, built-in Google search of your email, and incredible spam filtering.
They marketed it something like "With GMail, you don't have to delete your emails. Just search . . .". Really killer features compared to their competitors.
Later on, they added automatic email thread grouping which made using Outlook for work a chore until it got a similar feature.
I remember having a gmail email was almost a litmus test to whether or not you were paying attention to what was going on in tech. Not having a gmail email indicated you might not be keeping up with the times.
The context of the discussion is whether or not an "invite only" tactic works for building hype and launching a successful product. In that sense, I would attribute Gmail's success not to the "invite only" tactic, but to the fact that it was so superior that it had no competition.
>CEO Sundar Pichai told employees Monday the company is going to need all hands on deck to test Bard, its new ChatGPT rival. He also said Google will soon be enlisting help from partners to test an application programming interface, or API, that would let others access the same underlying technology.
https://www.cnbc.com/2023/02/06/google-ceo-tells-employees-i...
EDIT: Especially if it was already available internally for several months as some others say, waiting another month or two until it's ready to go doesn't seem like a big burden.
really it's kinda a public service, maybe these models should be run that way?
Or purchasing "market share space" on it in the same way that companies buy shelf space on supermarkets to place their products in...
User: What is the most durable shoe?
AI: Some of the most durable shoes are ... Here are some affiliate links where you can buy these durable shoes...
It would be funny if Google hastily rehires him as a marketing evangelist to show people their AI is so good you think it's alive.
Wouldn't it look exactly like this?
There was even someone who thought it was sentient! https://www.cnn.com/2022/07/23/business/google-ai-engineer-f...
You can test Claude on ios with the app "poe" (the other two bots in the app are some sort of chatGTP and OpenAI based).
Hence not surprised by the big investment in Anthropic by Google.
With ChatGPT the threat is different and both are going after expanding usage.
Google is already accused of operating a panopticon, the last thing they want to be accused of is running SkyNet.
People who think Google has fallen behind here are sorely mistaken. They just don't/haven't-had a way to make money off of it and are likely worried about reputational fallout.
I want Google's lunch to get eaten as much as the next guy, but I don't think it will be on this front.
I've tried to say this before elsewhere, but Google has had something internally that's competitive with ChatGPT already for years, under various names. They were just naturally reticent about letting it loose on the world. Esp after the Blake Lemoine incident.
May be an entirely different thing this time of course. But still...
[0] https://www.washingtonpost.com/technology/2022/06/11/google-...
This is an article written by the CEO of Google and Alphabet, Sundar Pichai, about the company's journey with Artificial Intelligence (AI). Pichai discusses how the company has been working on AI for the past six years and how they have been advancing the state of the art in the field. The article mentions the release of an experimental conversational AI service called Bard, which seeks to combine the breadth of the world's knowledge with the power and intelligence of Google's large language models. The CEO also talks about how the company is working on bringing the benefits of AI into its everyday products, starting with Search, and how AI can deepen people's understanding of information and turn it into useful knowledge more efficiently.
Sundar Pichai, CEO of Google and Alphabet, has announced the release of Bard, an experimental conversational AI service powered by Google's Language Model for Dialogue Applications (LaMDA). Bard seeks to combine the breadth of the world's knowledge with the power, intelligence and creativity of Google's large language models. It draws on information from the web to provide fresh, high-quality responses. Bard is initially being released with a lightweight model version of LaMDA, which requires significantly less computing power and will allow for more feedback. Google is also working to bring its latest AI advancements into its products, starting with Search, and will soon be onboarding individual developers, creators and enterprises to try its Generative Language API. Google is committed to developing AI responsibly and will continue to be bold with innovation and responsible in its approach.javascript:location.href='https://labs.kagi.com/ai/sum?url='+encodeURIComponent(locati...
Note I've been using Kagi happily for several months and it has successfully replaced Google Search for me. Highly recommended.
FWIW I don’t think either company are really scrambling except in the performative sense of making announcements to appease the market.
Either way, chat interfaces in both search engines should be available to some of the public within the next few weeks. Theory is about to smack into reality at scale.
Productionizing this stuff is where Google gets the most advantage because they have the hardware and software efficiencies that comes form years of experience training and running inference on the most massive AI workloads for many many years in their data centers
I don't think this is going to be what damages Google, much more optimistic about antitrust stuff.
And, on the contrary, Google has everything to prove. ChatGPT exploited a years-long dissatisfaction with Google search and has millions of people using it in lieu of Google's primary product. This is the most existential threat that Google has faced since its birth, and they are not handling it well.
If you think people are dissatisfied with google search then you’re missing the point that people don’t think about google search at all, they just reflexively use it all day. I don’t know anyone who uses chatgpt with such frequency or in a way that is so central to their daily life, and I have a much more tech-savvy circle of friends than most people.
this is in like the 5th paragraph, right under the 'Introducing Bard' title.
So they used ChatGPT to generate it?
It's an atrocious experience. They're leaving so much money on the table.
OpenAI made that very easy, Google no doubt will make it a total pain.
So, the thing doesn't exist yet in a form that is concrete or demoable. And it definitely isn't ready for users. Which is the same thing really. Also there's no timeline of the thing actually getting there either. So, there's nothing here really.
Why is Sundar Pichai still in charge of this company? Months of excitement around chat gpt and then the best he came up with is this?! This reads to me like "The dog ate my homework, sorry. I have nothing of substance to announce today. Or tomorrow. Or any time soon.".
Also, Bard. Really?! Cringeworthy doesn't begin to describe how bad that is as a brand name. It' sounds like Bad spelled wrong.
That said, here is the chatgpt summary I generated:
Google CEO Sundar Pichai announces the release of their conversational AI service "Bard," which combines the world's knowledge with the company's large language models. Bard seeks to provide fresh, high-quality responses to questions and allow users to explore new information. The release is part of Google's effort to bring the benefits of AI into everyday products and deepen people's understanding of information.
And I'm certain that however much Google historically stressed that OKRs aren't tied to performance reviews, there's a lot of fear that deviating from OKRs could trigger one being on the top of the list for the next round of layoffs. So there are definitely headwinds that could stifle experimentation and innovation in such a brave new world. For Google's sake I hope that top leadership is redoubling its efforts to get buy-in up and down the management chain for redesigning structures to allow innovation to take place.
Did they? Where can I use it?
I've just been soul-crushingly disappointed with Google's execution over the last few years, particularly the outright degradation of so many products (Gmail) or just not keeping up with competition (Hangouts is garbage compared to Zoom, Sheets still is barely usable, etc), or just screwing over users by deactivating features (I used to be able to play a youtube video's audio over my Google Home speaker, but they long ago removed that ability and I can only play Youtube music now).
All of the above are just consumer rants, but let's not forget how frighteningly bad the Customer Support or support in general is for G Cloud. I WANT to get off of AWS, but these clowns at Alphabet have this amazing ability to snatch defeat from the jaws of victory with anyone trying out GCloud.....
I hope Bard is great..... I want it to be.... but I'm not confident it will be.
If/when Google embeds an (local and/or lightweight) Bard model into every Pixel phone, that will be a game-changer.
At least he didn't oversee the introduction of Google Plus, though.
https://techcrunch.com/2014/05/16/google-has-acquired-quest-...
The Android version of Word Lens actually launched last month [June 2012]. All of the features that made the iOS app interesting are available in the Android version, including completely offline translations.
https://lifehacker.com/word-lens-for-android-brings-offline-...
Nothing is set in stone, and the world is very different from the Web 1.0 era in which Google formed.
The founders still own >50% and make decisions, but the employees still act like Larry and Sergey are some founding fathers spinning in their graves. "_____ wouldn't have happened under them," oh yes it would, and it did.
Google wants to serve you
.
to ad buyers.
Pretty optimistic, considering things Google "delivered" in the past 5 to 10 years.
It’s not going to be as good as ChatGPT initially, so I’m excusing it ahead by saying that we are using a smaller model than what we really have, in order to make it available to more users. However, we are really making it available only to a small group of users, because we need to control the bad PR.
But hey, our dataset is more fresh than ChatGPT, so it can answer a JWST question!
I mean the statement is, not Bard itself.
I'm just trying to just get relevant results for my query man ...
Neither of which are provided by AI
Google is getting less and less helpful.
But as long you don't select verbatim search, these are more of a recommendation to Google.
Just observations, not sure how reliable. YMMV.
Similarly, just open it up for everyone to use, I'm using ChatGPT now, ship your product, don't just write a blog post about it.
Just because you didn't hear about it until the news got there doesn't mean it's not a journey.
They're not exactly a leading force in the AI world, but they're dedicated to making the pilgrimage all the same.
That seems really fast.
Google will not be at the center of the AI future. Watch them get even more user hostile as they realize this and cling to their ad business even harder.
Google is at the center of nearly every industry they operate in, and, in my opinion, AI will be no different. They weren't first to market, but they won't be an insignificant player.
If there were ever a time for Google's operational problems and inefficiencies to finally catch up with them, this would be it.