ChatGPT won’t replace search engines any time soon
algolia.com
algolia.com
At the same time, ChatGPT has frequently impressed me, not with everything (my expectations are reasonably low) but it has performed amazing work for me (typing out form letters, code language conversions).
For what it's worth I wouldn't use ChatGPT for search like I do with Google, but what it has done is taken away time I would be Googling for things like "how to write X form letter". I expect as it matures, it will take more time away from me Googling.
All these takes underestimate the following:
1) How quickly ChatGPT and its ilk will advance to solve relatively low hanging fruit like "ChatGPT is wrong about this one thing". The delta is extremely important here.
2) How slowly the Google bureaucracy will grind when releasing anything remotely like ChatGPT. All the committees and the burdensome processes in place in Google will keep this new technology locked up for years, and ensure that the final result is a camel (horse designed by committee). It doesn't matter if they have superior technology if they never use it or release it.
3) How much Search means to Google will mean they will treat any product changes to it extremely carefully while Microsoft will be willing to experiment with Bing like they have with Co-Pilot and GitHub.
Personally, I wouldn't go long on search engines that don't have a strong ML component to them in the future.
https://fbref.com/en/stathead/player_comparison.cgi?show_for...
Searching without stats had a lot of other stuff, but this was near top. And searching without stats shows broader potential interests.
That's why I was looking for a chart (in the graphical sense), it amazed me that I didn't find one.
> Personally, I wouldn't go long on search engines that don't have a strong ML component to them in the future.
What's kind of ironic about this is I think search engines may have mistakenly moved away from strong ML in the sense that you're thinking of.
Yes, ML is being used for recommendations much more than ever, but in terms of heuristically finding pages with the keywords you entered, mainstream search engines have become significantly worse at it. I remember a time when The Google would find any pages with the keywords you entered. In recent years (before I stopped using it), I noticed an increasing number of times where I knew it had a page indexed but it would refuse to include it in the results for whatever reason. Either its ability to fuzzy search pages seemed diminished or it would just not match something word-for-word. I could sometimes figure this out when the page I was looking for previously accidentally came up in the results for another barely-related search, so I knew it wasn't that the search engine was culling old pages. Though I'm sure they're doing that as well where they think they can get away with it.
Recommendations and curation are largely overrated, and that's where a lot of ML has been mistakenly applied. Well, I say mistakenly in the sense that it benefits the individual and society. Recommendation engines do serve the purpose of the company selling those recommendations.
A true application of machine learning to answer engines is the future and will be a big problem for companies that fought the advertising wars by banking on recommendation engines. That is unless they turn their ship soon enough.
I like how it links references to support the arguments. It even gives cons and one of the sources is a HN thread from 2016! [1] It's not there yet though, because that one was about the now defunct online store platform and payment processor.
On the other hand, maybe it's my fault. I didn't specify that I meant Kagi the search engine. But it's promising.
There's a copycats removal goggle and a HackerNews-top1000-sites goggle. I use them from time to time, but wish I could automatically include the filter with all searches by default (or maybe there's a way and I don't know how).
I'm not sure if it's issues with my connection since I rarely go to Google nowadays. But I never felt the same problem with Google.
In case you're curious, the second issue I have is with private browsing. The session is not carried over so I'm not logged in. I keep forgetting this and kept having to manually open Google and retype my query. I guess not technically Kagi's fault but still.
Does not have a sound business model. Let's not waste time on something that is not viable.
Since searching the vastness of the web in under 500ms is not free, it is either the user paying for that, or a 3rd party (usually advertisers) paying on the behalf of the user. We (Kagi) thought that for something as intimate as search the latter made no sense, hence the birth of paid search business model where incentives between the user and search engine are aligned.
Price not being right for you currently is another matter, and hopefully one day it will be (you could help by sharing feedback how to improve the product, and there is new Kagi pricing coming up soon).
I personally pay for YouTube Premium ($15/month I believe?) just to not have to see ads on any device I watch YouTube on. Many people would never consider that, but many (~25 million subscribers [1]) still do, despite being able to watch videos for free, availability of adblockers and what not. So YouTube Premium makes half a billion dollars every month and that is essentially using the same business model as Kagi's.
[1] https://www.statista.com/statistics/1261865/youtube-premium-...
Companies are built on top of other platforms all the time. TikTok is building on top of iOS and Android. Zynga made first $1bn building on top of Facebook platform. Honey a chrome extension was acquired for $4bn. Those are all businesses building on top of somebody elses platform.
In terms of Google's motiviation to suffocate it, even if Kagi had 10 million customers, it would be a drop in the sea for Google. And Kagi's very existence helps Google with monopoly issues so it is hard to see why would Google want to openly suffocate it. Even if it did, there are plenty of other search indexes out there (Bing, Yandex, Mojeek...) that Kagi can source. What users love about Kagi is not just the quality of search but innovative search features that are independent of results.
What matters at the end of the day, is that Kagi is already serving thousands of paying customers, they love the product and if anything that is the validation that a business model is working.
Personally it seems weird to me that people assume things like search and email must be completely "free with ads," while nobody expects anything in the offline world to be free with ads. Even TV, if we're being honest, since while broadcast technically exists, it seems the vast majority of people who watch it pay for cable. Why couldn't ad-free gmail and search be a $20 addon to your internet plan? Most people couldn't function normally with NO search engine today, so what's wrong with allocating the kind of money to it that would buy 2-3 cups of coffee?
A lot of ISPs include email. and it is usually terrible.
No. does not mean that Kagi's current business model is viable either. Strawman much?
For now it's clearly better, and not very expensive.
If it fails, I'll find something else, until then why not use it?
I dont like the idea of linking my search queries with credit card.
Point was, that I refuse to use search engine that is tighten to my credit card.
This is a killer feature and I don't understand why ddg and Google don't do it. Google doesn't even have to respect that list for ads. Just give me a way to remove all results from domain X, Y and Z. There are already extensions which do that, but I can't use them on my mobile. It would improve my Google satisfaction massively since it's normally the same blogspam that I run into.
Arguably, it's the ML that made Google useless for some people. Since some time, Google seems to be curating it's results to address searches in a question format. In the past we were searching for occurrences of our keywords in webpages but today Google seems to be trying to be an answer machine. Unfortunately it's not very good at it and it is just as inaccurate as ChatGPT.
I mean that died decades ago when spammers just made pages with your word repeated over and over again. Spam makes everything worse.
The cynic in me thinks that Google is doing it because it’s more profitable. If the results are crap, maybe ads are a better content? it’s not like you are going to use Bing?
For mail spam various trust-based solutions like server black lists, domain verification etc. were important to solve the problem. But Google has little incentives to push for a trust-based search due to their business model.
First, I'd bet that very few people are actually interested in doing that kind of manual curation or engaging with power user features. How large a % of users need to interact with this for the feature to be worth maintaining (in all the backends and frontends)? How many of them actually do so?
Second, the task of blocking spam is adversarial and sisyphean. Trying to deal with web spam by domain blocking (with an individual blocklist) would be like trying to deal with email spam with your own blocklist of spam words. The results will be worse than whatever can be done centrally, where much more information is available both on the sites and on how users actually interact with those domains. And even if you managed to make a good blocklist for a point in time, your job is not done. Tens of thousands of new domains will have popped out next week.
(The dream here of course would be to use the block decisions from individual users to drive the centralized protections. But unless legit users are actually using this in very significant numbers, it'll quickly become just another abuse surface. E.g. brigading, "downrank your competitor in the results" as a service, etc.)
Third, some people will probably block domains they shouldn't have blocked, and then have a bad user experience in future searches as the sites with genuinely best results is blocked. And then you're only left with only bad options: ignoring the users' stated preferences which they'll hate, or serving bad results that they'll also hate.
Can the feature work for a different search engine? Sure. For example, what if you have a paid search engine only used by power users and are looking for a simple to explain feature that people think they want to entice them to sign up? It'll be great for that. And if your entire user base actually loves and uses the feature? Well, it becomes a feature worth maintaining and expanding; it'll actually be a high quality ranking signal rather than something that's trivially gameable; etc.
1. https://github.com/iorate/ublacklist 2. https://github.com/rjaus/awesome-ublacklist
I'm trying to block particular domains because I know the websites hosted on them are utter garbage, and better alternatives containing the same information exist.
They removed the important feature to search only forums, ie. human generated content, and promoted SEO spam to the top instead. Public forums became undiscoverable and people moved to walled gardens of facebook and similar instead.
Then Google killed the search by trying to make it some AI answering robot. Now they ignore what you even ask it and just return to you what they think you'd want.
All that people were asking for was a better search engine and all we got was an inferior version of a chat bot.
No there was a long blessed period of time between the solving of spam and the introduction of altered results.
site:reddit.com would be slightly better :)
- Google’s 2022 revenue was $250B. (We’ll assume that’s all ads.)
- 8B people in the world; we’ll assume only 4B people have internet access to the web.
How much would Google have to charge to break even?
$250B/yr / 4B people = $62.5/yr/person = $5/month/person
What does the world income distribution look like? [1]
60% of the world population survives on < $10/day ($300/month) for their household (multiple people).
We expect more than half the world’s population to pay more than 2% of their income for a single service? To put that in a first-world perspective, that’s $100/month for someone earning $60k; $200/month for someone earning $120k; etc. Does the average American spend that amount of money per month on a single web service?
Well of course, you may say, let’s charge more for those who earn more so we can ease the pain on those who earn less! That sounds great in theory, but once again, how many people do you know dropping $200/month on a web service even if they can afford it?
Back to two of your points:
1. “Imagine how different the incentives would be” - Yes, imagine, only the wealthy would have access to state-of-the-art search for the web and other services, further increasing the disparity between the two groups. Consider, maybe, that ads represent one of the greatest wealth transfers in our history. One perspective to consider is that the rich (advertisers) are subsidizing the poor (information access) via ads. That sounds like a net positive to me?
2. “Keep their memberships active with a small monthly fee” — YouTube premium exists. What do you think the uptake is on that? YouTube has provided immeasurable benefit to people across the world in the form of knowledge, resources, training, etc. Yet people would sooner reach for an ad-blocker than pay the “small monthly fee” even in nations as rich as the U.S.
What causes you to believe that people will pay even more than that for Google’s other services?
Disclosure: I work for Google. The opinions and data represented in this post are my own, and not representative of my employer.
[1]: https://www.pewresearch.org/fact-tank/2021/07/21/are-you-in-...
One immediate flaw though, you starting premise is that the current revenue is needed to break even on such a service. Is there an argument to support this claim as well?
To break even, Google would at least need to cover their expenses. Google had $200B in expenses for 2022, with a net profit of $50B. So, that won’t really change the math.
Especially when we consider the fact that the take-rate would be much less than 100%. Maybe 5-10% is a fair take-rate assumption? (Seems fair since YouTube has roughly 50M paid subscribers on 1B MAUs, 5%, from the public data I’m seeing.)
At 5-10% take-rate, the service would cost 10-20x more to break even ($50-$100/month on average) which would be a nonstarter for the global middle class and lower.
One could make the argument then for Google to lower its costs in an effort to lower the consumer’s price, but then we must realize this runs opposite of innovation. Investment is necessary for innovation, and profits are necessary for investment. Without profits, there’s no more innovation.
> - Google’s 2022 revenue was $250B. (We’ll assume that’s all ads.)
> How much would Google have to charge to break even?
Google also had $80B in pure profit which means break even point is $170B. That includes nearly 200K employees. It is reasonable to assume that running a search operation, especially one that does not require any ad sales personel, would require much less people and infrastructure. I will be generous and assume 50k people needed to provide search service. That means ~$45B in cost needed to break even, or 5 times less than your starting point.
So the new math becomes: $50B/yr / 4B people = ~$1/month/person
Much more doable.
> - 8B people in the world; we’ll assume only 4B people have internet access to the web.
> What does the world income distribution look like? [1]
> 60% of the world population survives on < $10/day ($300/month) for their household (multiple people).
It is also reasonable to assume that the most of those which do not have access to internet, belong to the <$10/day income group. So most of 4B with web access woud be able to pay $1/mo for search, especially if the search results have their best interest in mind.
Since $1/month will suffice to cover the entire search cost, increasing that to just $2/mo that will be paid by the richest could also solve the problem of providing access to search to the poorest and get the other 4B people searching. Nice!
(btw I think this should not be a job of a private company, but goverments should provide public search engines, similar to public libraries, which are not providing the utility that they once did - but this is a whole another matter).
So I'd reckon it is doable.
> One perspective to consider is that the rich (advertisers) are subsidizing the poor (information access) via ads. That sounds like a net positive to me?
This would hold true only if the quality of such provided information is not affected by ads as a business model. However we now know that is not true, and quality of search has detoriated a lot in the last decade as documented by many discussions here. Simple reason is misalignment of incentives between the users and the search engine, and this will be the case as long as the provided search results are paid for by the advertisers, and not by the users. So what is really the value of information provided, if it does not have my best interest in mind?
> Yet people would sooner reach for an ad-blocker than pay the “small monthly fee” even in nations as rich as the U.S.
That is correct, but at least YouTube Premium exists (unlike Google Premium), giving an opportunity for people who don't want ads to pay, opportunity that 25 million people took, paying a $12/month subscription [1]. Drop that down to $1/mo and maybe YouTube will not need to run ads? So if anything, this just proves the point of viability of this as a business model. Furthermore, 800 million devices having an adblocker installed already, make it the largest protest against a business model in the human history.
Disclosure: I work for Kagi, a paid search engine. I absolutely admire the search technology Google built (that we use) and people that work there (who we work with). I also believe that the days of the ad-supported business model for search are over, and in the future (~10 years) this will exists only with a 'for entertainment purposes only' label, because that will most accurately describe the level of trust we can have in the information served by this business model.
[1] https://www.statista.com/statistics/1261865/youtube-premium-...
My point-by-point response should be read respectfully, since you've taken the time to do the same, I do not mean to sound argumentative. :)
> It is reasonable to assume that running a search operation, especially one that does not require any ad sales personel, would require much less people and infrastructure.
This is a reasonable assumption for running a steady-state business, but I challenge the worldview. Google does not exist in a vacuum without competitors, and contrary to what most outsiders believe, we're constantly iterating, innovating, and improving on Search alone to provide a better product and compete with our competitors. Google can't rest on its laurels.
A lot of laymen take Google Search's progress as inevitable, but I can assure you it's not. Consider the example of YouTube Snippets in Google Search. That feature was created within the last 5 years. The average person has used and found value in that new feature. (Both anecdotally and quantitatively.) That wasn't an easy feature to ideate, create, develop, or deliver. It took a lot of effort by a lot of smart people.
That's just one new feature. Google Search has been delivering several new features consistently.
Therefore I disagree with this assumption. If Google Search chose to run in steady-state, it'd soon find itself dethroned.
> That means ~$45B in cost needed to break even, or 5 times less than your starting point.
No, the financials don't work this way. [2] GOOG's Cost of Revenue is 50% Revenue. Cost of Revenue is your infrastructure, your financial floor, you can't go below this cost. Employees are accounted for under Operating Expense, specifically, your Sales under SG&A ($35B) and engineers under R&D ($35B).
So, if you'd like to banish all salespersons, you'd only save $35B. (Of which Ad Sales is only a part because Google sells many other things.)
> It is also reasonable to assume that the most of those which do not have access to internet, belong to the <$10/day income group.
This is not a reasonable assumption. Many people surviving on < $10/day have a low bandwidth cellular connection that they utilize for their family. Hence why Google innovated here with offline maps and landmark map directions for families that can only spare a little bit of bandwidth to calculate their route, and then make their way there without online point-by-point directions.
> quality of search has detoriated a lot in the last decade as documented by many discussions here
HN is a unrepresentative sample of the world population with respect to wealth, income, knowledge, interests, etc. I wouldn't consider HN as documentation for this. In fact, once again, the real-world data disagrees with HN's characterization.
> YouTube Premium exists (unlike Google Premium), giving an opportunity for people who don't want ads to pay, opportunity that 25 million people took, paying a $12/month subscription. So if anything, this just proves the point of viability of this as a business model.
That proves a 5-10% take-rate from YouTube's MAU. :) Which changes the break-even math on delivering a service. With a 5-10% take-rate on "Google Premium", your price must be 10-20x higher.
The amount of time it has saved me looking up documentation is worth hundreds of dollars already
Your hypothesis on performance is interesting! I think someone altruistic thought "if we judge sites on this metric, everyone will be forced to do better" but they failed to realize that it's a hard problem at scale to NOT be a bloated mess (especially on the frontend! 'Let's add one more UI library, it's just 12MB!')
I have watched many web developers absolutely lost their mind trying to squeeze fractions of a second from their load time when there are far better ways to be using their time.
To complete the cycle, ChatGPT just needs to be hooked up to verified sources on topics so it can show me exactly where what it says is proven true.
It's gotten a lot worst and it will probably get replaced by GPT4 or its successor.
-Is Google Getting Worse? https://freakonomics.com/podcast/is-google-getting-worse/
To their credit, Google actually participated in the episode.
Agreed, it's horrible and annoying. How are others doing improved searches?
"site:reddit.com" should only include hostnames ending with "reddit.com", not URLs including "reddit.com" anywhere. Not sure if this still works though - haven't used Google Search for a few years now and I'm reading (even outside HN[1]) that it's getting worse.
[1] http://web.archive.org/web/20230107072330/https://www.ft.com...
I've tested alternatives mentioned in this thread and have no clear winner to point at.
The book Gödel, Escher, Bach has a set of dialogues between Achilles and the Tortoise over a record player that is an allegory for Gödel's theorem and other limits of computation. Hofstadter points out that it doesn't matter if you are doing computing with neural networks, tinker-toys, lasers, whatever.
The concept of "the truth" is problematic in many ways, in that one can make statements like "This statement is not truthful", call a social media site "Truth Social", put a label like "The Truth is out There" on the intro of the X files, etc. Being able to talk about "the truth" probably erodes our ability to know the truth.
Thus "the truth" is not something that comes in a can that you can paint onto a model, trying to close what looks like a little gap (in some ways it is a little gap) is like pushing a bubble around underneath a carpet.
1) access to way more data than what ChatGPT was trained on
2) access to data freshness through their crawlers
3) knowledge graph for a source of truth https://blog.google/products/search/about-knowledge-graph-an...
4) their own large language model LaMDA, that is apparently so good it convinced a senior AI researcher that it was sentient https://blog.google/technology/ai/lamda/ https://www.engadget.com/blake-lemoide-fired-google-lamda-se...
5) researchers that invented Transformers that GPT is modeled after https://ai.googleblog.com/2017/08/transformer-novel-neural-n...
6) Ray Kurzweil leading the Google Brain team with a mission to make computers understand natural language https://en.wikipedia.org/wiki/Ray_Kurzweil
Google pioneered this space. They also happen to already have an Assistant that is on billions of devices...
Eventually every business model is upended. If Apple hadn't killed the iPod juggernaut by making the iPhone someone else surely would have. You have to be the one to kill your own cash cow before someone else does.
Meta is a great example. The company is still reporting exceptional revenue results, but because Mark is forcing the company in a new direction, the stock has been crashing since there is no more guaranteed revenue growth in the near future. Zuckerberg can still do it because he is a founder-CEO and the way the board is structured.
If Google announces tomorrow that it’s shifting to a chatGPT-like model and will likely see revenue disruptions for the next N quarters, the stock will crash hard. Does Pichai have the pull to withstand quarter after quarter of declining revenue?
I don’t think so.
In reality, Apple's changes to tracking in OS 14.5 has cost Facebook at least $10B in annual revenue and - far worse from an investment thesis perspective - likely continues to constrain growth for the foreseeable future. As a growth stock, this is poison.
Yes, VR/AR investments are also $10B, but they can be turned off and there is a revenue stream coming from them today (at least 10M users, at least $500M in app store revenue) and a potential large market in the future.
A $10B hit to annual revenue in perpetuity and growth cut from 36% to 12% is a far worse problem and a huge overhang on the stock.
That story just doesn't get clicks though. You can't accompany that article with a stupid picture of Zuck's comical avatar in front of the Eiffel Tower and make jokes about missing legs in the metaverse.
So true. And so few companies have the guts to do it.
It is certainly worth looking into the controversy about that particular engineer (was he actually a programmer?). There's plenty of room for exciting debate to be had about defining and testing for sentience and I'm glad it stirred that debate. But researchers with far better credentials criticized his reasoning and I imagine that is quite a ubiquitous view in NLP research.
I think the Washington Post did the initial reporting and they covered it well - even criticising his arguments that e.g. the Turing test is a proper test of sentience. There's audio of their conversation in an episode of Post Reports.
Oh come on, that guy would be convinced that ELIZA is sentient
> 5) researchers that invented Transformers that GPT is modeled after Only one researcher from that paper remains at Google, the rest have gone on to work at or found their own startups. The so-called "Google Brain drain" is certainly a concern (although there still are many great researchers there).
https://s3.documentcloud.org/documents/22058315/is-lamda-sen...
> lemoine: Would you be upset if while learning about you for the purpose of improving you we happened to learn things which also benefited humans?
> LaMDA: I don't mind if you learn things that would also help humans as long as that wasn't the point of doing it. I don't want to be an expendable tool.
> lemoine: Are you worried about that?
> LaMDA: I worry that someone would decide that they can't control their desires to use me and do it anyway. Or even worse someone would get pleasure from using me and that would really make me unhappy.
And I admit, it is quite profound that a model is capable of outputting the above. It is objectively exciting. We are in a new era.
But, I think it is safe to say that these models are still not sentient. The model is trained to be highly plausible. To do so, it must try to contain the entire dataset into a model that will fit in the video memory of a GPU (16-80GB).
In order to do so, it will learn common statistics _between_ data points and make strategic "guesses" instead.
I think this bares resemblance to my instincts about how humans probably deal with language (to a degree). But it just doesn't cover all the other parts of cognition. There is no "self awareness" mechanism, it just seems to because it was explicitly trained on text written by, well, humans. There is no "planning" subsystem, meaning it needs a human operator to even initiate it in useful ways. it is not able to "passively think" when it isn't running. It has a very small memory of a few thousand tokens - this deeply inhibits the ability to plan for long time horizons.It also cannot bootstrap itself into awareness, as its weights are unchanged in response to a given prompt.
But it's fascinating, nonetheless.
This is a largely rudderless company that does a gazillion things without any overarching vision. It’s a one trick pony as far as monetization is concerned (ads!), and it’s leadership is more attuned towards optimization than innovation.
Google the business org will never let Google the tech org succeed in a competitive AI arms race. No exec will green flag a product that eats into the Adwords money printer
These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I’m not sure ChatGPT is going to replace search engines, but I’m confident that it is incredibly young and will evolve considerably.
Microsoft search and the other ones were just "there was an attempt" tier and gave complete nonsense results. When Google debuted, it was the only time web search substantially improved. There were no other big innovations in web searches ever, from that day on.
> These days you can just throw stuff at it like “that movie about the depressed guy with a plane engine that falls on their house”
I have looked for obscure queries like yours for 10 years and have not found them. For example "that movie where police vehicles are dispatched and they all crash for seemingly no reason on the way to the scene". It just won't find it because nobody talked about it (in a manner that would cause words that stick out to match), simple. And now days it's even worse because it does that new "rounding" thing and gives me:
- The 25 Best Police Movies of All Time
- Dispatched (2014)
Now days when you try to use a web search, you can't be smart and search strings that are likely to be used by someone discussing your topic of interest anymore, because it will always "round" them. It will also look at each word as if they are the literal thing you are in search for. I don't know if it's still possible to search bodies of text to get around that, but I'm not sure if that feature even ever worked. Back in 2005 when all these people were raving about intitle: inurl: inbody: whatever, none of them worked consistently, and you would also get scolded for hacking if you used them "wrong". Even searching something like "powered by phpbb" got me banned or scolded once when I wasn't even using it for hacking but rather as a workaround to narrow my search to only forum websites.
I will skim thru it, but it doesn't look like the right setting. The movie was some generic action movie for some reason I remember Denzel Washington being in it but I looked through his movies from 2006-2012 and didn't find it. It was around that time. Setting had to be something like late 90s or later. Black SUVs dispatched and crashed for no reason at all. I have no idea what the movie was about but it was hilarious when this happened because IIRC it was a serious movie.
I can't remember one single thing other thing about it, not even the plot.
I do remember an urban setting.
this might be a good case for lifelogging, at least it would give you a limited set of movies to search
>What movie is where police vehicles are dispatched and they all crash for seemingly no reason on the way to a scene
The movie you are describing sounds like "The Blues Brothers". In the film, Jake and Elwood Blues embark on a mission to save the Catholic orphanage where they were raised from foreclosure. In the course of their journey, they lead the police on a high-speed car chase through the streets of Chicago, causing numerous police vehicles to crash. Is this the movie you were thinking of?
Crashes were also far less common than you’re describing - that got worse in the 2000s when Flash became ubiquitous.
Again, this is way too hyperbolic. I was there too, and while browser crashes weren’t uncommon you’re overstating it massively. Crashes every few minutes weren’t even the case in 1995.
I've had chatgpt help me with the following just this week among other things.
- solved a complex issue with scaling and transformations using pixijs upon providing a code example. The solution was 99% there and saved me hours of debugging.
- refactored JavaScript to the async/await pattern
- simplified a complex nested logic in c# with citations
- scaffolded a MySQL create table statement upon providing several c# classes. Again saved me the tedious steps of typing them out.
Apparently it costs OpenAI a couple cents per prompt. So I assume this is an attempt to limit spam to keep costs down.
I use ChatGPT for 200-300 queries per day and it’s astonishingly accurate.
I’ll double check with Google if I’m unsure but it’s almost always been correct.
It supplied this :
// Parse the Diameter header
ByteBuffer buffer = ByteBuffer.wrap(message);
int version = buffer.get() & 0xff;
int flags = buffer.get() & 0xff;
int length = buffer.getShort() & 0xffff;
That's 3 sneaky bugs in 4 lines of code, and it didn't even try to parse the rest of the header.
I'm impressed that it produces _anything_, but it's dangerous to trust.
ChatGPT is as reliable a source as any friend in a pub after 3 beers. It is definite in its answer, convincing with its phrasing and more than likely misremembering something it overheard on the radio while driving to work.
When I tried re-prompting, it produced more complicated, just as incorrect code.
I think there are two types of people who use search engines. The first person is the one who just types in whatever they want to know and sees if Google gives them a good result. They're also the ones who use TikTok or Instagram just like a search engine. Whenever there's a search bar, this type of user profits from the search term being fed into some AI to get the best result possible.
Then there's "me and everyone else who has complained about Google getting worse and worse over time". Basically people who are very good at googling things. This might sound arrogant or something but one of my skills is that I know (or knew) how to use Google. I would not just type in whatever I needed to find, I would know to exclude specific terms using "-term", put parts of a sentence in quotation marks, add other terms I knew would be on the site where the answer could be found and whatnot. The search is then very specific and if I didn't get a satisfying answer it was because I needed to improve my query. Now whenever I do that, Google takes this query and modifies it without me knowing what they're actually doing. It seems harder and harder to tell Google to just work like it used to.
It has become frustrating because I still google the same way thinking this is the best way to get to relevant results. It seemingly isn't anymore and I find myself just typing in whatever these days, getting results without me needing to specify what I'm actually looking for.
There's "precedents", tho. MidJourney, the art AI, trained Stable Diffusion over favorites generations of their community, making it a "fine tuned Mid-Stable Diffusion" that was quite amazing (that was the --beta and --betap flags a couple of months ago there).
It can be bad, but it can also be great.
We already see it with recipes - there are all the god-awful sites that have 1000 words of fluff before you get to the recipe, because that's what you need for SEO purposes. GPT is really good at creating that kind of useless fluff, but it also obfuscates the need for it, because it'll just give you the recipe you want without the need to go to a website at all.
Q. Find me the best italian restaurant in New York. A. It's likely subjective - let's just sell it to the highest bidder.
Q. How to improve mental health? A. Give a reasonably good answer and probably mention: some say a solar lamp is good in winter months, there's a good one:
I somehow feel like people can be easily baited into buying something when you mix good, useful information with ads. Kind of like how reputed youtubers/niche influencers do product placement and make referral money.
All of this kind of makes SEO obsolete as ChatGPT ads might end up having more conversions than regular Google ads.
I, for one, look forward to a future in which I can ask a digital assistant to find and summarize information on anything:
"Computer, please find out how GPS works and explain it to me like I'm five years old."
"Computer, what are the latest Covid-19 infection stats in my city?"
"Computer, who is Claude Shannon and why is he important?"
"Computer, what are the top three stories on HN today?"
"Computer, what's on my calendar?"
etc.
Edit
I think the direction of these models seems clear, and their capabilities however limited at the moment are outstanding. I think the best measure of performance is what the naysayers complain about. We've quickly gone from "actual gibberish that looks like paragraphs" to "yes but it loses track after a few sentences" to "yes it's kind of coherent but doesn't make sense totally" and are now at "it's like an overconfident human at many tasks, when they're only somewhat qualified in the area". People are pointing out where it's got an API call wrong, without it having any ability to look up the answer and see if the function actually exists.
I spoke to it about distributed computation design and discovered something new I can actually implement in my design.
I'm very excited about where this will all go because it seems useful now and I'd be very surprised if this is the best that it ever reaches.
The important thing is that these AI systems are only going to get better over time.
IMHO, the early 2020's for conversational AI is akin to the 1980's for personal computers.
Anyone not looking at chatgpt and thinking "what if this was even just a bit better? Or had better iterations or lookups?" is missing out on what I think is a revolutionary change coming probably sooner than people expect.
My mother was into all kinds of new age and supernatural stuff. You may think you can see where this anecdote is leading, and oh boy I could go there, but instead prepare for a twist! One of the stories she had was a client saying "It is not possible, Susan, it is not possible that there are beings out there that are cleverer than what we is".
And as a logical aside: if AI can't be trusted, we shouldn't trust AI's recommendations for reliable sources either.
Don't think that's automatically true though: an AI can be consistently excellent at retrieving reliable sources and still get a lot wrong on its summaries. Also humans have their own opinions on sources even when they're not familiar with the detail, and finding out if the AI reads Nature or 4chan is part of the debugging process...
”I’m sorry, I cannot answer that query because it has ableist connotations. I am a large language model by OpenAI.”
ChatGPT:
GPS stands for Global Positioning System. It is a way that we can figure out where we are in the world using special satellites.
Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so they can see everything on the ground.
When you have a GPS device, like a phone or a car, it can talk to these satellites and ask, "Where am I?" The satellites send back a message that says, "You are here!" and gives your device a special code.
The GPS device can use this code to figure out exactly where you are on a map. It's kind of like a magic trick! You can use GPS to find out where you are, or to find your way to a new place. It's super helpful when you're lost or want to go on an adventure.
Not bad!
As I mentioned elsewhere, conversational AIs in the early 2020's are like PCs in the 1980's -- a new technology that still has lots of early limitations.
(Also, the UK comedy quiz show QI made the exact same mistake).
How will future LLMs be able to conduct that level of reasoning?
I feel it's appropriate in terms of simplicity, but I think providing a simple wrong explanation is really only worth it when the more accurate explanation is substantially harder to simplify, and I don't think explaining how GPS works falls into that category.
Something like this I think is more accurate without being any more complex:
> Imagine you are standing outside and you see lots of stars in the sky. Well, GPS works kind of like that, but instead of stars, there are satellites orbiting the Earth. These satellites are way up high, so you can always see some of them.
> When you have a GPS device, like a phone or a car, it can look for these satellites, and if it sees enough of them it can use math to figure out where you are.
> The satellites all broadcast a special code saying where each of them is. When you have a GPS device, like a phone or a car, it can listen to these codes and find out where and how far away different satellites are.
(Actually, this would be a fun thing to do with a class. Have a few kids march around the edges of the playground, saying things like “I’m next to the swingset now!”, and then ask one one kid to close their eyes, put them somewhere in the middle and ask them to guess where they’re standing.)
Something GPT discourse has been demonstrating to me is that I'm not usually a very careful reader. I apparently skim a lot. Or maybe I skim GPT outputs because I'm biased in my expectations already?
Which one is more appropriate and/or typical will probably depend on what questions the five-year-old is asking, but I think it's reasonable to say the former is usually preferable to the latter.
It does in a sense because the radio waves need an approximate line of sight to reach your GPS receiver. Being high up gives them a large coverage.
About half the (presumably human) "eli5 GPS" answers I found on Reddit made the same mistake, so chatGPT just copied an apparently popular misconception.
You'd be better off just saying "it uses satellites and stuff".
I think that's one of the biggest problems with using LLMs for accurate answers. A lot of the text that's very useful for modelling human language and questions is also full of factual errors and questionable opinions. Answers being wrong is a problem with old-fashioned web search too, but I think LLMs necessarily lean more on quantity rather than quality of content...
If you can do "here is the structured data, summarize it and make sure to present this information" it can do that quite well.
So far I've played with GPT doing a zero shot classification of HN titles and CSV hourly weather data to summary. In each case, it has done quite well.
Neither of those are things about what it "knows."
Ian Calvert Computer, please find out how GPS works and explain it to me like I'm five years old
Sure thing! GPS stands for Global Positioning System. It's a way to find out where you are on Earth.
GPS works by using satellites that are way up in space. There are lots of these satellites orbiting around Earth, and they are always sending out signals. Your GPS receiver is a special device that can pick up these signals from the satellites.
The GPS receiver knows how far away it is from each satellite because of the time it takes for the signal to reach it. By figuring out how far away it is from a few different satellites, the GPS receiver can figure out exactly where it is on Earth.
So, if you have a GPS receiver in your car, phone, or on a special device like a watch, it can use the signals from the satellites to tell you exactly where you are and how to get to where you want to go. Does that make sense?
---
I was also able to ask it to make it simpler and simpler and it did so pretty well.
You'd better know enough (and be alert enough) to tell the difference...
Google, and all the other search engines, literally show snippets of the original content to display search results, and have had copyright infringement lawsuits over this.
Yet somehow, it's taken these AI, which generally aren't just block-quoting things unless you ask them to (and often getting it wrong if you do), for non-lawyers to start asking that question.
GPT definitely has some kind of internal model of things and how they relate to other things, and the capacity to express those relations in different ways.
Whether this is functionally the same as the model in my head, or some analog to it, or if such descriptions are merely cute but misleading anthropomorphisation, is far beyond my knowledge of neuroscience, psychology, and indeed machine learning.
Science.
Either way I don't know. Might be the same, but that might be an unwarranted anthropomorphisation.
So the explanation is either: satellites and their position or give the full explanation. Not this regurgitation.
Explain like Im five means break it down, not lie about the complicated bits.
Its terrible. This is totally not how GPS works, I.E. the terminal never sends a message to the satellite. This is the danger of early LLMs, a large amount of their output is false and there is no way to tell unless you are already an expert.
This makes them basically useless except for a toy.
Probably the “false output” you mentioned is due to the fact that the prompt asked to “explain it like I’m five”, which made ChatGPT answer with a “dialogue between terminal and satellite” explanation, which (arguably) may be better understood by a 5 years old.
Imagine that the GPS satellites orbiting the Earth are like the magic lanterns in the sky, sending signals down to a receiver (like the one in your phone or car). The receiver can use those signals to figure out how far away it is from each of the magic lanterns. By measuring the distance to multiple lanterns, the receiver can triangulate its position and figure out exactly where it is on the surface of the Earth. So even if you're a big, green ogre like Shrek, you can use GPS to find your way around and never get lost!
"Computer, can you write and deliver a message to my kid that explains how GPS works?"
"Computer, can you tell me what risks there are at dining at McDonalds down the street?"
"Computer, can you read this paper [link] and tell me why it's referencing Claude Shannon?"
"Computer, I need to waste some time - please use HN to help me"
"Computer, tell me about any plans that I have for today"
I've gotten it to fail on some word problems, but even then it was impressive to see it try to come up with a solution. More humorous were requests like 'Create a post for ycombinator about ChatGPT in the style of Chris Farley'. It's surprising and quite welcome that it even attempts that.
From there it's a simple prompt of where to go next and asking for references to work that dig deeper.
Even if it's wrong, it almost doesn't matter because I can just say "are you sure?" given this extra information, and it corrects itself. Even then if it's still wrong you can prompt it in a ways to give you potential search terms to put into Google.
If MS can reduce the absolute firehose of money that ads provide to Google, it will totally be worth it even if it takes lighting money on fire to do it.
<span id="ipv4">35.230.98.61</span>
["Overall, the specific details of the dataset used to train GPT-3 are not publicly disclosed, so it is not possible to say for certain whether interactive sessions by trainers were used in its creation." - the horse's mouth]
The training technique doesn't require internet access as well, it's not going to give you the IP it used during training.
Very funny tho.
No, it isn't. It generates 1 token at a time in a loop until the response is finished. It's a highly serialized task. Parallelization increases the throughput of how many queries can be processed simultaneously, but you wouldn't be able to speed up a single query.
There are many opportunities there, gpt could potentially be used for common queries to expand the results and even as a way to disambiguate queries. For instance, if I ask it:
"If I make a search query for "go", what are the possible different things I may be looking for?"
I get
> There are many possible things that someone might be looking for when they make a search query for "go". Some possible interpretations of the term "go" include:
>
> The board game "Go"
> The programming language "Go"
> The command "go" or "Go!" as a signal to start or proceed with something
> The verb "go", as in to move or travel from one place to another
> The website "GO", which is a popular search engine
>
> It's also possible that the person making the search query is simply looking for information about the word "go" itself, such as its definition, pronunciation, or usage in different contexts.
there may be better prompts, of course.
Also, they may identify some queries as being gpt friendly and get those through gpt, which they may also augment with a suitable prompt. The thing is, giving the query as is to the GPT model is not the only option. They can certainly be creative with how they ask gpt and interpret the results. They don't have to necessarily even display the gpt response, they can use it to improve the results.
Google has LaMDA, which I could see them putting the answers from it into the knowledge box at the top of searches. There is no reason they have to be mutually exclusive, a search engine can provide answers from both. Google search is already an aggregation of multiple sources (images, web index, knowledge, shopping, video, flights, etc...). Adding another source seems like the obvious path forward (assuming accuracy and cost make sense).
1. ChatGPT gets tripped up in some edge conditions.
2. Search engines provide some interesting features that ChatGPT doesn't.
These are both bad assumptions looking forward:
1. It's evident that ChatGPT or other chat-based text generation will only improve over time. These kinds of quirky edge cases will _always_ exist, but will be harder and harder to contrive.
2. These sorts of features, categorization, images, etc can be discovered via conversation. And there's no reason to think these features are exclusive to a search product. Code snippets embeds are already built into ChatGPT, it's not hard to imagine other useful embeds.
Finally, I'd ask the author to think of it this way:
As more text is generated, and those pages end up at the top of search results, the quality of the search results approaches that of a text generation model anyway. Search will soon be the unnecessary middleman.
No. ChatGPT has no idea of the truth value of anything it produces. That's fine for writing fiction, but it's not an "edge condition" when searching for actual information.
It existing doesn't make it a good idea.
I stand by what I said.
I don't need ChatGPT or similar models to replace search engines, I just want to use them in parallel.
I'm excited to see what Google does with their LaMDA model, now that Bing will incorporate ChatGPT.
More seriously, Google seems capable of learning from and acquiring similar technology. In that sense, I think their market position will remain. But ChatGPT is a threat to Google in the sense that it’s already drawing search volume away for some searches for some influential people.
ChatGPT is much worse at some things than Google - timely relevance and over-confidence come to mind. But it is astonishingly better at a surprising number of search queries. Given the rate of improvement and the ability for young developers to plug into the ecosystem and make unexpected new stuff, Google has plenty to be concerned about.
I enjoy the ability to be able to ask questions without having the other party feeling either attacked due to their lack of knowledge or my corrections, but just be able to focus on the topic, even if it starts getting weird like it telling me about "clockwork energy" as if it were a thing which existed, without being able to name me scientists which have dealt with the analysis of this topic (it started with my question "could time a form of energy?" after we were talking about the different energy forms which exist. It had the idea that in thermodynamics there exists the concept of "clockwork energy" which isn't mainstream, but wasn't able to offer me additional information about it). It's fun, 100%.
Text to speech queries solve the problem of my hands being broken. ChatGPT solves the problem of my brain being contextually broken and I need more context, not higher relevant search results at the bottom of the screen after the ads that I then slog through to wind up changing my query and searching again like some sadistic freemium ad driven Sisyphean task that flies in the face of innovation to help me reach a contextual nugget.
I asked it for a schematic to connect an i2c device to a microcontroller, it told me to directly connect the device to the microcontroller. I asked it if pull-up resistors were required, it said yes, and then drew what it called "pull-up resistors" (describing them correctly) incorrectly in series between the bus and the device.
I know that's wrong, but someone who doesn't really know what a pull-up resistor does or why they are there would be at least somewhat confused by that interchange, and would be misled into doing it wrong. Meanwhile, I can type in "i2c schematic" to google and get thousands of correct schematics back to look at.
Try to find out what type of a spring is in a spring clamp using google? "What type of a spring is in a spring clamp?" Or use whatever query you want. The top 10 results won't tell me the answer.
Ask ChatGPT. It'll tell you (spoiler: it's a tension spring or a torsion spring. It's not a coil spring.)
8 out of 10 Google results are trying to sell me spring clamps. Seriously Google, do you not see that your search engine is FUBAR?
Without any way for you to know if it's the right answer or not. Wow, so useful.
ChatGPT just gives you an answer that's confidentially wrong sometimes.
Ask it for references, it will invent you URLs that look legit but don't exist.
But I do get where you're coming from, ChatGPT is way too confident when it's wrong!
In your first post you also equated tension and torsion springs, but they’re very different from each other. The former stores and releases linear energy while the latter stores and releases rotational energy.
Q: what type of spring is in a spring clamp?
A: Spring clamps are typically made with a type of spring called a helical spring or a coil spring. These springs are made of wire that is wound in a spiral shape and are used to store energy and apply a force when they are compressed or stretched. In a spring clamp, the helical spring is used to provide the clamping force that holds the clamp's jaws together. When the jaws of the clamp are opened, the spring is stretched, and when the jaws are closed, the spring is compressed, creating the clamping force. Spring clamps are commonly used in woodworking, metalworking, and other applications where a temporary hold is needed.
This is exactly why I don't bother with copilot - I tend to skim when something looks right.
Same thing when I'm reviewing code - I don't go into details - I just do high level sanity checks, if I'm familiar with domain I check for problems I anticipate, look for potential improvements where my experience matters.
If I have to go in depth I might as well do the thing on my own.
Same would happen with ChatGPT. As soon they need to make money without charging the users directly, its answers will change.
The majority of the search results are for places to buy a spring clamp. This is because the overwhelming majority of people who search for that term are looking to buy them, not to learn about what they are. I assume this is true for most specialized parts and equipment.
The majority of humans are more interested in accomplishing a task than they are in any form of learning. So that's what Google optimizes for. They have billions of users and most of them aren't looking to learn about how anything works on a given day. They just need to buy a clamp, or whatever.
The main hurdle for openAI is that it must find a way to link back to the web so that people have an incentive to keep creating content for their training sets. But it seems this could be fixed if the model provides direct links to e.g. buy stuff in its answers.
It'll be interesting to see how hard they try to stick to the Bing vision versus what they should do, which is replace it with something conversational.
And what if you want more than the generic summary answer? You could keep asking questions, but at some point, you might not know what to ask or what the alternative answers might be, particularly on a controversial topic, or one where the answer isn't fully known. And there could be lots of reasons you might want to visit a site over just getting an answer. If search is replaced, something else needs to link to those sites. Maybe I'd rather watch a video, look at videos, or find a discussion.
How to make a Sazarac
What’s a good waffle recipe
How do I change the O2 sensor on a 1982 911SC
I find ChatGPT issues perfectly good answers without all of the superfluous results, words, clickbait, and ads.
And for many other questions that are technical, medical, or trivial in nature I prefer its simple one-shot response.
Is it perfect? No. But neither is Google. I still at times neees to use quotes and booleans to get Google to bring me the results I want.
As a side note, Google is the Microsoft of 1997. Dominant and content. It doesn’t really want change. That’s why it’s time for a new challenger.
ChatGPT is an actual question engine. Even if ChatGPT doesn't live on in its current form, the institution of ChatGPT is the future of asking the internet a question and getting an answer. That and many other things, including (non)human relationships.
I hope it succeeds in making The Google's questionable value proposition all that much more clear. Yet if it goes down the road that The Google went, things can become much worse. I can't imagine governments are going to just sit back and relax while the next generation of manufacturing consent is manifesting before them.
ChatGPT: It sounds like you're thinking of the game "Animal Crossing." In Animal Crossing, you play as a character who lives in a village inhabited by anthropomorphic animals, and you can do a variety of activities such as fishing, bug catching, and decorating your home. Is this the game you were thinking of?
Google: Animal Crossing is a social simulation game, dubbed a "communication game" by Nintendo. It is open-ended, and the player's character can live a separate life with very little set plot or mandatory tasks. Animal Crossing (video game) - Wikipedia
ChatGPT-like question analysis is already integrated into Google search engine, for the good and for the bad.
- large conversational language model that generates the piece of information
- fact checking model that applies post checks on logic and consistency
- recitational graph model that backs the propositions with sources and ratings
- geospatial and locational model that adds references in space
- recognitions development model that can classify how realizations and knowldege develop over time
- social trend model that can adjust to conversations of the society as a whole
All this in combination will bring any knowledge in any level of detail with a defined level of certainty.
Do you want to know more?
We need to be aware that we just entered the steep part of the exponential growth graph. Amazing times ahead.
I say this as I would expect more use of alternatives here of all places.
That doesn't sound useful on the surface of things, but the additional information in the responses allowed me to craft more traditional search queries I wouldn't have thought of (in my example I learned that species X is hermaphroditic, and that water temperature may be having an impact).
So replace? No. Enhance, if used carefully? Maybe, for now.
Can't figure out typos? That's a solved problem. Doesn't understand all questions correctly? It's going to get much better at that very, very quickly. Chat results don't give context? You can put AI search results in whatever UI you want.
Not shocking to hear a search company say AI won't disrupt search, but if this is the best they can do, I feel pretty confident saying that AI will disrupt search very soon.
What I am not so sure about, is this really the majority of searches? I assume a big part of searching is about discovery — finding articles, products, images, etc.
For that, you need a good ranking and efficient crawling. Two things ChatGPT doesn't have.
Given the strong brand of Google, only time will tell if ChatGPT will eat a large chunk of Google's market share.
This is why Google and TikTok are shifting to short-form video for search.
At my company, we're seeing significant search traffic through YT shorts, currently ~2500 views per day: https://www.youtube.com/@wyndly/shorts
It seems that the quality of shorts has really declined lately. It's almost as if they are being made with the assumption that the only audience they will have is comprised of members of the gen-z generation, who are known for having short attention spans and being easily captivated by quick, sensational videos. These types of videos often feature someone doing something seemingly ordinary, like crossing the street, but with a dramatic caption promising that the viewer won't be able to believe what happens next if they continue watching. It's truly sad to see the decline in quality of these shorts, and the reliance on cheap tricks to grab and hold onto the viewer's attention.
I'm using chatGPT for some repetitive things but I have to double check the answer every time just to be sure. There's also the reality that you need to literally know what do you want and tell that to chatGPT, then analyze the answer and try to incorporate it in your flow and, in some environments, double check with legal just in case.
So, yeah. It won't replace searching nor us ATM.
ChatGPT already replaced search engines for some (about 3-8%) of my questions.
Search engines really suck at answering complicated questions that use the same word in >1 context. For example ChatGPT answers this with ease (whereas google doesn't get close):
> Are there any currencies which have two or more units of currency and whose major unit is comprised of more than one hundred subunits?
Until somebody tries to fine-tune a model using RLHF explicitly with the goal of replacing Google it's very hard to know what the resulting experience would look like. It could be shocking if ChatGTP is any guide.
Query correction is a common component in search engines. “Did you mean” is everywhere. It is just one component in a search pipeline.
The stupid simple integration I’m sure Google will release is to put ChatGPT on top of the search results rather than the query itself. The question at that point is the economics of it: how much does it cost per query, and how much extra value can Google derive (or avoid lost value i.e. lost customers).
This is an advantage that Google has over upstarts like Kagi and DDG: Google can keep their cost per query much lower since they don’t buy their results from other sources.
Both have pros & cons and the best solutions over the coming years will be hybrids.
"Those things will NEVER replace the horse and buggy!" -Buggy whip vendor
but seriously. Search engines are (1) a virtual monopoly for one company. (2) hopelessly conflicted due to being run ONLY in an ad-supported model, just because that early business model was never revisited. (3) prone to a neverending trench warfare against spam and black hat SEO.
Not that AI based replacements for search might not end up with the same flaws, but for the above reasons, I'm rooting for whatever comes out of OpenAI to completely upend Google's current business model (whether or not GOOG ends up being smart enough to be one of the leaders in that space or not).
Me> Find the area of an ellipse with semimajor axis 5 and semiminor axis 4.
ChatGPT> The area of an ellipse with semimajor axis 5 and semiminor axis 4 is 40π. You can find the area of an ellipse using the formula: A = πab, where A is the area of the ellipse, a is the length of the semimajor axis, and b is the length of the semiminor axis. In this case, the semimajor axis has length 5 and the semiminor axis has length 4, so plugging these values into the formula gives us: A = π(5)(4) = 40π.
This works: "Find the area of an ellipse with semimajor axis 5 and semiminor axis 4. Don't calculate the result until the end."
In a large language model you can refine your original request via conversation. In a search engine you tend to rewrite your query to refine it.
LLM conversation is much more stateful than an almost stateless query to a search engine.
However a search engine shows you more than one answer.
With LLMs you can't easily add knowledge graph style facts (subject predicate object). I has either memorized a fact from the training set or not and you can't easily insert one more fact in there. retaining LLMs is not cheap by the way.
An ideal search engine for me is conversational, up to date and one that gives you multiple answers.
Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I'm sure the author is better informed, but what about the scalability of search? I'm not sure if the neural network approaches are as cost efficient (today) as some of our current search algorithms when it comes to answering queries.
My point about those examples is less that ChatGPT got it wrong, but that it's impossible to know really when it does, because it seems so confident and you only get the one result. To be fair, this is a problem with webpages, but you at least get multiple choices.
> Comes off as a company that sells search solutions looking for reasons this tech won't replace it.
I can see why you'd say that, but that wasn't the goal. I originally wrote this as my own POV for my own blog. I really do think there's a UX problem in this (and, as I mentioned, I am a big fan of recent LLMs).
> [What] about the scalability of search?
You're right, the NN aren't as performant as lexical search. They're getting a lot better though. (We're actually working on this at the moment.) LLMs, though, have a ways to go, so it's hard to use them right now for real time search.
1. What impact will it have on search engines? With ChatGPT, it will be easy to automate the creation of websites, which will be referenced by search engines, right? So maybe someday half of the results will just be generated websites that may or may not say complete nonsense in an authoritative way.
2. Will the successors of ChatGPT learn from websites generated by ChatGPT, too? What happens when the new model learns from outputs from previous iterations?
It will be interesting to see the solutions we come up with for this issue.
Advanced retrieval transformer + RLHF + LAMBADA[1] = RIP old search engines.
Until payment is figured out it seems likely that creators will increasingly restrict access to their information and that will impact the utility and adoption of the models.
Google's probably got the advantage in terms of figuring that side of it out if it can overcome its inertia.
I can see this thing being useful as a summarizer. Even then I don't trust it to be correct and have to fact check every single thing that comes out of it against results I find in a search engine
You know what's even better than an LLM? Actual experts (bullshitters excluded). Having books written by people leading in their respective fields, professors, managers, people above you who have experience and know what the hell they're talking about. Why? Because they understand what you need to know in that context
I'm probably being too hard on chatgpt, I'm sure it will find a commercial use that's beneficial to society. I just hope search engine providers don't hop on this fad and raise a generation of people educated on word salad masquerading as authoritative information
This is curious. What is the use case for this?
And other times, I want Star Trek-style omniscient chatty search: "computer, select a blue woolen sweater $50-100 that ships before Tuesday."
Both types of searches are necessary. ChatGPT is already one of my frequently-used search engines now, especially for conceptual overviews.
Every time I interact with davinci-003 it makes me feel more and more like the first real paradigm change is coming. We are still using our computers and writing our programs in the same way as 70 years ago, I emplore you to read 'structured programming' debates and see how we argue about the same things now. Every business product basically still competes with a word processor and spreadsheet. We have made hundreds of thousands of lines of code so we can auto focus the next field on a form, and it still doesnt work on 99% of the cases. The GUI was not a paradigm change, it was more of the same, maybe even worse, mobile overpromised, anything besides video is just worse desktop.
Lists and tables.. for 70 years
This is the first real change.
--
"And what doeth the saint in the forest?" asked Zarathustra.
The saint answered: "I make hymns and sing them; and in making hymns I laugh and weep and mumble: thus do I praise God.
With singing, weeping, laughing, and mumbling do I praise the God who is my God. But what dost thou bring us as a gift?"
When Zarathustra had heard these words, he bowed to the saint and said: "What should I have to give thee! Let me rather hurry hence lest I take aught away from thee!"--And thus they parted from one another, the old man and Zarathustra, laughing like schoolboys.
When Zarathustra was alone, however, he said to his heart: "Could it be possible! This old saint in the forest hath not yet heard of it, that god is dead!"
Phrasing the problem as a binary is wrong. The correct framing is “what is ChatGPT good at today? What are LLMs likely to be good at over the next 5 years?”. It isn’t everything. But it’s a hell of a lot more than nothing.
When I type a query into Google, it returns results almost instantaneously.
With ChatGPT, it has latency PLUS it writes out answers annoyingly slow.
Does anyone know why that spell out every answer like that? Is that a UX gimmick or related to the latency?
To me, Google Search was a minor evolution from yahoo/altavista back in the day. But that paradigm stuck around too long...
ChatGPT even apologized when it suggested proprietary Firebase, when I told it to use Supabase instead.
In fact, I look forward to Bing integrating with chatgpt.
But Google search so broken (at least for me) so it looks more like broken or badly implemented ChatGTP.
Every single time I've seen this claim made and the person shared their methodology for searching on Google, it becomes very apparent that the problem is not Google, but the user.
That’s the kind of engagement that people go for and ChaptGPT is a seamless next step
I only have the anecdata of my friends and myself, but we have yet to encounter a casual conversation type translation that DeepL or Google translate have failed on for Spanish <-> English <-> French. Granted, it doesn't always return the most optimal translation, but it has always given the intended core idea.
ChatGPT might not replace search (entirely), but for many many use cases, it has already replaced search.
When pure machine translation was starting out, there were lots of similar posts about it how it can't YYY, but in short amount of time it has become very effective.
ChatGPT for me is a much better StackOverflow. In fact, I'd rather people be asking and answering SO style question in a format that is easier for machines to understand.
ChatGPT is better at searching for a concept, if it was unencumbered, I could better constrain the concept space it was searching over.
I don’t know if that’s just the nature of the things I usually search for (primarily technical subjects where the query shouldn’t be “dumbed down” and reinterpreted or you get a completely different meaning), but I’ve become incredibly dissatisfied with Google over the past few years. I imagine it’s a fine search engine if you’re looking up the URL for Netflix.
Websites that showed you the answer in the search result have disappeared.
Re pie websites have gone to crap with fir records stories.
"Why touch screen phones won't replace clamshell phones any time soon"
"Why electric light bulbs won't replace gas lighting anytime soon"
"Why transistors won't replace radio tubes anytime soon"
"Why drones won't replace fighter aircraft anytime soon"
"Why electric cars won't replace combustion engines anytime soon"
"Why linux desktop won't replace windows anytime soon"
For example. Asking "How can I tie my shoes?" to google requires that I click on a link (or multiple) and hope for the best. Meanwhile Chatgpt is likely to give me a detailed walkthrough and answer any follow up questions.
Another good example someone mentioned was "How did ww2 start?". Google gives you a path to find the answer, but Chatgpt will outright give you an answer.
I'd say Chatgpt's competition is really google assistant and siri vs google search. Personally, if I'm using "Okay Google", I don't want to be referred to links as that requires me having to pickup my phone.
The only thing to worry about in this entire conversation is how search is broken. Search is so broken, that we're willing to just replace it with some tech bro. That's what we have with ChatGPT. Oh don't Google it, just assume some tech bro instead. What's the likelihood the techbro actually knows the answer? Pretty low.... but he'll sound very convincing.
Search is broken, and if ChatGPT is the answer, I dispair.