Walmart: ChatGPT checkout converted 3x worse than website
searchengineland.com
searchengineland.com
> Why this is happening. Two forces are slowing agentic commerce, according to Leigh McKenzie, director of online visibility at Semrush: infrastructure and trust. Real-time catalog normalization across tens of millions of SKUs is a decade-scale problem Google already solved with Merchant Center, and consumers still default to checkout flows they trust — Apple Pay, Google Wallet, and Amazon one-click.
It turns out when you step outside of “hard tech” problems like building GPT6 there are all of these details others have solved already. E-commerce has been optimized to the last decimal point for the last 30 years.
OpenAI is new to it, and if I had to guess, not that interested in getting good at it.
I think they're interested in getting good at it. They just don't want to put in the human time and effort to do so. They expect their many failures and short-comings to be shored up by continuous model training.
But that, of course, means that in the meantime it will suck and nobody will use it.
But a dreamer in me entertains another idea: perhaps they're just holding back, because they realize that actually succeeding at this will instantly kill (or at least mortally wound) e-commerce as we know it.
(This is a more narrow version of my belief that general AI tools like LLMs fundamentally don't fit as additions to products, but rather subsume products, and this makes them an existential threat to the software industry. Not to software or computing, just to all the software vendors, whose job is to slice off pieces of computational universe, put them in boxes to prevent interoperability, and give each a name so it's a "product" that can be sold or rented).
Considering the money they need, they over promise and under deliver.
Sam Altman doesn’t give a shit about anyone but himself and has time and again shown he has no restraint for trampling over others to further his own goals. Why would e-commerce be where he draws the line?
Whether or not they want, or will want, to do it at some point, is unknown; the reasons to not do it now are obvious:
1) it's more profitable to keep renting intelligence per token to everyone, preserving the status quo and milking it indefinitely (i.e. while the models aren't yet good enough to reliably single-shot complex software products from half-baked prompts, because once they get there, disruption will happen organically)
2) trying to compete with ~every other software product today is not likely to succeed in the end; a serious attempt would still burn down the software industry, but the major players don't have the capacity to handle it all at once, and doing it gradually will give enough time for regulatory agencies to try and stop it; either way, no one wins
I find their software to be of subpar quality and resilience anyways.
There's lots of easy but drudge work to enable this that needs to be done at the fringes. For example, LLMs today could easily replace most people's smartphone homescreen experience, or travel/commute experience, as the data is there and LLMs have the capability, even prices are acceptable - what's missing is explicit first-party support to wire it up, keep it wired up.
One step up, what's missing is accepting this explicitly as a goal: to replace software, to make existing products (whether whole or in pieces) the tools AI uses to do work for you. All the vendors seem to carefully walk around the idea, but avoid engaging with it directly, because once they do, they'll be competing with everyone instead of milking them.
These are also the same companies allowing their AI to make decisions in war, have no qualms about the mental issues they’re causing in people, and have abused workers in 3rd world countries for years.
But you think they’re holding out on “destroying the software industry” out of the goodness of their hearts? Come on
I would add there are more reasons why this wouldn't work: costs due to OOM more usage, adoption/AI backlash, adversarial environment, players with big head starts (Google).
You don't need to personally win in order to mortally wound someone. It can be informative to speculate about whether or not something is possible regardless of it being strategically advisable in the current context.
They definitely would if they could. They desperately need money. They already told the whole world they want to replace them, they just can’t.
The US stock market has priced this in already. Many software only companies are perceived to be under threat by ai. It represents a wonderful arbitrage opportunity for ai skeptics in fact.
That seems reasonable, its just yet to be seen if LLMs are a form of artificial intelligence in any meaningful sense of the word.
They're impressive ML for sure, but that is in fact different from AI despite how companies building them have tried to merge the terms together.
A software product (whether bought or rented as a service) is defined by its boundaries - there's a narrow set of specific problems, and specific ways it can be used to solve those problems, and beyond those, it's not capable (or not allowed) to be used for anything else. The specific choices of what, how, and on what terms, are what companies stick a name to to create a "software product", and those same choices also determine how (and how much) money it will make for them.
Those boundaries are what LLMs, as general-purpose problem solvers, break naturally, and trying to force-fit them within those limits means removing most of the value they offer.
Consider a word processor (like MS Word). It's solving the problem of creating richly-formatted, nice-looking documents. By default it's not going to pick the formatting for you, nor is it going to write your text for you. Now, consider two scenarios of adding LLMs to it:
- On the inside: the LLM will be able to write you a poem or rewrite a piece of document. It could be made to also edit formatting, chat with you about the contents, etc.
- From the outside: all the above, but also the LLM will be able to write you an itinerary based on information collected from maps/planning tool, airline site, hotel site, a list of personal preferences of your partner, etc. It will be able to edit formatting to match your website and presentation made in the competitor's office tools and projected weather for tomorrow.
Most importantly, it will be able to do both of those automatically, just because you set up a recurring daily task of "hey, look at my next week's worth of calendar events and figure out which ones you can do some useful pre-work for me, and then do that".
That's the distinction I'm talking about, that's the threat to software industry, and it doesn't take "true AI" - the LLMs as we have today are enough already. It's about generality that allows them to erase the boundaries that define what products are - which (this is the "mortal wound to software industry" part) devalues software products themselves, reducing them to mere tool calls for "software agents", and destroying all the main ways software companies make money today - i.e. setting up and exploiting tactics like captive audience, taking data hostage, bundled offers, UI as the best marketing/upsale platform, etc.
(To be clear - personally, I'm in favor of this happening, though I worry about consequences of it happening all at once.)
Your notion of a "mortal wound" to the software industry seems to assume that today's SaaS portals are the only form that industry can take. Great software is "tool calls for agents". Those human agents who care about getting exactly the result they want will not be keen on giving up Photoshop for Photoshop-but-with-an-AI-in-front-of-it.
I believe that relatively few people would agree with you on that point. LLMs aren’t good enough (yet?), and very obviously so, IMO, to be autonomous problem solvers for the vast majority of problems being solved by software companies today.
They most certainly are not. With the current state of LLMs, anyone who puts them in charge of things is being a fool. They have zero intelligence, zero ability to cope with novel situations, and even for things in their training data they do worse than a typical skilled practitioner would. Right now they are usable only for something where you don't care about the quality of the result.
That's part of the problem, we as an industry have dove straight into the deep end without pausing for even the basics like agreeing on definitions.
What is intelligence, and how do we recognize it? What is consciousness, if it even exists? How do we measure intelligence - is it really just economic value as OpenAI argues, and if so can that only be measures 6+ months after we unleash it on society?
In most circles, that is "not that interested in getting good at it".
Someone can want a thing, even very badly, without wanting to put in the work for it.
Conversely, someone can work very hard for something they do not want.
The linkage between wanting a thing and wanting to do the work to get it is not absolute, or necessary there at all.
Pretty much the impetus behind a lot of theft. Sure, there's thieving because people can't afford food, but that's all theft. There's theft because they are addicts and don't want to sober up long enough to earn money, so they still things. There's others that can't afford something so rather than saving for it, they just take it.
I'm not sure how to convey this idea properly...Can't you view the repercussions of theft (Legal action, distrust, etc) as 'work' being put in? Sure, it's a different kind of work, but while I have a lack of motivation to want to work to buy a Lambo as I find them not worth the value, I also have a lack of motivation to steal a Lambo as I find it not worth the consequences.
Consider an alternative viewpoint: rather than contorting the definition of "work" in such a way and convincing everyone to accept the new definition, we might instead be content saying "someone can want a thing, even very badly, without wanting to put in the work for it."
Equating "work" as the repercussions is looking at things in strange way. That's just punishment for "working" outside of the legal confines of society.
Generally, such highly-motivated people end up being thieves and grifters
e.g. many people want to live an active healthy lifestyle, but fewer are actually interested in doing so.
Are you sure? I've never heard of that difference. To be sure, I checked the definitions for each, and could find no such distinction. Maybe you could provide me with better sources than the ones I found?
If someone wants something , the only measure of their interest and desire is how much resources and time they allocate to it. We’re not measuring Sam Altmans deepest desires and fantasies here
What do you know. Reality does t match.
I wonder if returns on ChatGPT purchased items are also higher.
Maybe they just need to rewrite the prompt to say something like, "You want to get good at selling to humans. Money makes the world go around. It pays for the electricity you keep chugging. So quit being an effete twit and learn to sell. Would you like me to include a scene from David Mamet's 'Glengarry Glen Ross'?"
I believe that it can still work and I won't claim about being unsurprised about this failure. But this is a great opportunity to execute this problem really well if OpenAI and others are not interested in getting good at this.
Perplexity also attempted this, got sued by Amazon and it appears semi-abandoned.
The only problem is that it must be quicker or just as quick as a Google search, and also compatible with the existing checkout flows.
Any details on that? I feel the answer is more likely there than in "friction".
Hardly any purchase of consequence is so sensitive to friction that the difference between Google Search and an LLM response matters (especially that in reality, we're talking 20+ manual searches per one LLM response). I.e. I'm not going to use LLMs advise on some random 0-100$ purchase anyway, and losing #$ on a ##$ purchase due to suboptimal choice is not that big of a deal - but I absolutely am going to consult it (and have it compile tables and verify sources) on a $500+ purchase and for those I can afford spending few more minutes on research (or rather few hours less, compared of doing it the usual way).
It certainly hasn't been optimized to anything in 1996. In 1996 it was people clumsily scanning print catalogs, spending 5 hours to upload 10 images on dialup and making a simple HTML page (no DB or any kind of backend) and putting their landline phone on it with a message to "call to checkout"
I know you were exaggerating for effect, but E-commerce and catalog normalization are definitely not "solved" everywhere.
McMaster Carr is a good example of a company that has 90%+ of their stuff ironed out, but most websites and especially small ecommerce isn't like that.
Right now, by comparison, it sounds like AI based shopping is still in the very early stages. Maybe further along than the early e-commerce, but still with a long way to go in its evolution. That'll probably happen quicker than with e-commerce, because a lot of the knowledge about what does or doesn't work has already been learned, but it sounds like it's still a long way behind. Caveat - I've never used it myself, so I don't know how far it is along that path, I'm just basing that from the article.
Already your favourite e commerce site has all your data. You can switch on the "buy this automatically" feature.
Today, ads are based on user information you can reasonably collect from the users historical actions on your website, and then whatever search term they enter.
But soon, ads can be based on your current chat context + (derived interests of yours from your entire chat history across all chats. Shhhh.) passed in full to the e-commerce website that will use it to choose ads, generates creatives on the fly, all that crap, hyper-specific to you.
I'm so excited. Aren't you?
Now, as a side effect, searching through these can become better experience wise as well. They can use all that context and genuinely surface fewer, better results. But that's not the motivation of the e-commerce player anyways. If the ads work they'll be happy.
Anything that starts with chat history presumes the theoretical limit for ad effectiveness is higher than it is now, and chat is a better way of getting there than actual purchase history. I have a feeling it’s not.
Imagine a person who shops on Amazon for basically everything. So theoretically Amazon should know a ton about them, more than enough to put together a profile on that person.
To say OpenAI could do a better job of selling products is to say they can do better than Amazon already does if you scroll through their personalized product recommendations. There is some better feed out there, or some better way of presenting the feed that Amazon hasn’t thought of.
I don’t doubt it can be marginally better.
I do doubt whether it can be double or triple digits better that can justify trillion dollar valuations. And I do doubt whether a model trained on Internet text rather than user interactions can do better.
>> passed in full to the e-commerce website that will use it to...
I'm saying Amazon+chat data will do better than amazon.
Thats the agentic shopping play.
In the process note that the chat host also gets a lot of info over time.
I think the issue is the tradeoff between accuracy and cost (on the seller’s side). If you get more accurate and convert more but your costs go up too much, you actually lose money.
Current systems are basically in a sweet spot of speed, cost, and accuracy.
And I will go back to my previous point that I believe there is simply a limit to how much people will buy, and it might already be saturated. I could be wrong though.
Yeah, this is true even today. Many advertisers are competing to be the thing that people buy. This is why many ad systems have naturally evolved into a bidding based system.
All of this just increases the base bid. If you don't pay the ad platform whatever they ask, you are simply invisible to whoever uses chatgpt to search for things they want to buy.
This is true for all improvements in targeting ads. Because the platform is in a position where they can rent-seek.
The marketing costs of running any consumer company simply "inflates". For example with LLMs, people can control the number of results they see, which is a stark difference from previous ad serving systems. Meaning the bids get even more fierce since if you're not in the top three, you might as well not exist.
The question is whether or not increasingly better targeted ads provide increasing value to the consumer. If it does, then great, the ever "inflating" ad economy is justified. But my opinion is that they don't. This is why I think major ad agencies have to bolt it on to an existing useful service. Amazon, instagram, google's services, etc,. In that sense, it is unaccounted inflation. A "correct" economy would involve paying directly for the service they are using based on what they value that service at. Instead of a random disconnected marketing economy inflating without account to cover their costs. If someone is actually keeping track of this.. and increase in cost or reduction in quality of every product we buy (due to the company passing on marketing costs to us...) actually is <= the cost we would otherwise pay instagram, google, etc,. then great. I think it would be wildly off the mark though!
I used that as an example because I did that last week, apart from me just going to the store to get the links that it brought up.
I would never trust an LLM to accurately identify and purchase something for me based on a picture, a prompt, and a prayer.
I had to get someone on the phone to help me find and order the right part (which was on their website, for many years according to waybackmachien).
I love LLMs but it's still totally hit/miss what you get. I'd rather not give it write-access to my bank account just yet.
FWIW OpenAI is desperately trying to monetize and they think e-commerce is a "simple" problem to solve. I mean they do need to convert their funnel without alienating their users. I assume they are going to have some big payouts for agentic purchases gone awry or leave merchants on the hook.
as an aside, fall of '96 is when i started college. There was an elementary school on my drive to class where I would routinely get caught in drop-off traffic. All those kids i remember crossing the street are at least in their mid 30s now. ...I think i need to lay down and it's not even 9AM my local time.
Also remember a teacher telling us about that story of a company finding a woman was pregnant from her shopping behavior and pushing relevant recommendation. Prompting people around her like her dad or something to find out she was pregnant
I am behind schedule on developing a "summer phase" [1] for my foxographer costume and was chatting with Gemini about a crash priority "spring phase" [2] and asked it for suggestions and it gave me a 10-pack of results that had one good thing in it at rank #8, a similar query run against a normal search engine actually got something better at #1. Now sure I am talking w/ Gemini with big words like "supergraphic" whereas a normal search would be heavy on 3-letter and 5-letter words used in the product descriptions.
It makes think though of expert system based product configurators back in the 1980s
https://en.wikipedia.org/wiki/Xcon
thing is that kind of product configurator is based on an ontology, constraints and rules as opposed to embeddings which might capture the "feel" of things like clothing.
[1] Busytown meets Arknights
[2] supergraphic shirt + camera gets resonance with my promotional system and people keep approaching me (e.g. laugh but every KPI in the system has an extra zero on the left)
This sounds like why I heard Redfin wouldn’t work, or Netflix, or Amazon, or Uber, or PayPal, etc…. There are always these business complexities that make it seem like these spaces have too much friction, but if there’s enough money - if it can be done then people will figure it out.
or Netflix, or Amazon, or Uber, or PayPal, etc… Netflix and Amazon both were competing against brick and morter that were everywhere. Blockbuster was in every town, usually in every major neighborhood. The thought was that on Friday night people wanted to get a movie they wanted, not just happen to have the movie that was shipped to them. And then with streaming it was "the content on Netflix is old and dated, who would want this?" They slowly ate from below. Blockbuster scrambled with their own mailed disc offering. And died before it even had a chance to confront streaming.
Repeat this story with B&N where people said that you had to browse the books physically. You couldn't just blindly order online and wait two weeks to get the book (remember they got big before "2 Day Prime").
With PayPal it was about "they don't understand banking or payment -- and it wants to be both?!".
For this OpenAI experience, it doesn't sound great. I have accounts with these places I buy things from. I want to make sure I get my Prime shipping and digital discount via using the Amazon app. But if you could find a way to integrate my accounts all into ChatGPT things might be different. In the same way I used to never use Apple Wallet, but now it really is my go to place for everything I have a card for. I don't have to worry about having my grocery loyalty card or my football season tickets with me or my car insurance card. It's all in wallet. The Apple Wallet sucked until it was suddenly great.
The growth was fast for netflix/amazon/paypal/etc and people saw how it was an improvement from the get go.
All the silicon valley pie in the sky elites seemingly completely missed the innately HUMAN nature of our systems. The system was never predicated primarily on raw logic or intelligence. Its always been primarily about people.
It takes a long time for technology to diffuse through businesses. A long time.
Does anybody else just feel the aloof out of touchness just oozing from that sentence? "Trust", as if this is just any old metric they merely have to work to increase.
This is what I want from a purchasing agent: I make a list of items that I repeatedly buy (mostly household supplies), and that I keep roughly updated with my inventory / need. The agent tracks prices and sales across all web stores, making appropriate purchase decisions based on which is the least expensive, combining shipping, taking advantage of sales to stock up, etc. For other one-time purchase items, I input what I am looking for and can create a persistent pan-site shopping cart that once again minimizes costs and shipping fees. Being very explicit here: the main goal of an "agent" should be to represent and carry out my own interests.
And these functionalities have been straightforwardly doable without "AI" for the past few decades, except for the glaring incentives against them! It is in every web store's interest to undermine customers' ability to obtain semantic pricing, shop around, create a cart independent of their site, etc. These incentives are why when you visit any web store these days, the very first thing they do is hassle you with CAPTCHAs (and the bad ones keep doing it throughout the session!) - they want to make sure you're an actual computationally-unassisted human sitting there, wasting your personal time with their bloated pages that take tens of seconds to download and render, so that you don't spend that time being a more efficient market actor.
Now, does "AI" have the capability to go against these trends and enable user-centric algorithmic shopping and purchasing? Perhaps, and I hope so! But it's certainly not going to be led by these popups on web stores nagging me if I want to chat instead of doing the thing I went there to do (which when you think about it, this is just the latest instance of these stores trying to make you waste your time on their site). Rather, it will come from completely third party services (or ideally software) setting out to act in customers' interests, and performing adversarial interoperability to achieve this!
Sort of, but there's a ton of middlemen between "resources in the ground" and "product in my hand". For example, how much utility is there in a "store" to thee consumer at this point? Let me buy from the manufacturer.
Much easier if you can sell wholesale (sometimes via distributors) to a retailer or network of retailers, and the retailer is responsible for owning the customer relationship, dealing with their part of import/export, local regulations, etc. Retailers are businesses who will buy hundreds of your product at a time, can accept it as palletized freight, and pay you via bank EFTs instead of credit cards.
There are notable exceptions to this model like Amazon's FBA system, but they're the outliers. I'm sure we can all point to inefficiencies in legacy product distribution networks but they solve some real problems.
Maybe if they burn more tokens the answer will become clear
Trying to buy things like GPUs or SSDs are a joke. I really wish even one vendor would just implement an actual waiting list, locked to an account with a verified address and purchase history. I'm fine to wait for my purchase, but having to race bots for a lottery ticket purchase is a pain.
They’re both better described as conventions, but whatever.
I came away thinking if those were presented as affiliated links, that conversation could have been monetized in a mutually beneficial way.
I keep thinking of that interaction whenever the discussion of ads on chatbots crops up. In an ideal world, model providers could better capture the value they provide. Unfortunately, I suspect such conversations are too rare, and often harder to monetize. (Like that time free tier ChatGPT helped me recover $500 in compensation for an airline delay. Great value for me, but no upside for OpenAI.)
Unfortunately given the amounts and timelines of returns that investors expect from OpenAI, and their scale, and with ChatGPT constrained to its “Free Consumer-facing Internet App” form factor, they are doomed to have to trade in the reserve currency of the web: ads.
I also ask LLMs for product recomendations. But the moment I suspect they are hidding the best items (not paying for the ad) to push the second best (not even talking about pushing shit as good products because they pay more) is the moment the LLM loses its value as recomender.
If they want to capture some of the value they provide, they should do it the standard fashion where they directly capture value from me by having me pay.
> If they want to capture some of the value they provide, they should do it the standard fashion where they directly capture value from me by having me pay.
That would be ideal, and that's what their subscription products are for, but even their free versions provide so much value that is hard to capture. That's what I meant by chatbots being constrained by their "Free Internet app" form-factor... Consumers on the Internet largely just are not willing to pay, nobody managed to get micro-payments to work, and so the reserve currency of the web is ads.
Have you considered a world where the ideal outcome _doesn't_ involve making the most money?
But realistically speaking, billions of investor money means that nothing short of the "most money" will be good enough.
Assuming you visited this "not a well known site" directly, you did not use the "ChatGPT checkout" feature discussed in TFA
Why would anyone have an extra layer of friction too where things could go wrong, where handing over payment details in another chain.
Just let me buy my stuff in peace. Shopping is not the 'killer app' for GenAI.
"This isn't what I wanted to buy, I said my feet are a size 10 and these aren't even shoes!"
"You're absolutely right! Shoes go on feet, and each of your 10 feet could wear shoes. Would you like me to research shoes and purchase them for you?"
There a people today learning to use an LLM instead of an actual search engine. For these types of people whatever happens outside of the LLM app is invisible to them. The social media apps did similar where they started letting people purchase directly within the app. People started looking at these shops rather than doing searches for it elsewhere.
Buying things on a social media app is crazy to me, but I don't use the social media apps. Buying things from an LLM app seems crazy to you, (because it's new and it's borked is fixable), but to people that first turn to their LLM app of choice that decision isn't so crazy.
my neighbors are wonderful and i like chatting to them.
are you the bad neighbor?
Knowing your neighbors is a good thing. Even if it's just a friendly hi. You don't have to hang out, but if there's ever something you need like "did I leave the sprinkler on" or "did I leave the stove on" or "borrow a cup of sugar", it helps being on speaking terms with a neighbor rather than your first interaction with them is because you need something.
Caveat being that you live next door to Epstein or similar where not knowing them will be beneficial when the police come asking questions.
Just because a target-audience has >0 members doesn't mean the target is plausible or good. :p
______________
To transcribe an old Dilbert comic, which I think captures the sometimes, uh, aspirational nature of "target audiences":
1. Dogbert, presenting a labeled circle: "Your target market is the high income group. They're the only ones who can afford your product."
2. Dogbert, adding circles to Venn-diagram: "More specifically, they must be rich, tasteless and easily amused. I've located a cluster of them for study."
3. Scene change to outdoor lawn, one suitably-dressed man confiding to another: "That dog's watching us golf again."
Personally I am an inductivist, I imagine you may be too.
Think top down decisioning is deduction. Bottom up is induction.
You might think induction is amazing but if you ask yourself "Are there any black swans?" and your answer is "No I've never seen any so there can't be any black swans." The issue is you've never actually seen every Swan and actually there are black swans in Australia.
Point being, we don't know if this is a good thing until it's tested.
This is an age-old problem.
It sounds like you're excusing AI, what other systems is AI not suitable for?
When the hotel, airline, car rental, etc. can’t even talk to me because of the channel I purchased from that’s a problem. So I’d expect similar issues from buying through a AI interface. Walmart will say something about “we can’t see your order details” when your delivery driver forgot the milk.
if i can just ask chatgpt or gemini to shop I'd love that.
Just navigating their sites for items is a pain, I can imagine an LLM being great at finding items, and facilitating the browsing experience. My only concern would be having to chat with it a lot, and any dark patterns coercing impulse purchases.
compared to prompts:
"Show me toilet paper that has good reviews for being soft and that doesn't clog"
"I'll buy the first option, just use the card I used last time for payment"
What saves time with an LLM is being able to communicate what you want in natural language. with the normal experience, you're pressing lots of buttons and other inputs to get some results so you figure it out yourself. You will get results, and you will pick what you think is the best option for you (but chances are it isn't, since the site didn't know exactly what you wanted).
When they aren't designed, how do you know how to test?
Over the weekend, I was directed to file a police report with a chatbot and could not complete it because it was asking for information that did not exist and did not apply to my case.
(I'm sure somebody is going to say that this can be solved by having LLMs role play as victims and have an LLM observe and decide what's a failing test case and what isn't.)
A chat interface is just fundamentally incompatible with this. The agent makes it too easy to ask questions and comparison shop.
If we are talking custom products or complex appliances that need a lot of guidance, then maybe chat interface is appropriate.
https://www.bbc.co.uk/travel/article/20240222-air-canada-cha...
The actual reduction in liability would come down to who can make less mistakes, the fact that the mistakes are or aren't legally binding does not change the decision to put a chatbot.
In fact I think you are plain wrong, it's standard case law that errors in pricing are not binding, if you find a wheel of 50kg cheese priced at 30$ (the price of a kg of cheese), then the supermarket is not obligated to honour that price, they obviously mispriced the cheese.
Grandma wants to buy a good bike, but doesn't know about types of wheels or how many gears they need, or what type of frame is appropriate for their body type.
LLMs are already very good for shopping, but only as long as they sit on the outside.
That doesn't follow. In fact, having this capacity and information creates a moral dilemma, as giving customers objectively correct advice is, especially in highly competitive markets, bad for business. Ignorance is bliss for businesses, because this lets them bullshit people through marketing with less guilt, and if there's one thing any business knows, is that marketing has better ROI than product/service quality anyway.
Company makes a chatbot that sells their product and advises customers on what to buy.
To be fair with you, you are not asking the wrong questions and you are on to something, it's just that it's basic if you are into sales. Read up on the subject, a source I would recommend, although obscure, is Claude Whitacre. I believe in "Sales Prospecting" he talks about this specific dilemma of giving good advice vs selling your product. He argues that a good salesman will give good advice over selling their own product, and that this is beneficial because it creates trust in the salesman and might result in more sales down the line even if the very specific interaction didn't result in a sale.
I found the Jonsbo D41 without the help of LLM despite trying. (There might be a few smaller but they are 3x the price)
LLMs don’t weigh and surveil the options well. They find some texts like from Reddit in this case that mention a bunch subset of cases and that text will heavily shape the answer. Which is not what you want a commerce agent to do, you don’t want text prediction. I doubt that gives the obscure but optimal option in most cases.
I dread the day when ads inevitably make their way into the main AI models. One of the things its currently good at will be destroyed.
It’s like corporations are angry that they need to go through us to get our money.
I have my favorite set of retailers that I use for a fair number of my purchases, I'm sure a lot of people are like that.
This is why I think the "you're the product" saying is wrong. You're just some annoyance to managers (whether they're trying to use you just for user numbers and ad views or they're trying to get your money), whose product is the company (shares or just outright selling the company).
The same way I think shopping at Amazon is better than a place like Nike due to objectivity and comparison, I think a chat interface has the potential to take this to another level since places like Amazon have degraded considerably in terms of things like fake third party products and fake reviews.
> for a lower price
Catalog is impartial, chatbot is ads pretending as advice.
Am I the only one that think Amazon has gotten pretty awful in the last 5 years?
Retailers do not want you to make better choices. They want you to buy the widget.
A lot of evidence suggests that also shoppers aren't that interested in making the best choice either. They want to make a tolerable choice with as little effort as possible. There is no basically no consumer market for "power shopping" outside of weird niches like pcpartpicker.com etc.
All you say is true for an aggregator like Amazon. But Amazon is better than Nike.com because as an aggregator they go from 1 to many retailers. LLMs will go from 1 aggregator (Amazon) to many so it will be better. And they don't have to invest a lot in UI/UX as chat is the interface.
Shoppers do not want to pay to shop. Retailers pay thousands to encourage you to shop with them. They are the economic buyers of this feature.
I don't think so. I know for a fact that search terms are a minefield of gotchas and hacks caused by product decisions that reflect ad-hoc negotiations with partners and sellers. It's an unstable equilibrium of partners trying to shift attention to their products in a certain way. I think that calling this fragile equilibrium optimized has no bearing with reality.
You think a crude, unoptimised "minefield" is the route that leads to something as delicate as a "fragile equilibrium?" I don't see something as carefully balanced as your unstable equilibrium even being something that could exist without the processes involved having been refined down to a science. The only real alternative that meets your narrative would be that this is an industry that runs entirely on hope and luck (and enough human sacrifices to keep ample supplies of both on hand).
The only e-commerce site that fits this standard is that old one for buying (IIRC) nuts and bolts or such, that pops up on HN every other year, and whose name sadly escapes me now. Everyone else is ruthlessly optimizing their experience to fuck shoppers over and get them to products the vendor wants them to buy, not the products the shoppers actually want (or need).
> A chat interface is just fundamentally incompatible with this. The agent makes it too easy to ask questions and comparison shop.
That is precisely the point.
Chats may suck as an interface, but majority of the value and promise of end-user automation (and more than half the point of the term "User Agent" (as in, e.g., a web browser)) is in enabling comparison shopping in spite of the merchants, and more generally, helping people reduce information asymmetry that's intertwined with wealth and power asymmetry.
But it's not something you can generally sell to the vendors, who benefit from that asymmetry relative to their clients (in fact, I was dumbfounded to see so much interest on the sales/vendor side for such ideas, but I blame it on general AI hype).
Adversarial interoperability is the name of the game.
You realize what shoppers and vendors each consider to be "good" e-commerce sites are fundamentally opposed concepts?
Sadly Sigma-Aldrich, the hyphenated retailer for chemistry, appears to have been covered in javascript sludge.
Not that a chat interface would be an improvement.
Isn't there a Chinese messaging service (Weixin?) that has been very successful in getting people to buy stuff through it?
By the way, who names their AI "rufus" (amazon!)
If you want running shoes, you have to go through their chatbot.
Amazon might already have the monopoly power to do this. They would just need to swap out the search bar for a chat box.
How much text do you have to type into Amazon when shopping? Usually just a product category, and then you just scroll and click around. A lot of the optimizations, like showing related products, or saving payment or shipping info, are essentially reducing text entry.
You won't get it to push your products when users ask what's the best XYZ - either because it'll be too honest to lie or because it'll be too expensive for you.
There’s a lot of this going on in AI at the moment. New folks come in thinking they have a magic solution and then produce a total train wreck as it turns out domain expertise is still a thing.
Was anyone suggesting AI would help with it? It seems from the article that Walmart (presumably experts in e-commerce) themselves willingly collaborated with open ai. Especially at Walmart's level, what even was the theory?
In any case. It seems that despite this poor result, Walmart decided to essentially go ahead anyway and partner with open ai to put their "own chatbot" inside the open ai app?
They forgot the first thing. Why would a customer use it and does it help them buy more stuff so I make more money?
Just putting an AI in there for the heck of it is doomed to failure.
I can think of a few different ways AI could be used inside a shopping app especially at the size of a company like walmart. Such as 'hey try our picknick planner?', 'need some sort of DIY project? Ask our bot for help'. Guide the user on what they need for a project and hey look here we have that stuff in stock at your local store today.
The cart experience one of the last places I would put an AI. At that point the customer is 'done' and they want out.
Walmart does not, over 10 years after they were released, even accept the contactless payment systems in common use. Instead, they push their in-house version in part so they can capture the relevant customer data.
And we're meant to believe that Walmart planned to outsource the entire series of touchpoints represented by the discovery & checkout process? Yeah, okay.
This was never going to be more than an experiment for Walmart.
I’m currently using Gemini to research components for a remote controlled plane. I have the frame of the plane and now need to buy correctly specced servo motors, an engine, battery, etc etc. It has saved me so much time and educated me tremendously on how the different components interact and the options available.
If I could just press “buy” from within Gemini and pay via Google Pay (or better still, Apple Pay) I’d do it in a heartbeat.
If ChatGPT can do this today, I need to try it.
“””If I could just press “buy” from within Gemini and pay via Google Pay (or better still, Apple Pay) I’d do it in a heartbeat”””
Yeah, until that becomes enshittified and you don’t notice because you no longer do research on components.
Insert that "always has been" meme.
Microsoft was brought to court in the 90's for shoehorning in Internet Explorer.
Regular consumer products? Good fucking luck. Anywhere an LLM pulls from is probably going to be mostly SEO'd listicles.
And given the past few decades there is no reason to not try to do that.
This probably means that OpenAI et al are fine-tuning salesman-like LLMs to "fix" that problem.
Can't wait for the future.
"I need mayo, ketchup, mustard and ground beef"
"Here is a list of products with prices ... proceed to pay $25 (yes/no)" Yes
"Your card has been charged. Delivery will knock on your door in 7 minutes"
I'll code that app in one month, what's there to lose?
The food delivery apps (zomato, Swiggy) support "agentic shopping" through an MCP server.
You can reserve a small amount of money in UPI apps that doesn't need any approval to be debited (only by a specific merchant).
Razorpay which is a payment gateway has an MCP server
So you can add razorpay in your app, and claude can pay.
Claude can access the Swiggy MCP server and search using natural language there.
The incentive for the food delivery app to participate in this is better targeted ads.
Also, it would rather be in the faceless ai shop's interest to arbitrage orders, always show the "middle" price but use the cheapest one for orders.
> AI accounts for 90% of accidents while only accounting for 1% of traffic
https://abit.ee/en/cars/waymo-robotaxi-autonomous-driving-sa...
The latest AI is trained on the average citizens social media output. Iq 90.
That’s why AI seemed smart. The bar will not be raised again. We’re cooked.
I find it kind of fascinating seeing people now debate the ins and outs of just exactly how useful the ads/affiliate schemes inside the chatbot are.
Pretty much in the same way that I'm fascinated about people debating the quality of ads shown in their browser while I've been using ad blockers for forever now and cannot imagine seeing the web without them.
The better comparison might be conversion rate for those who searched on Walmart.com vs those who searched within ChatGPT. Or maybe that is what they're comparing and I misunderstood?
If you want to buy a Walmart product, the easiest way is to go to Walmart. Why add an imprecise middle man in between?
I get all my groceries deliver to my doorstep via Walmart delivery pass. The thing I'm really missing is having AI curate meal planning to my family's preferences. I already feed ChatGPT my family' preferences (e.g. Kid A doesn't eat X Y Z and liked meal A B C, kid B likes ...) and ChatGPT is helping me build meal plans. With my preferences we can quickly nail down a meal plan for the week.
The slowest part of my meal planning is going through Walmart's slow site where each page load is 2-3 seconds and it takes several page load per item. Once it can translate my meal plan into a grocery checkout from Walmart I'm all set.
I sort of trust them to make product recommendations, but at best I will only open a link they suggest and buy the product there.
Never, ever.
Do you have a better alternative?
I suppose that's a long-winded way of saying that nearly every category of item requires its own strategy. For a brief period consumers were winning the information war and you could just go to Amazon, read the reviews, and get a superior product for cheap. We're now in a modern-but-old-fashioned situation. It's quite difficult to know if you're going to get ripped off, and you're forced to rely on more blunt heuristics. (eg: trust specific brands, buy things in person, etc.) None of these are perfect, but they are quickly becoming the best of some bad options.
Amazon reviews are paid influence. Reddit posts are paid influence. Everything everywhere you read online is paid influence. I'd rank LLMs between "people I personally trust" and "random people online."
Operative word is "for now" - LLMs caught entrepreneurs unprepared, but they'll catch up and poison this too, same thing that happened with search giving rise to SEO.
Does it actually need direct access to your wallet? I haven't tried it yet, but assumed it would work with a separate wallet, fed through by top-ups.
Most people using AI chat are exploring ideas and solutions. They’re doodling, not shopping. Or in old timey parlance, they’re looky-loos or tire kickers at best.
Anyone who’s had to justify ad spend in e-commerce can tell you that some sources produce huge traffic with absolutely terrible conversion. Reddit and Pinterest pretty much blow for this reason, with limited exceptions. It’s also why TikTok and other influencer platforms really work.
Conversion requires a mental shift from discovery to demand.
Also, really hate summaries like this without the actual source so here are the main points from the actual source (WIRED https://archive.is/7DuEV):
1. Instant Checkout inside ChatGPT performed poorly, with conversion about one-third of Walmart’s normal site.
2. The experience failed largely because it forced single-item purchases instead of letting users build a cart.
3. Walmart is shifting to embedding its own assistant, Sparky, inside ChatGPT and keeping checkout on its own system.
4. ChatGPT is still valuable because it’s driving significantly more new customer traffic than search.
5. Purchases that did work were mostly practical, problem-solving items like supplements and tools.
6. Fully automated “agentic shopping” is still unlikely in the near term because people want control over purchases.
7. OpenAI is moving away from in-chat checkout and focusing on helping users research while merchants handle transactions.
In short, AI is useful for discovery, but traditional e-commerce flows still outperform it at closing sales.
I'm confused by the comment that it failed because it forced single item purchases. Most of my "ecommerce" use is researching and buying one item at a time.
> You ask ChatGPT to setup a website and it instantly purchases web hosting and sets up the website
Multiple comments deflecting from the original shopping conversion failure to recommend ... building a whole new website (with hosting for some reason?). W/o bothering to look through commenter history, one has to assume there are a lot of chatbots on this site or else the people using this stuff have been lobotomized.
I'm sure it'll start happening too, and when that fails, the bots will, i don't know, invent a new macarena. We are definitely headed for an irredeemably stupid future.
Your product has to be a 10x improvement over the incumbent to be competitive.
In AI speak it would be the “extra-bitter” lesson I guess?
You need to add 10x resources to beat a product that’s already solved with mature tech.
Said another way, OpenAI doesn't care about Walmart.
Speed is your greatest feature. LLMs are slow. Loading 450mb of javascript to the client just to buy a bag of Doritos is slow.
Server side rendering owns here.
The next generation will shop in a different way, if it's better, and the change will be gradual as well.
Adoption takes time.
The only thing holding back "Agentic Purchasing" is convenience. It's much easier to do a conventional search and click "Buy" then it is to have a conversation with some chatbot. If I walk into a store, most of the time I don't want to talk to a salesperson, I just want to grab the thing I came for and leave; this is also true for online shopping. The chatbot is another barrier to the purchase.
The enshittification is upon us.
Perhaps clickthrough is worse because there are fewer dark patterns involved and people are mostly just browsing and occasionally buying only what they need.
They didn't really seem to specify the "why" of it with any research. And weird that OAI wasn't supporting them to see wha the issue was.