What is really going on at Amazon Fresh?
emaggiori.com
emaggiori.com
There are lots of "this works but not instantly and not 100% reliably" answers. "It was a mechanical turk all along" is not the only logical conclusion from the presented evidence.
And "usually takes a couple of hours"...isn't very bad? Is it ideal? No. Is it enough to ask "How Has Amazon Come To This?" Also no.
You can't even tap a MasterCard to pay in any of the top 3 brick and mortar retailers in the US. Nobody is going to walk 2 more blocks past an Amazon Go/Fresh because they want their receipt faster. Or at least, the demographic who would is likely not the demographic Amazon is targeting. These stores are going up in neighborhoods where people are relatively unlikely to look at their receipts anyway.
That's my guess. They wait until the end of the day and see what they have in the store, since that's got to be the easiest of all of this to automate with cameras.
Then they see how far off the AI was for the day and work on error correction and send receipts.
More of a legal/organizational question. But assuming Amazon Fresh is a legally-distinct entity with service agreements with Amazon Web Services, wouldn’t offering those services at below-market-rate be considered anticompetitive (and also call into question the legal separation of the two entities)?
If all the processing were in the cloud and the data were going in the other direction, AWS might charge 5c / GB, so 35 cents / hour. Peanuts.
But AWS owns the infrastructure, and they don’t charge for ingress anyway. A 1Gbps link might cost $2k per month (wild guess, but not far enough off to matter). That covers 64 streams. This cost is minimal.
Of course, AWS could also stick enough of their processing at the edge to essentially eliminate the bandwidth cost entirely.
It's further worth remembering that Amazon happens to, y'know, own AWS. Since this is not a time sensitive result at all, they could be trying to do it in spare capacity by specifically downscaling when their customers tend to use things. It's complicated to pass that efficiency on to customers, but it's probably super easy to take advantage internally.
Hell I cannot remember a time when I was _charged for something I didn’t purchase_.
I have been charged MORE than advertised but paying attention while ringing up the item and escalating to the cashier or manager typically solves the problem. I have also been charged less and notified the cashier of the issue.
Why would you check your receipt _after_ you left the store? Now you have to drive back, go inside, go to customer service and wait.
Seems way easier to pay attention and scan the receipt when it is handed to you.
Just to pick one user story, it's a lot harder to check / challenge an itemized receipt if you receive it hours later, once you're already packed things away or eaten some of them.
It's easier to challenge an Amazon Go receipt than any other store. I just click the link in the email and get my money. At other stores, I have to go inside and wait in line behind other people at the customer service desk. I prefer to get on with my day rather than argue over the price of a cabbage.
amazon seller inventory reports may have 2 or 3 days of delay on a bad day.
I am sure there are manual reviews when it gets flagged, and I am also sure there are mistakes in receipts (I had an error once out of 20 or so purchases).
I have a much simpler explanation of why billing can take anywhere from a few minutes to a few hours: they process the video feed in batches, and maybe even optimizing for cost depending on compute load in AWS. Video processing takes time, and each store probably has hundreds of cameras - if you want to do multi-person multi-object detection at scale on HD feeds from hundreds of cameras at so many stores, the amount of compute needed is probably not trivial. Instead of processing the video everytime after each person leaves the store, it is much more efficient to do it at intervals and process multiple individuals at the same time.
This line surprised me because it isn't even remotely true in my part of the country, except during times when the stores aren't busy. The self-checkout machines almost always have a line and it's often quicker to go to one of the cashiers instead.
Amazon Go doesn't make any money and doesn't scale.
Amazon palm pay at Whole Foods doesn't work. There's no way to easy way to opt-out of it or change cards, and took WF employees 30+ minutes to figure out how to remove it.
My point was that Amazon could support another retail brand beyond WF.
Their target demo is Mercedes or BMW SUV driver (I'm mostly joking): elitist, hyper-self-absorbed, conspicuous consumption Before Amazon acquired them, their prices were relatively absurd and the products were even bougier.
It's also true you can't get some staples at WF because they need to ensure each SKU makes enough profit / shelf area.
The priciest of WFs are the small ones in major metros in mixed use commercial-residential areas that are essentially convenience stores.
WF post-Amazon also discontinues items that seem to me to be desirable, but I guess I'm not an average WF customer.
One positive point of WF is many locations have bulk dry goods that can sometimes be almost sanely priced.
Disclosure: I live ~1 mi from 3 WF and their HQ.
PS: I recall I was walking on the sidewalk crossing a WF parking garage, and out of nowhere, a white woman driving a white Mercedes SUV with the window rolled down tried to physically push me with their vehicle, laid into their car horn, and shouted profanities at me for the crime of not moving fast enough. Makes one consider wearing several GoPros when they venture forth into the world.
Got to the turnstiles, and the devices there demanded a palm print, no apparent scanners to scan a phone anywhere. We walked out and will never walk back in, especially considering the jackass carting out in the parking lot ridiculed us for refusing to surrender biometric data to give Amazon our money.
PS I also have a ton of experience with barcode printing and reading. As a matter of fact, software I have written has printed over 30M unique QR Codes for track and trace purposes.
It’s that little white thing next to the palm reader. It even has an Amazon app icon.
https://www-ocregister-com.cdn.ampproject.org/i/s/www.ocregi...
If I were building it, I would do it the same way: start out with the whole thing being people watching cameras and labeling training data, and slowly apply software/ML engineering to improve the automation rate.
In fact, this approach is the opposite of putting the cart before the horse: you test the market risk before you solve all the technical problems. The right way to find out if customers want this type of experience is to bring it to market and find out.
I only shopped once at an Amazon Fresh store and the total receipt took less than an hour to show up.
Personally I'm not too fussed if it takes a long time, but I'm very interested in how their procedure works if there is an error to my disadvantage.
It seems like a nightmare to me to send people a receipt later because they could always say I ripped them off and charged them too much, maybe they are right or maybe they are just trying to rip me off. If the transaction is complete in a short amount of time then the window for disputes is less.
Regarding second paragraph, note that the store literally has a dozen high res video feeds from all angles that recorded the entire duration of the visit. There is ample of evidence to prove things one way or the other. Customers caught repeatedly filing fake disputes would simply be banned from entering those stores, which could even be enforced using the same hardware (banned person trespassing detected through AI? autocall the cops). I understand this has a slight dystopian/surveillance state note though.
This video from a year ago goes into more engineering depth than promotional videos: https://www.youtube.com/watch?v=S5t6aYhj6pU
the non-obvious challenges of working in this space are that you have to deal with real world constraints that you just don't have to think about in 100% cloud based software. hardware goes down! internet connections go out! electricity goes out! how much processing can you do in store vs out of store?
that's just the tech in the store. what about humans? humans are now in your programming state moving shit around. in-store associates miss-stock items. kids do kid shit. people don't place things back exactly where they found them.
Then you have interesting distributed problems: how do you handle late data? what should be a massively parallel problem is really a graph of interactions that have to be resolved in just the right order so you can generate an accurate receipt.
and you know what's crazy? the vast majority of the time it works! exactly how they say it does: with machine learning models.
bonkers.
As a customer, I get the experience of a cashierless store. I don't really care how that's accomplished.
As an investor, I might care that I'm being sold snake oil, but I doubt that I chose to invest in Amazon solely over the Fresh stores.
As someone working in the industry, it might certainly affect my perception of what's possible or feasible, but I'm already used to the fact that most of what's being sold as "sophisticated AI" is snake oil.
It just feels to me like this article strikes a bizarre tone about it all, not to mention the fact that other comments in this thread have offered up equally plausible explanations.
...and the recently renovated Whole Foods on Riverside in Sherman Oaks, CA.
Used it myself last week. All the bananas come in 6 banana bundles.
[0] https://www.geekwire.com/2023/amazon-closing-eight-amazon-go...
I think it would actually be harder to have a bunch of people watching and recording your items manually, than the actual system they set up with 4 scanners on the sides of your shopping cart reading bar codes.
And for folks who drive to work, have them shipped to their place of work. Places of work make accommodations like have parking for electric cars and cafeterias -- having a place for delivery storage strikes me as an obvious perk that has tremendous benefits for both employers and employees as well as the environment.
Bananas, in particular, I need them at a specific point in their ripeness otherwise they go bad on me way too quickly.
Why is everyone so quick to defer thinking to AI? Do we all want to be dumb as fuck?
How many basic skills have been largely forgotten because people have instead learned to lean on technology services like a crutch?
We're all turning into the fat lazy fucks on WALL-E