I really wouldn't trust 'picked up off the shelf' detection, not without the whole thing looking like a giant vending machine. RFID tags on products probably works better.
From my first quick take of the video, the app+turnstyle is used to identify you to the store. The video system then tracks your position as you walk around.
When you walk out, the items are recognized and tallied by a large RFID sweep. Funneling you back out through a turnstyle makes sure the vision system knows it's you. Notice that you don't need to barcode yourself on the way out, and the exit system is phone agnostic (it's not checking for an NFC or Apple Pay tag or anything).
The whole "tracking individual items as they come on and off the shelves" task is a very complex thing. But tracking bodies as they walk around a 1,500 square foot room isn't that hard.
Examples: https://www.youtube.com/watch?v=p1LykdRWTfk https://www.youtube.com/watch?v=-hCA8L7v-R8
What is really needed is a way to talk to multiple RFID tags quickly without any crosstalk. Think of a handbag with a dozen candy bars in it. How do you scan them all quickly without missing any?
The goal has always been to scan something large in one pass, say a shrink-wrapped pallet of items that could number in the hundreds or thousands. Obviously this is technology that Amazon could benefit from as well, I wouldn't be surprised if there's a lot of technology overlap here.
Most RAIN Rfid readers have a read rate of 800+ tags/second. Door portals are a solved solution and now phased arrays are on the market which scan all items in realtime. Take a look at the impinj marketing material for more information. For example the RS2000 chip: http://www.impinj.com/products/reader-chips/indy-rs2000/
But the video clearly shows the items being recognized as someone takes them from a shelf (and puts them back). The items don't need to be recognized at the time you walk out, the store just needs to know that you've walked out.
I think that's just visually giving you an idea of what's happening. Like I said earlier, I'm giving a simple naive presentation of how the system might work. Or, at the least, how I would design it without dealing with finicky shelf sensors. Ask anyone that has ever worked in a hotel with in-room minibars how well those things pan out in real life.
Unless that store is just a mockup and not how it really looks and works, I see nothing on those shelves or in the sky above it that is watching you put that cupcake back on the shelf.
The real way to find out is if someone in the demo video can pull their phone out of their pocket mid-trip and see their current inventory and total price. But I didn't see that in the video.
The girl in the video at 1:26 looks at her list and total, but after she's been through the exit turnstyle.
The video makes it look like it's constantly up to date, but that sounds quite complicated (despite dropping the ML, DL buzz words).
A friend at Amazon said he tried to trick it by taking two items at a time and it got it right every time, so yeah it does appear to track when you take the items.
In essence, RFID tag technology already exists, and there's a reason why it hasn't taken off in grocery stores.
That's already handled in grocery -- people can mess with regular barcodes if they want. it's a known cost of business
BUT what if their goal isn't to make razor thin margins at grocery stores, but to test out their tech on human tracking. You've got to believe that they have a more long term plan than making a better self-checkout.
They can now tie in your Amazon Go app with you as a person. Most likely all the sensors and cameras in the store will be both tracking you via your app with Bluetooth, but also perhaps body heat sensors and the like that track you based on your unique body pattern.
Then they can tie this in with your Amazon Prime account to better improve recommended purchases to you, they can tie this in with your Prime Now account and your home address and more immediate delivery needs. Drones, Kindle, Echo as well.
The goal is deeper analytics and tracking on individual humans. That's how Facebook makes their money, and Amazon can perhaps have deeper data on individuals than any other company tracking browser cookies.
Using these as EAN barcode replacement also has the problem that the system can't tell if you bought the product from Amazon GO or if you just happen to be carrying it with you when you enter the store.
Maybe you could use it to make some guesses about a person. An older guy buying frozen dinners, beer, and not much else is probably single or divorced. A woman buying prenatal vitamins is probably pregnant. Someone buying both Special K and Fruit Loops might have a family. But someone can already make these determinations from using their eyes. RFIDs might make it a bit easier but don't really add any new infosec facet to this grocery store experience.
The problem with electronic surveillance is never the surveillance part. It's the part where we can surveil billions of people at once without the need for billions of spies.
I am sure that amazon/walmart/tesco/whoever will need more than 1000 at a time and so obviously get a larger discount. Probably down to well under a penny or two - starts to make a lot of sense when you factor in the other benefits (lower staff, easier stock-tracking/taking etc)
https://www.alibaba.com/product-detail/China-Professional-ma...
So, it would require a $10 item to break even on a €0.10 tag.
Basically if Savings from reducing sales persons > Cost of New Technology, you'll add the new tech.
I don't think that's actually what Amazon Go is doing though.