McDonald's is ending its drive-thru AI test
restaurantbusinessonline.com
restaurantbusinessonline.com
I learned two things from this project. Our team built a complete English language model for every possible permutation of ordering every menu item. No LLMs, no gpt required.
The hilarious part is that McDonald’s stopped responding to our team and abandoned the project only about a month before we were going to tell them that it was not technically feasible to do it scale. The problem is not bad AI nor a lack of data. The problem is all of the real world interface challenges, keeping speakers and microphones working outdoors across many different climates and weather conditions, and temperatures and humidity is incredibly expensive. And on top of that, the hardware needed per store at the edge and the permanent network infrastructure required to keep everything running makes the whole system substantially more expensive than just having a single human being running drive-through even have much higher wages than minimum wage.
So McDonald’s never got this conclusion from us and instead spent several billion more dollars and another six years of R&D to come to the same conclusion.
My other take away from this project was that I will never give McDonald’s another dollar of my money. Working with over 200 fortune 500 companies in my career McDonald’s is far away the most evil heartless and ruthless company I’ve ever dealt with.
They don’t care about their customers, their users, their franchisees or their employees. The only thing they care about at all is their stock price.
> The problem is all of the real world interface challenges, keeping speakers and microphones working outdoors across many different climates and weather conditions, and temperatures and humidity is incredibly expensive.
I thought this was a solved problem already, by McDonald’s in fact?
Servicing industrial edge compute is insanely expensive for B&M businesses. This is a big part of why so many companies have spent the last decade offloading everything humanly possible to public cloud workloads.
Probably this.
Also what ML model did your company use?
And, when you are traveling, and just taking the highway exit for a quick order, what, now I have to stop, park, download an app then order.
Or do you mean, like mount Ipads in kiosk mode? I think that gets back to the expense and weather.
And by the way, we all wanted employees to have a better pay. It is just pure survival mode now by mcd. Since paying humans is expensive now, one has to find ways to make them expendable even more, so that the company remains profitable for the franchise people and keeping the product price low.
(And after all, was McD not a real estate company?)
Their plan longterm is exactly this - have a robot box with a window that only accepts orders through your phone and removes all humans from the process.
You might be surprised how many people absolutely hate using apps, and do not want to interact with a business through a mobile phone. Older folks especially, but also there is a large portion of the population that only use technology when necessary.
We build a heuristic model from scratch. We used BART for some NLP bits and Azure speech to text for the basic mic -> raw text. My memory is very muddy on this bit as I didn’t work on the algo portion of the project, I was working on the UX workflows and interaction / conversational workflow design and validation.
Fun fact, McDonalds menu is a graph database that tracks every single ingredient as purchasable entity. You can order damn near any combination of elements and they will sell it to you - want 32 pickles and the bottom piece of a bun with a chocolate chip cookie on top? No problem.
And every promotional item and name ever put on sale is persisted in the menu forever.
It is not just older folks. It is also anyone mildly or more security/privacy aware. App's offer far more opportunities for spying/tracking and as well "advertising" (push messages), none of which I am interested in giving the business.
In my case, if an app is required, I move on to the next business where an app is not required.
Yep, but not because I'm older but because:
1. I do not want McDonald's (or anyone else) to link my orders together. It's none of their business.
2. App fatigue. I don't even eat at McDs, but I don't want an app for every grocery chain i buy from either.
3. Any app, even if not "AI" based, is going to fail in more annoying ways than a human.
They definitely seem to be squeezing more out of their franchises than ever at this point. They will push technology optimizations, but they're at a point where service and price just isn't there. If I'm spending $15+ for lunch, I may as well go to Applebees/Chili's, etc.
It’s comical how bad the gouge customers and franchises alike.
It's a weird aside, but maybe worth noting that Ray Croc was probably not just greedy. Ray Kroc was probably also racist against the Irish. A tell-tale sign is using a clown as the company mascot. The US strain of clowns were heavily influenced by "pale white face" racism jokes about Irish immigrants [0] from some of the same minstrel shows notorious for "black face" and "yellow face" and "red face". Ray Kroc was from a generation that would have easily been aware of that and would have been "entertained" by it. Ray Kroc's behavior to the actual McDonald's founders is rather easier to explain assuming it included quite a bit of Old Fashioned Racism than assuming just pure greed. Sometimes it is useful to remind ourselves that past isn't as clean as corporate memos want to paper over it.
[0] Notably, among other things: red hair, big feet, freckles, red drunken noses, loutish drunken behavior. Even the "clown car joke" is the exact same "joke" as "Mexican pickup truck" transposed across a couple of decades and about a different working class immigrant population. (Racists seem pretty lazy in how they reuse old material.) So yeah, if you ever wondered why clowns don't seem all that funny in the modern era, congratulations you probably aren't a racist. Also, now that this past horror is in your head I'm sorry for ruining Disney's Dumbo which uses all the worst of clown stereotypes and "jokes" all in the same place and eats up a lot of runtime with it, if you weren't already concerned about the "black face" crows in the movie or thought you could dismiss them as not central characters or contributing that much to the runtime.
I haven’t verified these claims, nor do I have the background knowledge to form an opinion. I don’t care. Seems plausible and I think Irish were within the definition of “black” a century ago.
Is it because you don’t believe in this? Or do you think it is astroturfing with bad intentions similar to the anti women memes plastering social media (where it is some wrong committed by a woman with 100s of bot comments saying all women are bad)? Kinda like repeating long ago marginalizations of white people to stir up a feeling of wronghood among older more conservatives white men?
Sorry your AI partnership with them failed, guy.
>> There are over 380 Ronald McDonald Houses in 64 countries. These accommodate families with hospitalized children under 21 years of age (or 18 or 26, depending on the House), who are being treated at nearby hospitals and medical facilities. [...] Ronald McDonald Houses allow families to stay free of charge.
https://en.m.wikipedia.org/wiki/Ronald_McDonald_House_Charit... https://www.charitynavigator.org/ein/362934689
As far as targeted charities go, they're near the top in terms of delivering something everyone agrees is good.
Worst was going ketovore... how the hell hard is it to put two pieces of meat in a box without a bun and nothing on it? OR getting 3x "2 strips" of bacon and counting to 6.
When "the business" is literally outside your company, you get the same dynamics as when it's in another org structure, except even worse.
That's a great point. Shazam, which has a "simpler" challenge, still trips up with identifying music in challenging environmental conditions.
This sounds like a problem for humans too - are humans just better equipped to deal with bad audio?
1) Sign taped to intercom: "Drive up to window to order". Order at window.
2) Worker standing at the intercom with a notebook and pen and a runner trading sheets between them and the kitchen.
3) "Drive-Thru Closed; Order Inside"
4) Restaurant's phone number posted over intercom.
I'm sure there are a lot more variations I haven't seen. Humans are pretty resourceful when machines fail them but they still want to get paid.
As the commenter says below, a human can intervene in many more ways when equipment malfunctions or customers have special needs that an AI just gets blocked by.
There are literally hundreds of edge cases where a voice powered drive through just stops working, from high winds to pouring rain, to thick accents, broken equipment, out of stock or seasonal items not available. Just a few of the ones I encountered personally in the wild tagging 15,000+ orders.
Humans can generally figure it out eventually.
Pretty common for two humans to have trouble understanding each other through the microphone setups, too.
Plus a touchscreen works for hearing impaired people.
Still slow, but touches register more consistently.
Much cleaner.
Most drive-thrus I've ever seen (Ohio, US) have zero cover and you are completely exposed to whatever mother nature wants to throw at you. Even the service window itself may have a tiny 24" awning that barely helps. In fact in my experience during rain it makes it worse as you get runoff right on top of the vehicle since they don't have rain gutters.
They were no doubt just told by their boss, and their boss's boss, to 'prove that McDonald's is using AI.'
Can we expect 100,000 customers per year to get within touching distance of a touchscreen, without hitting it?
Everyday I see people need to open their car door and half lean out to reach the screen. But what is option, drive away and not have money?
Couldn't we just have the screen move towards the user, using a basic proximity sensor? We've had car wash systems adapt to the car's size for quite a few years now, so it should be a relatively cheap solution, no?
I really doubt it would be cheap. As soon as you bring motors and the electronics to control them into the mix, you're going to have an increased rate of of failure. Even if the initial install is inexpensive (which I doubt), the maintenance costs would increase a lot.
simply select what you want, have your favorites saved and go. When I had my first blackberry I envisioned that you'd be able to order everything there.
Ordering a "big mac with fries and a coke" at the drive-thru window takes less time than reading this comment. Why would anyone want to spend five minutes installing an app, creating an account, figuring out how the app works, and finally placing an order?
The argument is don't force all, 100% of customers to install an app, because it gets frustrating, and then you loose some percentage of customers that just walk away to order somewhere easier.
I sign in to the apps with SSO (usually Apple, sometimes Google), so I don't need to "make" an account.
And I pay with Apple Pay which is already set up in my phone too.
If I'm on a road trip, and I need a quick meal, and my choices are "buy from place that makes me use their app" or "buy from place that doesn't make me use their app", I'm gonna choose the option that is more convenient to me, which is, the one that doesn't make me download anything, sign up for an account, and pester the hell out of me 2 hours after I have already forgotten about that particular fast food joint to let me know if that if I only buy 39 more Big Macs that I can get a free small fry.
I don't really want fast food to begin with. The food quality at any vendor has continuously gone downhill (or my standards have raised, could be some combination therein). If they want any hope of capturing my business still, they'll only get it if they keep that process as frictionless as possible.
Doesn't even need to be app driven.
Heck, it could even be written in less than 500 lines of perl.
Automating the jobs that humans are good at and pushing people to do repetitive soul wrenching jobs that machines should de better is the wrong path IMHO.
The human at the counter sets the tone and the mood, which is part of the eating. That's true even at fast food mega chains even when the order taking employees follow a script and do it time after time.
While not too dissimilar, I like to think of it as ordering something from the replicator like in Star Trek.
They do not. :(
As far as the human at the counter setting the tone and mood: I have had very few fast food interactions where this human made it a positive experience. "Meh" at best, and often not-good to bad.
Even more, for people unfamiliar with it or who aren’t comfortable with computer interfaces.
Not good for drive-through throughput.
And where Amazon “pioneered” one-click checkout, what, 20 years ago? Those kiosks have a path like view cart -> checkout -> decline upsells -> don’t use app to pay -> choose credit card -> enter table number -> tap card -> choose receipt.
Each one of those is a whole new screen. On a 40”-ish monitor! It is insane, and could easily be one screen with a couple of options. And just tapping a card should checkout and pay at any time.
Not a fan of McDonald’s but I wanted to see what they were doing with self serve tech. Nothing good, it turns out.
Another case like self checkouts of “automation” in the form of making the customer do more work than it took a paid worker to accomplish the same thing, but not actually automating a damn thing.
That's the result of the advertising department desperately wanting to track you for their metrics.
> Each one of those is a whole new screen. On a 40”-ish monitor! It is insane, and could easily be one screen with a couple of options.
This was most likely the result of some UI designer fearing that they would scare away too many folks with choice fatigue if they put all the options on a single screen all at once.
Sadly they forgot to add a button for "I'm a technically adept user who would like a single screen with all options please" to let those of us who would not be scared away by that complexity select that version as an option.
If you've never seen a Sonic, it's organized like a gas station. Replace gas pump with kiosk.
I also dislike having to use a screen rather that interacting with a person for this sort of thing in the first place, but that's personal preference rather than anything about the kiosk system in particular.
Besides, the world is full of germs, in fact you would die without them. They are no big deal.
But in a drive-through, somehow, I don't think there will be options. I bet it would be one thing or the other.
Aka some IBM b-team screwed up yet another implementation, but the contract language prevents McDonald's from saying so.
If IBM screws up a contract, it's IBM's fault for failing to execute.
If a lesser known contractor screws up a contract, it's the VP's fault for not picking a bigger firm.
When reading the article I also jumped to conclusion it was IBM's problem. But then thought it sounded an awful like McDonalds was 'hinting' that it was IBM's problem while not being able to say so.
So kind of a reverse stab. McDonalds is acting like there is contract language saying they can't say what happened, and subtly making it seem like it was IBM when really McDonalds is saying: "It was IBM but we can't say because of contracts, wink wink, but really we were a horrible customer and wrecked the project and trying to spin this onto the vendor".
Versus paying some kid $7 an hour to hit buttons on a screen.
As uninformed as I may be here, my gut says that it doesn’t make a lot of sense.
Keeping that system working is already difficult enough with a highly adaptable human at the other end of the line which can interpret your order with a broken microphone, bad language skills, a loud motor interfering and in any weather.
And openai cannot handle the "please drive though to the first booth" if all else fails.
On another anecdotal note completely off-topic: I wonder how it would interpret a horse, back in the days when I worked at the local McD we had plenty of them going though the drivethough.
Surely you don't intend to imply that the horse is ordering. Give that, there are all manner of sensors to detect the presence of an automobile or horse in the drive through and notify staff or the AI that someone wants to place an order.
My understanding is that this is a solved problem. Staff get an alert/announcement chime or recording about someone waiting because the electronics detected something. Why a horse would pose a challenge is a mystery to me. Sure, there are some devices that will work less effectively as sensors for a horse than a car, but your particular franchise just adapts to the correct sensors for its traffic.
Haha no (should have explained), but riders are too high to be heard due to the directional microphones installed and you only effectively hear the horse. Same for very high seated cars (though I suspect this might be different in the US). Trucks are too tall and would hit the building so they are not a consideration for the one I worked at.
As you said, a simple alert system might work to solve this.
Some more discussion: https://news.ycombinator.com/item?id=40701868
Sounds like a recipe for a failed project
We are currently only pushing it where the existing process involving people is ridden with imperfections -- and AI can do no worse or better, but still not perfect.
Here in the UK I always use the McDonalds app and either order for pickup or delivery to my car or table inside the restaurant.
Yeah, it's ending the current partnership, not the imminent doom of fully automated restaurants.
RIP Morgan Spurlock