Air Canada is responsible for chatbot's mistake: B.C. tribunal
bc.ctvnews.ca
bc.ctvnews.ca
I once had the misfortune of generating a batch of defective enterprise-grade SSD from a S company. That S company requires all RMA to go through the sales channel you bought the SSD from, but the sales company we used was out of business.
S has refused all attempts to RMA by stonewalling us saying that we need to return the drives thru the bankrupted company. When we explained that the company is bankrupted, S just ignored us. When we created a new RMA request, S's rep says we already have an open case, and that we need to return the drives blah blah blah.
After 5 months, in a fit of rage I typed up a 2000 words complaint, gathered all the emails/phone calls/photo evidences, and submitted a complaint to CRT ($75 fee). I wasn't expecting much, but within 3 weeks I got a call from a corporate lawyer in S company's Toronto office, asked me for the situation, apologized profusely, and asked if I can drop the case if they RMA all affected SSDs.
That day was great, to say the least.
Aside:
The CRT posts all their cases (that reached arbitration) here- https://decisions.civilresolutionbc.ca/crt/en/nav.do
Reading the cases is quite am entertaining time passer.
I had the misfortune of trying to use them for a company which had just stopped responding and in the end even though I did get the default judgment in my favour, actually enforcing the judgment still required me to go through the normal courts (which in my case was not worth the cost). But the process of dealing with CRT was nothing short of delightful.
I've also had to deal with their lack of Canadian RMA for 2 SSDs. Had to go back and forth with them and trying to convince Amazon and the Amazon seller to replace the defective drives.
Not buying any more Samsung memory products due to their essential non existent warranty in Canada
MemEx is an authorized dealer so they'll take it back and send it back to Samsung for warranty work if you buy from them.
Apparently I'm not alone and the only fix is to throw the thing away.
One of the core functions of the government is enforcement of contracts. While there are the courts, they are out of reach for most people either due to skill level or financial constraints.
Having a simple, low cost, easily accessible way to resolve contract issues puts every member of society on a more even footing when it comes to economic interactions. If we're going to build our society based on capitalism and the ability for parties to enter into contracts for things like employment, buying/selling, housing, etc, having an efficient means to resolve disputes seems like a no-brainer.
From this case:
> I find that if Air Canada wanted to a raise a contractual defense, it needed to provide the relevant portions of the contract. It did not, so it has not proven a contractual defence. [...]
> In its boilerplate Dispute Response, Air Canada denies “each and every” one of Mr. Moffatt’s allegations generally. However, it did not provide any evidence to the contrary.
From https://decisions.civilresolutionbc.ca/crt/crtd/en/item/5254...
> Despite having the opportunity to provide documentary evidence, Air Canada did not do so.
From https://decisions.civilresolutionbc.ca/crt/crtd/en/item/5249... and https://decisions.civilresolutionbc.ca/crt/crtd/en/item/5188...
> Having reviewed the evidence, I am satisfied, on the balance of probabilities, that [Air Canada] received the Dispute Notice and did not respond to it by the deadline set out in the CRT's rules.
From https://decisions.civilresolutionbc.ca/crt/crtd/en/item/5230...
> Based on the proof of notice form submitted by the applicant, I am satisfied that [Air Canada] received the Dispute Notice and did not respond to it by the deadline set out in the CRT's rules.
(I also found a fun one that hinges on an Air Canada employee's apparent inability to do basic arithmetic: https://decisions.civilresolutionbc.ca/crt/crtd/en/item/5225...)
Usually the large entity puts in little effort and relies on the fact that its (much more expensive) lawyers generally have more sway with the court (judge), are more persuasive even when their arguments are nonsense, and can just drag cases on for years until the smaller party is burned out.
> In his statement, Mr. Mackoff described distinct conversations he had with each employee, provided the supervisor’s name, and submitted the diagram he drew while trying to explain to the employees how to count the 10 calendar days. As Mr. Mackoff’s witness statement includes so much detail, and as Air Canada has produced no contrary statement, I accept that Air Canada refused to transport both Mr. and Mrs. Mackoff on February 15, 2022 and so breached its contract with them.
(But another constraint of the CRT is that you can't bring representation -- so while it was an Air Canada employee involved, it wasn't their legal team.)
All garbage. They are all falling apart now or became became uneconomically repairable within 5 years. Every single appliance repair businesses I called flat out wouldn't touch the fridge for example.. Apparently they don't provide service information or parts to 3rd parties (at least for the fridge).
I have moved onto a different brand, but waiting to see if it's any more reliable..
Sure enough, it's year 3 and the washer has stopped working. Repair guy came and decided he needs to order new parts to fix it. It's been a week or so without doing any laundry. Glad we purchased the extra warranty, but maybe we should have gone with the LG like the sales lady recommended.
I'm at the point where I don't trust any brands at all anymore. The next time I need to make a major appliance purchase I'll buy a subscription to Consumer Reports and blindly follow their recommendation - I still trust them.
My parents bought an Miele washing machine, rock solid even after pushing ten years.
It's depressing to me that we have to think about those things. I mean, "buyer beware" has always been the case, but it seems like we have to be more wary (or more wary of more factors) than we did a decade or two ago. Or maybe I'm just getting older. I dunno.
I didn't mean that, though, and I don't think it's what the other people in this thread did, either. I was thinking of the practice whereby private equity funds purchase companies and exploit the "brand equity" they've built up over the long term, whilst deliberately enshittifying them, in order to make a short-term profit for the new owners. That's been normalized, in some places, but I wish it were not, and would prefer that financial markets be regulated in ways that make it un-profitable.
Personally I have had issues with Bosch and don't trust them anymore.
The result is that now either I car about specific look, some specific features, etc and pay a bit more for them, or I just go for cheapest.
I had a squeaking drier fixed under warranty, only to have the same issue reoccur multiple times, because the rollers are just junk. It needs a new set like clockwork every year and a half or so, I have the replacement procedure memorized now.
The washer seems to be allergic to water and soap. I keep the unit in a dry location, leveled and raised off of the floor, yet, the body of the unit is rusting out, the chrome finish on the door is peeling, and when I clean it, the cycle labels wipe right off the front panel. The pump has also failed due to rust on the motor.
Absolute trash. I probably would have been better off with a $400 top loader.
Happened to my ~2016 Samsung. Turns out a hose clamp wasn't properly installed and water was dripping onto the steel floorpan and rusting everything nearby.
Fixed it and all rusting halted.
If I have to double-triple check elsewhere to make sure that the chatbot is correct, or if anything the chatbot tells me is non-binding, then what's the point of using the chat bot in the first place? If you can't trust it 99% of the time, or if the company says "use this, but nothing it says should be taken as fact", then why would i waste my time?
If a company is going to provide a tool, they should take responsibility for that tool.
This is a new frontier in short sighted customer service staffing (non-staffing in this case). The people who are on the frontline communicating with customers can convert unhappy customers to repeat customers, or into ex-customers. There's a few brands I won't buy from again after having to jump through too many hoops to get (bad) warranty service.
Please do not navigate away from this page while the trial runs. You will receive a notification when the verdict has been reached. This may take up to a minute.
Already we can’t manage to prosecute ex presidents in a timely manner before the next election cycle. If delays seem absurd now what will it be like when anything and everything remotely legal takes 10+ years and already sky-high costs triple?
I think your claim might be based on anecdotal testing. (I used to have that same feeling after my first implementation of RAG)... Once you get a few thousand users running RAG-based conversations, you quickly see that it's "good enough to be useful", but far from being as dreamy as promised.
this shit gets sold as a way to replace employees with, essentially, just the middle manager that was over them, who is now responsible for managing the chatbot instead of managing people
while managers are often actually not great at people management, it's at least a somewhat intuitive skill for many. interacting with and directing other humans is something that many people are able to gain experience with outside of work, since it's a necessary life skill unless you're a hermit. furthermore, as a hedge against managerial ineptitude, humans are adaptable creatures that can recognize their manager's shortcomings and determine when and how to work around them to actually get the job done
understanding the intricacies training a machine learning system is a highly specialized and technical skill that nobody is going to pick up base knowledge for in the regular course of life. the skill floor for the average person tasked with it will be much lower than that of people management, and they will probably fuck up, a lot
the onus is ostensibly on AI system vendors to make their systems idiot-proof, but how many vendors actually do so past the point of "looks good enough to close the sale in a demo"? designing such a system is _incredibly_ hard, and the unfortunate reality is that if you try, you'll lose sales to snake oil salesmen who are content to push hokum trash with a fancy coat of paint.
these systems can work as a force multiplier in the hands of the capable, but work as an incompetence magnifier in the hands of the incapable, and there are plenty of dunning-krugerites lusting to magnify their incompetence
Sure. They are now out about $600. They probably already laid off 500+ customer service jobs costing conservatively 30k a year each. Not including mgmt,training,health,ect. I don't think it will make a difference to the ivory tower C levels. We will just all get used to a once again lower quality help/product. Another great "enshitification" wave of the future with "AI"
It also assumes that the customer service people dont make mistakes at a similar level anyway.
Another "new normal" How come anything that is "new normal" is never good?
If it allows them to reduce costs (and there's enough competition to force them to pass that on as reduced prices), I'm fairly happy with a new normal.
See also how air travel in general used to be a lot more glamorous, but also a lot more expensive.
i found the bug.
People love to hate eg RyanAir, but their effect on prices is felt throughout the industry; even if you never take a single RyanAir flight.
(And even without looking up any data, I find your 'record profits for the last 20 years' hard to square with my memories of covid.)
EDIT: I tried to find some indices for airlines. The closest I found was https://finance.yahoo.com/quote/JETS/performance/ which didn't exactly have a stellar performance.
So I'm not sure where you get your claim from?
Airlines are weird. I think warren buffet said something about airlines being the most complicated way to guarantee losing money as a business or something like that once.
The bar LLMs have to clear to beat the average front line support operations isn't that high, as your own experience shows. And compared to a large force of badly paid humans with high turnover, LLMs are pretty consistent and easy to train to an adequate level.
They won't beat great costumer support agents, but most companies don't have many of those
A human will be more likely to say "I don't know" or pass you along, rather than outright lie.
However I would not expect an airline customer support to make up a completely fictional flight that has never existed. Maybe they could confuse flights or read a number wrong, but making one up?
Case in point, I asked my bank if they had any FX conversion fees or markup. Guy said no. I asked if there was any markup on the spread. Said no. Guess what? They absolutely mark up that spread. Their exchange rates are terrible. Just because there isn't a line-item with a fee listed doesn't mean there isn't a hidden fee in there. He's either incompetent or a liar.
90% of my job was undoing and compensating passengers for the incorrect information either the phone agent or gate agent gave them. The other 10% was dealing with workarounds to technical issues in our booking software.
And especially for something where it’s just pulling data from an internal system. There is absolutely no reason to invent made up information and saying “well humans do it all the time” is just an excuse.
On the phone with a customer service rep, I might understand a little wishy washy answer, slip of the tongue or slightly inaccurate statement. I've never really had a rep lie to me, usually its just I don't knows & them escalation as needed.
There is something about the written word from a company that makes it feel more like "binding statement".
They only need to increase the lawsuit/settlement amount by less than the amount the companies saves by automation.
I recently asked some LLMs "How many gallons in a mile?" and got some very verbose answers, which turned into feats of short story short stories when I refined to "How many gallons of milk in a mile?"
If part of the training was to only use knowledge sourced from a vector db and that it is allowed to use its trained knowledge only for grammar rules, phrasing or rewriting information then I think it would do a lot better.
Doesn't seem like many models are trained on prompts like "Question Q"->"[no data] I'm sorry but I don't know that" = accepted during training.
This would help immensely for not just for chatbots but for personal use too. I don't want my LLM assistant to invent a trip to Mars when I ask it "what do I have to do today" and my calendar happens to be empty.
If a company representative told me in writing (perhaps via email support) that I could claim a refund retroactively, and that turned out to not be their policy, I would still expect the company to honor what I was told in the email.
Phone calls are difficult more because there is no record of what was said. But if I had a recording of the phone call... I'm not actually sure what I would expect to happen. It's just so socially unusual to record phone calls.
Is it? I can not remember the last time I called some business where I did not get a “this call may be monitored or recorded for quality and training purposes…”. whatever perceived social hangups the company had they got over them and you don’t even need to ask in a 2PC jurisdiction, it’s already taken care of, just record the call.
A chatbot should be either 100% or 0%. Companies should not replace humans with faulty technology.
Coincidentally the audio recording of the conversation was apparently deleted …
This is simply applying the exact same standards to a chat bot.
but if that is your standard, you can't have an airline either
Another thought experiment: If a portion of the company's website was at least partially generated with an LLM, does that somehow absolve the company of responsibility for the content they have on their own site?
I think a company is free to present information to their customers that is less than 100% accurate -- whether by having chatbots or by doing something else silly like having untrained, poorly-paid support reps -- but they have to live with the risks (being liable for mistakes; alienating customers) to get the benefits (low operating cost).
I expect we'll see this sort of thing a lot more in the future, and probably a bit of a subsequent reversal of all of the sackings of humans once the issues (... and legal liability!) becomes clearer to people.
This got so bad that when a customer support agent at Amazon genuinely resolved my issue well once, I was surprised that it actually worked out as promised.
You’d use first contact resolution, average handle time, and their ability to stick to the flow they’re meant to (like transferring the customer to a survey after the call).
Like you say, satisfaction encourages lies. Much like sales commissions.
If I'm trying to interact with a company and they direct me to a chatbot, I expect to get useful help 0% of the time, because if help was available via a mechanism on their site I would already have found it. I expect a chatbot to stall me as long as possible before either conceding that I need a human's assistance or telling me some further hoop to jump through to reach a real human.
And if that isn't the case, I've mostly found that contrary to stereotype, many first-line tech support people are not such rote script-followers that they can't deal with skipping most of the script when the problem is obviously on their end and going to need real human intervention.
The point is quite literally to make you give up trying to contact customer service and just pay them money, while getting their legal obligations as close to a heads-I-win, tails-you-lose situation as possible. That's not the mysterious part. The mysterious part is, why did they even let this drag into court for such a small sum?!
Because most people wouldn't bother taking it to court.
If they rolled over and paid up every time their chatbot made a mistake, that gets expensive, and teaches customers that they can easily be compensated if the chatbot screws up.
If they fight it tooth and nail and drag it all the way to court, it teaches customers that pursuing minor mistakes is personally painful and probably not worth it.
Scorched-earth defense tactics can be effective at deterring anyone from seeking redress.
It's the same fundamental reason why customer support is so hard to reach for many companies - if you make it painful enough maybe the customer will just not bother. A valuable tactic if your company imagines customers as annoying fleshy cash dispensers that talk too much. Having flown many times with Air Canada I can confirm that they do seem to perceive their passengers as annoying cash dispensers.
Wait, couldn't they have tried to settle as soon as they realized it was actually going to court? I thought that was the modus operandi in the US... is it not a thing in Canada?
After the "estimated fix by ETA" came and went, I reported my ISP to the FCC. That resulted in a quick follow up from a real human.
"This is a remarkable submission," Civil Resolution Tribunal (CRT) member Christopher Rivers wrote.""
From https://www.cbc.ca/news/canada/british-columbia/air-canada-c...
Will Air Canada be legal for my friend going against company policy?
What they told it to do was to behave very unpredictably. They shouldn’t have done that.
These ones do what they "learned" from a lot of input data using a process that is us mimicking how we think brains could maybe function (kinda/sort off with a few unbiological "improvements").
Say your webserver isn't scaling to more than 500 concurrent users. When you add more load, connections start dropping.
Is it because someone programmed a max_number_of_concurrent_users variable and a throttleExtraAboveThresholdRequests() function?
No.
Yes, humans built the entire stack of the system. Yes every part of it was "programmed", but no this behaviour wasn't programmed intentionally, it is an emergent property arising from system constraints.
Maybe the database connection pool is maxed out and the connections are saturating. Maybe some database configuration setting is too small or the server has too few file handles - whatever.
Whatever the root cause (even though that cause incidentally was implemented by a human if you trace the causal chain back far enough) this behaviour is an almost incidental unintended side effect of that.
A machine learning system is like that, but more so.
An LLM, say, is "parsing" language in some sense, but ascribing what it is doing to human design is pretty indirect.
In a way you typing words at me has in some way been "programmed" into you by every language interaction mankind has had with you.
I guess you could see it that way, but I don't think it's a particularly useful point of view.
In the same way an LLM has been in directly "programmed" via it's model architecture, training algorithm and training data, but we are nowhere near the understanding of the process to be able to consider this "programming" it yet.
Its behavior incorporates randomness and is unpredictable and hard to keep within bounds on purpose and they decided to tell a computer to follow that unpredictable instruction set and place it in a position of speaking for the company, without a human in between. They shouldn’t have done that if they didn’t want to end up in this sort of position.
This is also a management failure in badly evaluating and managing the risks of a new technology.
We disagree in that I don't think that its behaviour being hard to predict is on purpose: we have a new technology that shows great promise as tool to work with language input and outputs. People are trying to use LLMs as general purpose language processing machines - in this case as chat agents.
I'm reacting to your comment specifically because I think you are evaluating LLMs using a mental model derived from normal software failures and LLMs or ML models in general are different enough to make that model ineffective.
I almost fully agree with your last comment, but the
> they decided to tell a computer to follow that unpredictable instruction set
reflects what I think is now an unfruitful model.
Before deploying a model like this you need safeguards in place to contain the unpredictability. Steps like the following would have been options:
* Fine-tuning the model to be more robust over their expected input domain,
* Using some RAG scheme to ground the outputs over some set of ground truths,
* Using more models to evaluate the output for deviations,
* Business processes to deal with evaluations and exceptions, Etc
The legal system recognizes that people, or groups of people, are subject to legal authority. This is a story about a piece of software Air Canada implemented which resulted in them posting erroneous information on their website.
It is very different than if an employee were to, in writing, make a statement that a reasonable person would find reasonable.
If the chatbot told them that they'd get a billion dollars, the courts would not hold Air Canada responsible for it, just as if a programmer put a decimal wrong and prices became obviously wrong. In this case, the chat bot gave a policy within reason and the court awarded the passenger what the bot had promised, which is a completely correct judgement.
If they decide it is reliable enough to be put in front of the customer, they must accept all the consequences: the benefits like having to hire less, and the cons, which is that they have to make it work correctly.
Otherwise, woopsy, we made our AI handle our accounting and it cheated, sorry IRS. That won't fly.
There is a common misconception about law that software engineers have. Code is not law. Law is not code. Just because something that looks like a function exists, you can't just plug in any inputs and expect it to have a consistent outcome.
The difference between these two cases is that even if a chat bot promised that, the judge would throw it out, because it's not reasonable. Also, the firm would have a great case against at least the CS rep for this collusion.
If your friend of a CS agent promised you a bereavement refund (As the chatbot did), even though it went against company policy, you'd have good odds of winning that case. Because the judge would find it reasonable of you to believe and expect that after speaking to a CS rep, that such a policy would actually get honored. (And the worst that would happen to the CS rep would be termination.)
If your friend promised you something reasonable in the course of carrying out their duties, and you honestly believed them, I think that would be legal and enforceable just as this case suggests.
If a random AC employee gave you a free flight, on the other hand, you'd be entitled to it.
Anyway, the chat bot has no agency except that given to it by AC; unlike a human employee, therefore, its actions are 100% AC actions.
I don't see how this is controversial? Why do people think that laws no longer apply when fancy high-tech pixie dust is sprinkled?
The company would be entirely within their rights to say 'this employee was wrong, that is not our policy, goodbye!'. This happens all the time with more minor incidents.
This chatbot merely said something was possible, no legally binding agreement occured.
So there was no contract but a consumed flight. The court has to retroactively figure out a reasonable contract in such cases. That Air Canada couldn't just apply the reduced rate once they learned of their wrong communication marks them as incredibly petty.
If you were standing at the customer service desk, and instead they said: "sorry about the delay, your next two flights are free", then all of a sudden this is "reasonable".
[0] perhaps because they're disgruntled and trying to hurt their employer
[1] generative models are not an exception; they're a way of telling computers to generate text that sometimes contains falsehoods
I'm sure that if the bot had said that the airline would raise your dead relative from the grave and make you king of the sky or something equally unbelievable the courts wouldn't have insisted Air Canada cast a crown and learn necromancy.
And if it was a billion dollars?
The chatbot instructed the passenger to pay full price for a ticket but stated they could get a refund later. That refund policy was a hallucination. The victim her just walked away with a discounted ticket as promised not a billion dollars.
So replacing all their customer support staff with AI that misleads customers is OK? That's pants on head insane, so why spend time trying to justify it.
One major difference is the AI wasn't your friend, another is that you didn't get it hired at Air Canada, another is that the promise wasn't $1B, etc...
If you are promised something reasonable by an agent of the company who you are not conspiring with, then the company is bound to follow through on the promise because you do have a reasonable expectation that what they are telling you is the real policy.
("Chatbot says you can submit a form within 90 days to get a retroactive bereavement discount" sounds perfectly reasonable, so the doctrine applies.)
But, to your question, my guess is that would basically be telling people not to avoid their chatbot, which they don't want to do.
Obviously that wouldn't fly. So why would it fly with the AI chatbot's advertising discounts?
You wouldn't normally expect an AI chatbot to be authorized to make offers. Its purpose is to try to answer common questions and it has been widely covered in popular media that they hallucinate etc.
I think only software engineers would think this. I don't think it is obvious to a layperson who probably has maybe never even used one before.
And they did get sued. Next time maybe they'll make sure software they connect to their website is more reliable.
They didn't program it to do that, it's a characteristic of the technology that it makes mistakes. Which is fine as the public learns not to blindly trust its answers. It seems silly to assume that people won't be able to figure that out. People are capable of learning how new things work.
This is like the people who set the cruise control in their car when it first came out and then climbed into the back of the car to take a nap. That's not how it works and the technology isn't in a state where anybody knows how to do better.
There's absolutely a technology available to make a chatbot that won't tell lies: connect a simple text classifier to a human-curated knowledge base.
The result would be a de facto ban on AI chatbots, because nobody knows how to get them not to make stuff up.
> I'm glad they're not allowed to use such unreliable, experimental technologies in their airplanes (737 Max notwithstanding).
If you use unreliable technology in an airplane, it falls out of the sky and everybody dies. If you use it in a chatbot, the customer can e.g. go to the company's website to apply for the discount it said exists and discover that it isn't there, and then be mildly frustrated in the way that customers commonly are when a company's technology is imperfect. It's not the same thing.
> There's absolutely a technology available to make a chatbot that won't tell lies: connect a simple text classifier to a human-curated knowledge base.
But then it can only answer questions in the knowledge base, and customers might prefer an answer which is right 75% of the time and can be verified either way in five minutes than to have to wait on hold to talk to a human being because the less capable chatbot couldn't answer their question and the more capable one was effectively banned by the government's liability rules.
No, the result would be a de facto ban on using them as a replacement for customer service agents. I support that for the time being since AI chatbots can't actually do that job yet because we don't know how to keep them from lying.
They could put a disclaimer on it of course. To be sufficiently truthful, the disclaimer would need to be front and center and say something like "The chat bot lies sometimes. It is not authorized to make any commitments on behalf of the company no matter what it says. Always double-check anything it tells you."
That's the point of it -- you don't have to wait on hold for a human to get your answer, and you could plausibly both receive it and validate it yourself sooner than you could get through to a human.
But what does that even mean? If Ford trains a chatbot to answer questions about cars purely for entertainment purposes, or to get people excited about cars, a customer could still use it for "customer service" just by asking it questions about their car, which it might very well be able to answer. But it would also be capable of making up warranty terms etc., so you've just banned that thing and anything like it.
> I support that for the time being since AI chatbots can't actually do that job yet because we don't know how to keep them from lying.
It's pretty unlikely we could ever keep them from lying. We can't even get humans to do that. The best you could do is keep them on a script, which is the exact thing that makes people hate existing human customer service reps who can't help them because it isn't in the script.
> To be sufficiently truthful, the disclaimer would need to be front and center and say something like "The chat bot lies sometimes. It is not authorized to make any commitments on behalf of the company no matter what it says. Always double-check anything it tells you."
Which is exactly what's about to start happening, if that actually works. But that's as pointless as cookie banners and "this product is known to the State of California to cause cancer".
I expect something that's presented as customer service not to lie to me about the rebate policy. As long as what it says is plausible, I expect the company to be prepared to cover the cost of any mistakes, especially if the airline only discovers the mistake after I've paid them and taken a flight. Compensating customers for certain types of errors is a normal cost of doing business for airlines, and the $800 CAD this incident cost the airline is not an exorbitant amount. The safety valve here is that judges and juries do test against whether a reasonable person would believe a stated offer or policy; I can't trick a chatbot into offering me a billion dollars for nothing and get a court to hold a company to it.
If Ford presents a chatbot as entertainment and makes it really clear at the start of a session that it doesn't guarantee the factual accuracy of responses, there's no problem. If they present it as informational and don't make a statement like that, or hide it in fine print, then it says something like "the 2024 Mustang Ecoboost has more horsepower than the Chevrolet Corvette and burns less gas than the Toyota Prius", they should be on the hook for false advertising to the customer and unfair competition against Chevrolet and Toyota.
Similarly, if Bing or Google presents a chatbot as an alternative to their search engine for finding information on the internet, and it says "Zak's photography website is full of CSAM", I'm going to sue them for libel.
Sure, but a billion people could each trick it into offering them $100, which would bankrupt the airline.
> they should be on the hook for false advertising to the customer and unfair competition against Chevrolet and Toyota.
But all you're really doing is requiring everyone to put a banner on everything that says "for entertainment purposes only". Because if something like that gets them out of the liability then that's what everybody is going to do. And if it doesn't then you're effectively banning the technology, because "have it not make stuff up" isn't a thing they know how to do.
If that means companies can't use chatbots to replace customer service agents yet, so be it.
But what does that matter? So someone posts on Reddit how to trick the chatbot into offering a rebate and then 75% of their customers have done it by the time they realize what's going on and now they're out of business.
> If that means companies can't use chatbots to replace customer service agents yet, so be it.
You're still not articulating any way to distinguish "customer service" from any other functioning chatbot. A general purpose chatbot will answer customer service questions, so how does this not just ban all of them?
If, instead of a chatbot, this was about incompetent support reps that lied constantly, would you make the same argument? "We can't hire dirt-cheap low-quality labor because as company representatives we have to do what they say we'll do. It's so unfair"
If Microsoft puts ChatGPT on Twitter so people could try it, and everybody knows that it's ChatGPT, and then it started offering companies free Windows licenses, why should they have to honor that? It's obvious why it might do that but the purpose of letting people use it wasn't so it could authorize anything.
If the company holds a conference where it allows third party conference speakers to give talks, which everybody knows are third parties and not company employees, should the guest speakers be able to speak for the company? Why would that accomplish anything other than the elimination of guest speakers?
I think that banning lying to customers is fine.
It sounds like you meant to say that they didn’t _intentionally_ program it to do that. They didn’t find the system under a rock and unleash it on the world; they made it.
Yes. And therefore people should be able to assume that the answers are correct.
Some people have heard of ChatGPT, and some of those have heard that they hallucinate, sure. But that's still not that many people. And they don't know that a question answering chat bot like this is the same technology!
Why is that a necessary requirement? Something can be useful without it being perfect.
If I have to double-triple check elsewhere to make sure that the chatbot is correct, then what's the point of using the chat bot in the first place? If you can't trust it 99% of the time, or if the company says "use this, but nothing it says should be taken as fact", then why would i waste my time?
Awhile back another commenter called them a “demented Clippy” which about sums them up for me.
Because you can ask it a question in natural language and it will give you an answer you can type into a search engine to see if it's real. Before you didn't know the name of the thing you were looking for, now you do.
> If you can't trust it 99% of the time, or if the company says "use this, but nothing it says should be taken as fact", then why would i waste my time?
The rate at which it makes stuff up isn't 99%, is the point. For common questions, better than half of the answers have some basis in reality.
What you're describing isn't what you expect, it's what you wish were the case even though you know it isn't.
A couple weeks ago Oman Air cancelled the return leg of a long distance flight due to a change in schedule.
They offered to reroute me with the Qataris via Doha.
I preferred to cancel the (in principle non-refundable) flight and make different arrangements.
The money,~2'500$, was credited to my card 4 days later. Including fees paid for preferred seats.
It's a shame that they stopped service to my city. Beause it's a great airline, which always provided stellar service.
What happened to the OP is, therefore, unusual.
I flew through them last year. Check-in was a bit of a hassle but the plane/service and the layover were great. The price was very competitive (cheapest) and yet the plane was empty. I expected them to either fold or downsize.
> "While Air Canada argues Mr. Moffatt could find the correct information on another part of its website, it does not explain why the webpage titled 'Bereavement travel' was inherently more trustworthy than its chatbot. It also does not explain why customers should have to double-check information found in one part of its website on another part of its website," he wrote.
The chatbot included a link to the detailed rules, which contradicted what the chatbot told the customer.
> "While a chatbot has an interactive component, it is still just a part of Air Canada’s website ... It should be obvious to Air Canada that it is responsible for all the information on its website ... There is no reason why Mr. Moffatt should know that one section of Air Canada’s webpage is accurate, and another is not."
https://www.businessinsider.com/car-dealership-chevrolet-cha...
In the Air Canada case, it was a clear good-faith effort to understand the rules of a fare he was legitimately entitled to.
There are lots of things that an employee might say that would not be reasonable, even if they had no malicious intent.
"Generally, the applicable standard of care requires a company to take reasonable care to ensure their representations are accurate and not misleading."
"I find Air Canada did not take reasonable care to ensure its chatbot was accurate."
"Mr. Moffatt says, and I accept, that they relied upon the chatbot to provide accurate information. I find that was reasonable in the circumstances."
What is described here do happen when employees send mails with explicit promises, but gets harder when only the company has proof of the exchange (recording of the call). Chatbots bridge that gap.
But Chat Bots often provide a "Save transcript" feature, or even default to emailing you a copy if you're in a Customer Service type environment where it knows your email. So those are both a lot easier than setting up call recording.
This is an appropriate outcome, in my view. I'm as pro-AI as they come. But I also recognize that without a clear standard of service delivery, and in an industry inundated with M&A instead of competition, that a chatbot isn't a helpful thing but a labor cost reduction initiative.
Good.
Of course the argument is absurd, but this is exactly where companies like this would love to go. Virtually free support staff that incur the company absolutely no liability whatsoever.
It's literally just corporate ignorance as a business strategy. What's sad is, on the large, it works in their favor.
In any case, it's an equally good signal that you don't want to fly with them.
It seems a little weird to be able to (and also practically do) rule differently in the same situation
They may consider precedent rulings as factors in the decision, but those earlier rulings themselves do not automatically become law for all future cases on the same subject.
Not quite that simple. The word we use is jurisprudentie but it means the same. Opening up the Dutch Wikipedia article and clicking on the English version of the article with thst name, you end up with "Case law, also used interchangeably with common law, is a law that is based on precedents"
I dove into this when I first heard of the difference between continental law and common law, and found it to be mostly a matter of wording. The principles are opposite but the effects very similar. It's not as though common law countries have no politicians making legislation, or as though there is no precedence in continental law countries.
Dutch foundational law (I think the very first article) says "everyone is treated equally given equal circumstances": such an equality principle would be incompatible with different rulings in identical situations. I imagine most countries have equality as a foundational principle, hence I'd be interested to learn: In which country would rulings not set precedence?
No two court cases are completely identical. Precedents are an important reference but they themselves do not automatically decide the outcome in civil law jurisdictions
It’s like “yeah we don’t really know what it’s going to do and it might screw up but whatever just launch it”
Really surprised me that so many companies are willing to jump into LLMs without any guarantee that the output won’t be something completely insane
The future is mass unemployment with all resources being split between a handful of trillionaires who control the AIs.
You're presuming the rich care about "the system". That they have morals or ethics. They do not.
Of course they have. You're parroting some low quality extreme left talking points. Rich people are people just like you and me, with their own motivations, goals and internal values. Dehumanizing them won't solve society's problems.
Wealth is the ability to trade with people. If there were somehow only 10 "employed" people in the world, then the economy is 10 people big, which means none of them are wealthy.
One way to see that this scenario is absurd is that it's literally the plot of Atlas Shrugged.
And the money saved with staff will end up in the stockholder's pockets, who will consume more.
How does using a computer suddenly wash away any responsibility? Like if Air Canada's desk agents were all a separate company and they told the guy the wrong information isn't Air Canada still on the hook for training their sub-contractors?
Why are you saying this as a comment to an article where literally the opposite thing happened?
You'd best understand this today.
AI is cool, but the spoils are going to go to staggeringly few people and in most of the West there are no real safety nets for people to fall on.
If I lost my job tomorrow to AI then any desk job I aim for might be gone in another 2-3 years, before I even finish retraining.
Any content creation is going to be flooded out and most creators don't even make any money even today. It's a marketing role with strong Pareto distributions.
That mainly leaves physical labor and person to person jobs.
I would say great, freedom from labour, except that's not how it works.
They appear to mainly be going to Nvidia investors, which is basically anyone with a retirement account.
I have also noticed an increase in automated call systems just flat out hanging up on me. As in: "We're experiencing higher than normal call volumes and cannot accept your call. Please call at a different time. Goodbye. <click>" How am I supposed to get help in such cases?
We've allowed companies to scale beyond their means, and they're getting away with more and more.
UPS destroyed a suitcase of ours and basically told us to go f ourselves. We could have sued in small claims court, but that's what they're betting on, that most people just give up.
And the chatbots are just terrible. And these days, the human representatives available have even less information than what the chatbots are provided with.
I keep wondering how this could possibly be superior to just showing the 4 buttons…
Corporate software has always been sloppy especially if said corporation isn't centered around said software (true for Air Canada)) and the technology is in an early adopter stage (true for LLM chatbots).
The decision makers aren't well-versed in these technologies themselves, because getting to where they are did not require knowing how to properly use those technologies.
They get to automate a large chunk of their customer service and when the chatbot does something really stupid they can tap on the sign that says "We're not legally bound by our chatbot"
Maybe they were thinking, if we spend a bit more money on lawyers, we can try this crapshoot.
However, that's not what Air Canada wanted. And I'm not saying they should have won either. Just that, that's what they wanted.
Because if they can ignore the output of the chatbot when they want to, they can gut their customer service department.
Pretty outrageous for the airline to try to claim the chatbot was its own legal entity.
But at some point we're going to see more cases in the grey area in between. What's the important difference?
In both cases I think the result would be the same if the chatbot had been a human. GM doesn't have to honour every promise a sales rep makes, even if that rep is nominally entitled to enter into contracts for the company - otherwise someone might agree to sell their whole stock to an accomplice for $1. The same applies to Air Canada, my buddy there can't "advise" me I get free flights for life and have that honoured.
So where is the line? Is it about good faith by the customer, or about what a reasonable person might think the company would offer them?
[0] https://twitter.com/ChrisJBakke/status/1736533308849443121
The law doesn't protect a company in the world you laid out, internal compliance and controls do. A sales rep in a company with bad controls may well do exactly what you laid out.
I'm a cashier at Walmart. One day I sell to you for $100 - not just everything in the store, but the building, the local distribution centre and even the corporate headquarters.
And in your view - Walmart has no legal recourse against me or against you? They should just peacefully vacate the buildings and hand over the keys? Their remedy to this is to discipline me according to their internal controls - maybe put me on a PIP and remind me that the company handbook forbids these deals?
No, this just isn't a deal that can happen. It's not about reasonableness, consideration or unconscionability - the deal is void even if the buyer agreed to pay $100 million. It's not about good faith on the buyer's side - it's void even if you thought I was the VP of Real Estate. I can't sell Walmart's property at any price, even though I'm otherwise empowered to do business on behalf of the company.
This judgement remains in the sensible part of law and in doing so, sidesteps a massive, unexplored, and highly problematic can of worms.
Anyway, let's take a moment to thank Air Canada for this progressive stand for the individual legal autonomy of artificial persons.
More likely, they blamed a third party vendor that developed, configured, or hosts the bot
Which sounds like a similar situation to when your taxi breaks due to a mechanic's shoddy work: it's not the passenger's fault that your mechanic sucked, you were contracted to get them from A to B and may be on the hook if you stated you'd get them there on time. Here, it's not the user's fault that the chat bot was shoddy and stated something that they now don't want to fulfil. If AirCan wants to blame their vendor, they can go right ahead but this person has a right to this reduced flight price independently of whether AirCan gets the money from their vendor
But explaining all that instead of saying "haha they claimed the chat bot is an independent entity!" probably gets shared less (it's yesterday's top comment after all) and thus fewer conversions from website readers into subscribers
They should have paid the customer immediately and then took it up with their vendor. If they want to take their vendor to court, they can do that separately.
"Give me a real human" is usually what I say when it seems like I'm talking to a bot. Unfortunately, there have been times when I later discovered that the "bot" was actually a real human that was just acting like a bot!
While AI may seem to be improving, I always keep in mind the possibility that the opposite is also happening; and if you don't want your job replaced by a bot, perhaps you should not be acting like one.
Call center folks, especially the first couple of layers of them, are on scripts. They have decision trees and what to say written down. Bots can be much more dynamic than them. It's a pretty terrible job unless you're fairly uniquely predisposed to liking that kind of work.
They were notorious amongst the stranded-abroad community during COVID for selling tickets on flights they weren't operating and had no intention to operate, then refusing to refund, except with credits that also expired before they intended to operate.
Scammers from top to bottom.
Didn't they freeze hundreds of people's bank accounts, with no due process, for peaceful political demonstrations? https://www.bbc.com/news/world-us-canada-60383385
No better than Nigeria: https://www.hrw.org/news/2021/02/11/nigeria-finally-unfreeze...
> "In effect, Air Canada suggests the chatbot is a separate legal entity that is responsible for its own actions. This is a remarkable submission. While a chatbot has an interactive component, it is still just a part of Air Canada’s website," Rivers wrote.
This could be an article on The Onion. Unfortunately, I suspect this won't be the last time companies try to weasel their way out of the consequences of how they use AI in relation to their customers (i.e., us)
A human at least can bear responsibility. A chatbot cannot. That responsibility has to be absorbed by something. A company should be much more responsible for the actions of programs they run, than people they hire.
They are buried in compensation claims right now due to them claiming that crew shortages in 2021 and 2022 were out of their control, and the government regulators disagreeing with them.
My guess is that no one with any power bothered to look at this until it was too late to settle, and they thought it was worth the cost of fees to see if they could get out of paying.
How'd anyone let this go to 'court' (I'm not Canadian, it's a tribunal idk what that is) for $600. And I'm guessing it's Canadian so it's more like $400 US. What kind of point were they trying to prove here.
I legitimately think you could talk amazon support into giving you that over a broken product.
The Civil Resolution Tribunal is better known as "online small claims court". It's something BC introduced a few years ago to streamline the process.
They have legal authority, but you can always appeal to the provincial courts (which almost never works out, I think they agree with CRT decisions in 95% of cases)
Air Canada baffles me. Their front line employees are powerless and frequently hostile. But I have never submitted a complaint to corporate without being given at least $200 CAD worth of flight credit. Most recently I was yelled at and hung up on by a customer service agent, I got a coupon for 20% off any itinerary with up to 4 passengers. I’m not even a member of their rewards program!
So I submit my “complaint”.
A few months later, I get a response clearly showing that they didn’t read what I wrote, and almost certainly didn’t put in any plan to fix it; but they gave me a $400 credit.
I wasn’t even angry in my “complaint”. Maybe I need to be nicer in my actual complaints in general in the future.
Thought it was a scam when I got the response but I’ll take it.
FYI, to a Canadian, $600 CAD feels like what $600 USD feels like to an American. Canadian wages aren’t 30% higher in numerical value than US wages.
You can look at random things like groceries, homes or car insurance to see that this isn't really true. 600 USD (or even 600 CAD) goes a heck of a lot further in most of the USA.
[1] Guess but not complete wild-guess multipliers
The ideal situation is you get a remote job in Canada for 1x salary and live in a nice place with 0.3x houses. That is my current setup.
I was talking with an ML engineer that told me they had a lot of success fine tuning a LLM on their internal docs. A chatbot could solve about 70-80% of questions without the need of human intervention.
However their next big idea was to fine tune the LLM with the company financial data so that the finance department could get the information they need without custom queries or tech skills. We are just a few steps away from LLMs feeding hallucinations to decision makers and then those acting on bogus data.
Today, the robot showed more humanity than Air Canada's human leadership. That was an accident; the future could be the opposite of that. You could program machines to be "better" employees than humans, more aligned with organizational goals like "maximize profit at absolutely any cost", or "win wars at absolutely any cost", or "win elections at absolutely any cost". We humans aren't completely aligned with our teams; we have moral scruples that limit us—we can't achieve the 100% "absolutely any cost" part. I think that might suddenly change, and we might find ourselves drowning in an unexpectedly in-human world.
(This was inspired by, I forgot who wrote it (?), a writer's observation about the evils of war being easier when morality is distributed. The commander who orders the atrocity, doesn't do it; the solider who commits the atrocity, has no agency in his actions. Both feel reduced culpability, and can go farther in effecting their goals than an individual acting alone).
But before either they quoted me a solution or escalated to support.
Now it makes up a non-working solution.
I’m with you. They should be held to the information they give out. Short of an employee purposely maliciously giving out bad information it seems like not making stuff up should be a basic requirement for them to operate.
In liability, none, but it'd at least be more understandable if it was an LLM, rather than something that should have been hard-coded with the right answers.
LLMs don't somehow invalidate the work of their predecessors. Chat bots aren't new.
I'm not really sure why you brought up LLMs at all. Are chat bots synonymous with LLMs now? I sure hope not because then this sort of scenario only gets worse.
I didn't
> I wrote an "AI" chatbot in highschool and it certainly didn't reproduce hard-coded "right answers"
Sounds like it would have been a poor choice for a customer-service bot then?
And it would certainly have been a poor choice for customer service, but I have definitely used chat bots that are far worse than that one was.
You're right. LLMs are now the de facto standard implementation for Support Chatbots. Almost every chatbot platform offers a AI Chatbot product in some form.
- they're also frequently shown to be prone to hallucinations - and also shown to be tricked - and can be gamed into breaking it's prompt cage
https://twitter.com/ChrisJBakke/status/1736533308849443121
This case therefore sets a precedent for these scenarios, with or without a disclaimer that you should confirm this information with the dealership. If you assume the liability for the accuracy of "ye old bot" responses, then it raises the possibility that you assume the liability for the accuracy of "ye new bot" responses.
My opinion is that once the AI wild west phase has ended, and the legal reckoning is upon it, everyone will learn that using AI does not absolve one of liability. This would essentially kill the dream of full self-driving automation, among other things.
Replicable, intelligent but fallible and disposable minds have incredible potential to positively impact our society. But somewhere there is an ethical and moral boundary to be crossed.
It's the journey, not the destination.
That's a completely different argument and much less alarming.
Actually, it's just as alarming. The entity behind the chatbot might be a separate legal entity, but that doesn't absolve the airline, who outsourced a function bound to their terms and conditions.
If they literally tried to absolve blame by assigning personhood and liability to the bot, that's insanely bad.
This is the same company that just got slapped for making a disabled dude crawl out of the airplane when air cans special chair didn't show up. I shit you not.
Would you use a company tool that even the company doesn't have faith in?
And if companies remove human reps from the equation and only offer chatbots, then I'm sure eventually the regulatory agencies will step in.
https://www.thestar.com/news/canada/air-canada-apologizes-af...
https://www.cbc.ca/news/canada/british-columbia/cta-fine-air...
https://www.theglobeandmail.com/news/national/dog-that-escap...
Someone could make a killing making a fake airline, taking payments for flights, and then cancelling every single flight and only refunding the people who fight back hard enough.
Given how (relatively) significantly-regulated airlines are in the US, I doubt that this would work for long, and I expect that the FAA would make sure that the company lost a substantial amount of money for trying that shit.
They did, it's called Air Canada. I was scammed out of Japan tickets by them during COVID, they have still not refunded me and they expired the credit for the tickets. I have tried every customer support avenue to no avail. It's a scam airline.
It required some persistence but Air Canada did abide by the regulations in my case last year. My flight from Vancouver to Toronto was delayed by a couple hours and I missed the connection to Edinburgh. They, as required, booked replacement flights via Air France (not a partner airline) as the next Air Canada or partner airline route would have departed more than 9 hours late.
[0] - https://rppa-appr.ca/eng/right/flight-delays-and-cancellatio...
"This chatbot is provided for entertainment purposes only. If you trust anything it says, you only have yourself to blame. Reading this message waives your right to sue us."
I'd love to stick it to Air Canada too, but Canada is (hopefully) less litigious than the US.
Hallucinating chatbots, automated YT copyright strikes, Insta accusations of bothood because you clicked too fast, Amazon nuking author accounts because it gets confused, self-driving cars that don't self-drive - and so on. They're all the same problem.
At best these are large corporations automating processes with unacceptably poor reliability. At worst, they're hiding deceptive and manipulative practices behind algorithms so they can claim they're not responsible for damage caused.
Maybe my flight tickets aren't valid either because I foolishly purchased them on aircanada.com and their website is known to have bugs? Or the ticket lady punched the wrong thing into her computer? Or I should have known the plane was going to be overbooked?
Cost of Air Canada lawyers well exceed the judgement here. Why would they bother fighting this?
"But when he applied for a refund, Air Canada said bereavement rates did not apply to completed travel and pointed to the bereavement section of the company’s website."
Does this mean they actually took flight with no issues, but then requested a refund afterwards because it was for a funeral?
Then looked up the word and my initial association with something negative was correct: it's about death. The only explanation that I think fits all the article's statements is this:
0. AirCan offers a discounted rate if you fly to someone's death, probably because of the country's size (it's a weird concept to me as a Dutch person!)
1. Person asked the bot how to get this discounted rate, what papers AirCan needed to see
2. Bot said (not shown in article, the only relevant-looking link goes to some stupid news category page): you get your discount afterwards, not before. This seems to be phrased in the article as "refund", but I take it to mean "partial refund to the amount of the discount"
3. Person flies, then applies
4. AirCan now says: you can't apply for this anymore after the travel was completed
Would be interesting to see if a cottage industry can open up around prompting inaccurate information from company AI info to reap via lawsuits.
1. Replace customer service agents with shitty LLM
2. Distance yourself with shitty service by shitty LLM
3. Profit.
If it's the latter, I'd assume that Air Canada would be able to go in and check why the bot would give a wrong answer, most likely outdated policy information, or a misreading from whomever entered the answer to that prompt.
However, if the bot is based on an LLM, then what's the point? It's apparently worse than an old school bot, in that it cannot be trusted to give correct answers, it's just better at understand queries.
There was a quote in the article "I'm an Old Far and AI Makes Me Sad":
“If we open up ChatGPT or a system like it and look inside, you just see millions of numbers flipping around a few hundred times a second,” says AI scientist Sam Bowman. “And we just have no idea what any of it means.”
the chatbot as a separate legal entity? do they mean like a contractor’s service, or did they mean like a distinct AI creature
Like, could you spawn up a local LLM, have it take out some loans and transfer you the funds, and then "kill it" (^C), so the loan liability dies with the LLM?
Edit: Whoa, apparently several do. In my 30 years of flying, I never knew this! https://travel.usnews.com/features/bereavement-flights#alask...
If an immediate family member passes away, some airlines will give you a discount.
Editing the HTML code of a webpage that is open in the browser is a key step in one of the popular IT support scams that are covered by YouTubers.
But how else to present evidence of chatbot misinformation, I’m not sure.
This to me is a cautionary tale against deploying cheap small LLMs instead of using larger models. I think 7b models are very tempting to many business people who may value profits over just about anything else.
This is a fundamental issue with the underlying technology, one that many companies in this space refuse to reckon with.
A lot of "this can be automated with AI!" startups are relying on the basic assumption that hallucinations can be tolerated - cases like this really narrow the field of use cases where that is true.
Imagine all the person had to do was type "bereavement" in a search bar and it instantly matched with the bereavement policy. What more does a person need?