Hacker News Reader

Comment by Karto | Hacker News Reader

It might be a stunt, but stuff like that happens. Many years ago I used to work for a large European company that ran a booking system for several hundreds of airlines. The system was used as a back-end by more than one famous online booking site. Currency rates were updated automatically. Once, on a new year's eve, at midnight, a glitch slipped into the Canadian dollar conversion rate, setting 1 CAD to 0 EUR. All of a sudden, all Air Canada flights were for free for European customers, at what might be the worst time of the year : "- so, this year's resolution : we take time to travel. - hey look honey, Canada looks cheap. - deal done, book right away, and let's open one more bottle of champagne !" I don't know how many AC flights were booked before someone realized and an on-call guy fixed the conversion rate. However, I know that all those free bookings remained valid, and were offered by my company who payed all of it directly to Air Canada.

View on HN

A cigar retailer I used to frequent had a buy 1, get 1 free special on a variety of cigar 5 packs on their site. One day, a cigar I loved was no longer part of this deal, so out of interest, I found the id of the product and placed it in the request. It worked!

That was a problem. So I emailed them to let them know that it was an issue, and received a phone response. I was told "oh, that's not a big deal." I responded by telling them to place a $500 box of cigars in the field.

NOW it was a big deal.

Amadeus?

I remember at some point the French computer retailer had an issue when running a "buy 2 get 1 free" offer on blu-rays. One could remove the two bought items from the cart and the free one would remain. Repeat ad-libidum.

Last year Amazon ran an offer where all photographic gear cost $94. All of it, even lenses which retailed for over 13k. https://www.usatoday.com/story/tech/2019/07/19/amazon-prime-... If there is one regret I have in my life, it was not checking the prime day that fateful morning.

I won the lottery on a similar issue. I was looking at monitors on Amazon using a business account, which sometimes offers volume discounts. Instead of a $10 discount, someone had set the unit price to $10 if you bought more than one, for a $1100 monitor. I bought five. I expected the order to be cancelled, but a week later five monitors showed up.

Along the same lines, I was shopping for fuel injectors and someone listed a particular part number at like $1.50ea. So now, years later, I have a peanut butter jar fulled of 24lb/hr EV1 injectors that are not the right flow rate for anything I own but hey, maybe I'll use them eventually. It's nice to find a pricing error but it's rare to be in a position to actually use or be able to re-sell multiple units.

Similarly long ago, a french amazon like site offered (alapage?) a 5e rebate for new customers, one coupon per person, and a person being identified by its delivery (fullname, address) pair. Except that they used string equality as comparison which means every typo combination got you a new account.

Suddenly anything <= 4.99 was open hunt. A month or so later, most item were now 5.01. And possibly uniqueness was now put on credit card number, people can't get more than a handful.

I never could find a figure nor an article in the news but considering how fast this spread around, it was an expensive mistake.

Recently my company (well known company/brand) someone created a coupon code that made your total 0 dollars.

Apparently around 400 orders had been made using it before it was found, which on the scale of things isn't as bad as it could have been

From what I followed it didn't seem malicious, it was a broken attempt to implement a referral bonus

On what level do those happen? Actually tweaking code, or some higher-level interface? (I wonder why this is even possible to do)

CMS type interface

Technically it didn't make any order free actually... just 200$ cheaper and you could use it as many times as you liked.

They made a discount with the correct amount, but it was supposed to require a referral from a much bigger ticket item and they missed that

I always wonder why systems like this so rarely do even basic plausibility checks before data is updated.

IKR

Once my brother wired 10M EUR from Austria instead of 10M HUF. The exchange rate is above 1:300. Needless to say, he didn't have 10M EUR on that account. Like, ever. Not even close. I still have no idea why the bank let him wire more money than he had especially on this magnitude. They reverted it but we needed to cover the spread which caused an almost 10k eur loss. That hurt.

I have once accidentally copied my one time password into my ebank for payment (I was trying to copypaste the amount from somewhere else, it didn't take, that happens often with PDF and then the clipboard contained the previous copy) and it would've let me do it if I don't stop it at the confirm screen. It's mind boggling.

It saves so many headaches. I used to support a system with an automated fallback. If today's feed wasn't in by cutoff then the previous day's feed was used.

Unfortunately we'd sometimes get partial or corrupted feeds. Partial feeds triggered investigation and possibly a manual rerun and corrupted ones often halted the system.

Because we only used monthly numbers for reporting, delaying and rerunning any other day was pointless beyond standard root cause analysis to prevent recurrence. And this system had hundreds of feeds so at first there were almost daily issues.

So I added a check to throw out any deviations over two sigma from the median of the last 30 days' good feeds which knocked out 99% of our data quality issues. I got in a boatload of trouble for different reasons but that's another story.

>I got in a boatload of trouble for other reasons but that's another story.

You cannot just drop that line and walk away. Storytime?

This predated DevOps: we were using the Carnegie Mellon Capability and Maturity Model (CMM) which split roles into Plan, Build, Operate and Control with me in the Operate tier. I proposed the change, got Build to schedule and release it but I didn't get buy in from the architect of the system. He was toweringly, incandescently angry that I had dared touch his design. To the point where I think he tried to get me fired over it. Fortunately cooler heads prevailed because really I was right and it was all just an ego thing on his part. I took a page from my boss who was masterful at dealing with these kinds of things. We buried the architect in all things operational to the point where he eventually asked us to stop and grudgingly allowed we knew what we were doing. I don't know if he believed it or just wanted the emails to stop.

Reply on news.ycombinator.com