Hacker News Reader

(I didn't mention that there was a pregnant pause after she clicked the Admin Home link & saw the admin page in all its glory. I think the only sound was, as Scott Adams put it, the sound of eyeballs getting really big.)

Do remember that for every real serious problem such as this, American Express and any other large company receives a hundred or a thousand calls for non-issues like phishing or minor issues like a compromised card number to cancel and reissue. The system is optimized for these common cases; the edge case of a real public vulnerability will require extra effort and it's not a failing of Amex that the system is so.

when the cost of a false negative is so damn high, you'd think they would know not to filter so aggressively

at some point, all the people dealing with the false positives will be costlier.

Seriously? I won't take American Express anytime soon, I can tell you that. If this gets picked up in mainstream media, it is devastating.

More likely: people will be assured the problem is fixed, and then not care.

Let me rephrase: you'd hope they would know...

If they didn't filter aggressively, that would mean that their more technical staff would be deluged with crap.

And then quit.

I was operations/development at an ISP once, had three layers of acceptably competent support techs between the customer and I and a serious reputation as a fire breathing dragon if a non-issue was escalated to me - but -still- regularly lost half my day dealing with escalated "urgent problems" that were neither.

I agree in principle, but a core part of their business is maintaining the integrity of a system. When you combine this with some of the statements they have made regarding their password policy, it makes me as a paying customer very concerned about their approach to cyber-security. THey have excellent fraud prevention, but I'd really like to see them make some strides in this area.

> it's not a failing of Amex that the system is so.

Actually, it is. They failed. Their system was open for years in secret, and at least hours after someone tried to point out the problem to them.

It's not the CS rep's fault. But it is their boss's fault, all the way up to the top.

You misinterpreted me. What I said is that the extra effort needed to report this vulnerability isn't a fault of the Amex customer service system. Of course it's a faulty deployment process that allowed this to happen in the first place. But as far as escalating a trouble report from an ordinary consumer, what happened does seem pretty reasonable.

Flip it around: what if Amex (or any other large company) made it easy to escalate everything to a technologically capable supervisor right away? Those supervisors would be deluged in uninformed, irrelevant, and just plain wrong security reports. Filtering out the signal from the noise in the security landscape is a monumental task in itself. As tech savvy hackers, we always think we're entitled to say "I know what I'm doing so escalate me over the idiots", but how does a company or CSR tell whether that's actually true?

I would assume a credit card company or bank had a technical security support team.

You are absolutely correct.

The dozen or so hackers in this thread that expect that "security vulnerability" is some magic keyword that gets you talking to the head technical honcho of the security group have probably never answered phone calls for a big company. Phone support for somebody like AmEx is a huge burden of cost and manpower; the structure of the tree has been set firmly in place since the 1980's to take care of the most common 90% of issues using the least-paid person available. I'm sorry, if you're in the long tail you will just have to expect to wait extra. That goes double if you are not a cardmember (read: paying customer).

I am surprised that the above person in Australia got through at all, and that the CSR had latitude to try to spend time replicating the issue. In my opinion, for a credit card company, 20 minutes and a positive conclusion for a matter as rare as reporting a webapp vulnerability is a success.

Sure, none of you should be thrilled about the situation because as technically-oriented people with generous motives the system is not set up to serve you. But that's not a failure of the system, except maybe from your own individual perspective. Believe me, AmEx has done the cost-benefit analysis and they are saving boatloads of money by having those rare well-intentioned hackers listen to some hold music, because it is too expensive to sort you out from the thousands of loonies that got a phishing email. Security breaches are an acknowledged risk and they are already prepared to absorb their effects on multiple levels.

In a way it is a failure of the system, in that it is much easier to simply post the vulnerability on your blog or a full-disclosure mailing list than 'officially' report it. This could potentially cost them large amounts of money.

Why would finding a vulnerability give you the moral imperative to waste so much time reporting it? Especially if you're not a customer or otherwise affected by it? I know I wouldn't.

This is why companies like Google have a security issue submit form. Sure, some lower-wage people will be filtering it, but at least they will have had training to separate the important from the unimportant problems. And for a bank, security is even more paramount.

> Sure, none of you should be thrilled about the situation because as technically-oriented people with generous motives the system is not set up to serve you. But that's not a failure of the system, except maybe from your own individual perspective. Believe me, AmEx has done the cost-benefit analysis and they are saving boatloads of money by having those rare well-intentioned hackers listen to some hold music

Which is why we shouldn't jump through their hoops. If we do we let them get away with it. If we didn't they'd be forced to pay more attention.

The well-meaning person in this thread did them and us a disservice by going so far out of his way.

Funny thing is: he didn't even try that, because he didn't accept under any circumstances to communicate by phone, fax or snail mail.

He made this big, boldface disclaimer with twitter screenshots and all trying to claim "best effort is good enough" and "they won't listen", but all I saw was them listening and him refusing to speak.

Erm, no. Dude if I spend my time figuring out vulnerabilities to your system and don't exploit em, instead help you close them, and I am not even a cardholder... I will not jump through any hoops for any amount of my time. They owe me, I owe them shit. I am being kind and generous by not exploiting or giving the exploit to others, or using it to fuck up AMEX reputation.

Especially true if I want my anonymity preserved.

Jumping through hoops? Dude he got in contact with someone. Instead he got a "piss off" response.

Well, I didn't get a "piss off" response, I got a nice warm thank you. Perhaps he could have persevered just a bit? I had to do a bit of hand-holding to get the CSR to whom I spoke to understand the problem, but once she did, it was easy.

At first glance you made some progress and that makes him seem unrealistic in his demands for electronic communication, but what makes you think that the issue was reported properly upstream? I think that you got lucky to find someone who understood that it was a real problem, and unless they have an internal escalation procedure in place, there's a decent chance it will die with her or her supervisor. There's really no way of knowing if your report had any effect or not. Other large corporations have measures in place to handle vulnerability reports, it seems like a problem for a large CC provider to not have a clear procedure in place for handling these issues.

I have to agree with the others here, while you may have done what you believe is the "right" thing, you have absolutely no idea if that avenue of inquiry went anywhere, and based on my experience working in an enterprise, I would guess that even if it did go anywhere from there it would take weeks for meetings to get scheduled, and months for people to get assigned to actually do anything about it.

When a non-customer does you the courtesy of pointing out serious flaws in your system, you do not ask them to detail it publicly via twitter.

Nor is it a good idea to make them jump through hoops. You know what method of disclosure doesn't have hoops? Posting an email to the Full Disclosure mailing list.

Look. I'm all for giving the company a chance but if you put up arbitrary hoops up for me to jump through... Why shouldn't I take the path of least resistance again?

I for one think it's a seriously unrealistic expectation to think that AMEX or insert large corp here will handle security vulnerabilities over twitter.

It's the equivalent of telling a teller or their doorman about it.

He was trying to handle it over email. He was just asking the doorman for directions to the manager's office.

No, he did not appear to want to discuss it over twitter either. They offered that.

I assume he was looking for a specific email address and perhaps a PGP key. Sure, that would be nice. But using the telephone is a pretty common method of transmitting important, time-sensitive information.

Worrying about keeping the information private is inconsistent with posting it publicly.

To be fair, he asked repeatedly for a proper security contact and claims they don't publish one for whatever reason.

If the only way to contact you is through clueless support people who have a script that doesn't include your option, yeah, that's a problem. But usually it's the customer who is screwed by this. This time, it bit the company instead.

That said, you can always ask if someone knows a security contact on BugTraq. Someone there will probably know.

As someone who does community management/marketing, I take claims like security issues very seriously. If someone at 4pm messaged such over Twitter to my startup, I'd call the CEO and all engineers immediately, regardless of the time. I don't think I'm a doorman there, but rather the first line of defense/listening.

The key word in your post is "startup". Most CSRs of AMEX have probably not even seen their CEO in person, let alone "call him immediately".

Their bosses bosses' have never seen their CEOs. I bet you there's a good chance a CSR doesn't even know their CEO's name.

I certainly didn't back when I worked at $IMMENSE_FINANCIAL_INSTITUTIONs

The hacker didn't want to use Twitter either...

And what do you think the doorman/receptionist is for?

> I for one think it's a seriously unrealistic expectation to think that AMEX or insert large corp here will handle security vulnerabilities over twitter.

a) Agree. b) That said, I think the fact that the person on the other end of the American Express Twitter account was accepting to talk to the guy over DM, and thereby actually /was/ willing to handle a security vulnerability over Twitter, is the most damning argument against this guy's rant; he insisted on using a "modern protocol", but apparently telling someone using Twitter, when they were perfectly happy to let him do so, was not modern enough: he insisted it be on his terms or no terms, e-mail or nothing.

i agree with you that preserving anonymity is a valid goal. spending 20-30 minutes on the phone is not how one should run something like a whistle-blower's hotline.

i don't agree with the idea that you are "...being kind and generous by not exploiting...".

I'm curious, if notifying them instead of exploiting the bug doesn't qualify as 'kind', then what do you call it?

As far as im concerned that's being bloody gracious and generous.

yes, notifying them is kind. simply not exploiting them is not.

it's like saying i'm being kind for not robbing someone.

Is more like, I found your wallet here it is and all the money is still there. Perhaps honorable is the right word we are looking for here.

hardly. exploiting the vulnerability is clearly and objectively illegal. It is likely to affect not only the company itself but also any innocent customers one might defraud.

I disagree. If you want to be a black hat and exploit or sell the vulnerability, then fine. But if you're going to claim to be a good guy, you need to make more than a half-hearted effort to do the right thing.

In this case, the exploit is so simple and obvious that he could have fit it in a twitter DM (which is a method of communication that was specifically offered to him)

No. The only way to be a bad guy is to exploit the vulnerability. He didn't do anything wrong, he did something very right that most people couldn't and wouldn't have done, and he was rebuffed for it.

It's not like they're owed this. If not for this good guy wasting his time trying to contact them and publishing this they'd have probably been vulnerable for years.

The person in the thread who made the call could only have done so with the help of the initial disclosure. He couldn't have helped make Amex more secure until the security researcher showed him how.

Now Amex is more secure than yesterday.

Why should he? The company was trying to be hard to reach to control costs. It cost them in another way. Life is tough.

This is financial infrastructure, there's a higher bar. I think the government should mandate that we give the hacker who discovers the hole 2% of the company's profits for the year... If not, nationalize them and rid us all of the useless frictional costs.

Someone signed up for their Wells Fargo account with my email address. For weeks, I tried to get in contact with Wells Fargo about the problem. I spent a lot of time on the phone with the 'security team' and nothing ever came of it. In the end, I kept receiving all of this person's banking information.

It wasn't so much a security problem so much as it was annoying to get this person's banking info all the time. (I assume I would have needed more than access to email to get into his account.)

In the end, after several phone calls and then tweeting at Wells Fargo, it was the Facebook reps that were able to get someone to call me and sort the problem out. The rep who called even verified that the first three of my social didn't match the account holder, so it wasn't identity theft.

Completely annoying but in the end, it just took finding that one person that understood and cared to help.

Ironically the page is available over both https and http.