StockX was hacked, exposing millions of customers’ data
techcrunch.com
techcrunch.com
StockX is valued at $1B and aside from their cataclysmic choice of using MD5 + salt as their way of hashing passwords (They obviously don't take security seriously) the company failed to inform their customers of this security breach as soon as it happened and left it very late for the customers to change their credentials. I would expect any unicorn valued company to have some form of incident-response system to immediately inform staff of the breach and to instantly reset all user credentials and to notify their users.
Instead they didn't inform their customers after the breach and now someone is calculating all those MD5 collisions and attacking all accounts with common passwords.
In the case of StockX handling this security breach, this is un-professionalism at its finest.
We used to joke that working in security was always chasing "zero". If you've done your job perfectly, an attack isn't successful, just like every other day. If they had stored passwords properly and had the various other layers worked out, the attempt on the environment might not even have been seen[0]. It's an easy thing to put off when all you're looking at is today's balance sheet. For months/years[1], the site hummed along without a breach[2].
Now you're the head of security, you make a whole bunch of points about the risks discovered within all of the systems. The person who holds the purse strings reads each risk (in a vacuum, of course) and sees each individual risk as minor, plus that new feature will bring in revenue while spending money on security only causes a fuzzy category of "cost avoidance". Forget about the fact that convincing the low-level non-technical purchasing manager is going to be a whole lot easier than when he/she takes that same justification to a C-Level executive.
When the problem isn't understood, it looks a lot like a choice between spending money to reduce a risk (that the audience is going to under-estimate) versus spending money to give customers a new feature. Do customers want security? Sure, they say they do, but most of your users use the same password everywhere despite years of being told not to. They think that breaches are routine and that if their account is breached that it's probably not going to matter[3]. It all speaks to an expectation of security (a baseline, a 'zero'). It's somewhat ironic that users take security as seriously as typical developers/maintainers.
And then there's the core problem of securing a system. It's a problem who's solution has a variable half-life. To prevent a hack, you have to be right 100% of the time, to be a successful hacker, you have to be right once. There is no limit to the amount of money (good money after bad) that you can spend securing your systems and you can spend all of it on that and still fail. It's been my experience that when defining success is difficult, and the amount of money that is required to be spent to achieve success has a large range, the amount of money spent will be the lowest amount suggested by the first person who can convince management that their solution is "good enough"[4]. It's also easy to look at logs, see a bunch of dropped packets/thwarted attacks and jump to the wrong conclusion that "our defenses are working just fine as they are[5]" rather than "wow, we're under constant attack, unrelenting attack!" Of course, if your physical home was attacked as much as your web application, you're more likely to put up a stone wall/fence rather than be thankful that nobody has figured out the out to breach the 5-tumbler dead bolt that anyone with an internet connection can learn how to breach.
All of that said, it hurts to write this. I'm from the metro Detroit area. StockX recruits like crazy over here and as a result I have a number of friends who work at the company. At least of the people I know, they've got some great developers over there[6] -- we've all had more than a few hundred conversations about best practices around password handling (frequently centered around "don't if you don't have to").
[0] Sure, they could be logging every dropped packet, but even then.
[1] Not sure how long StockX has been around but they employ a lot of my friends.
[2] As far as anyone knows. It sounds like the breach was discovered not by internal monitoring but by the existence of credentials for sale.
[3] I've had my account credentials published several times, I've had my SSN published publicly on a web site (in 1998). I'm actually surprised I have had little in the way of attempts on my credit.
[4] We salted our hash and have appropriate ACLs set up on the database. Our application firewall prevents all but our corporate IP and the web host from attaching to the database. Sure, there's a lot of IP addresses that exit that proxy. We also use MD5, but the salt protects us from rainbow tables and the other protections should add enough layers to the onion. I mean, after all, an attacker will just move on to an easier target when they hit (pick one of the three defenses).
[5] That ranks right up there with "That's what we have business insurance for!"
[6] Can't pick on them too much; all of them are recent hires and would have been unlikely to have the authority to do much about it (or even the knowledge of the code-base required to identify that anything had to be done)
What now? What do MD5 collisions have to do with passwords?
If you know the hash is, either will match the hash.
A few higher level people who were all let go with me ended up going there, and having met up with them a few times, I've heard some absolute horror stories about everything ranging from dev workload, to security, to extremely unqualified devs being hired to fill seats.
I'm not surprised by this in the least, and frankly, I'm surprised it's not worse.
Instead of joining the bashing party and lieu of making a broad statement why don't you detail what some of those red flags were?
"Delete your app and start over. Ruby on Rails is trash. Real programmers use C#. With C# you can create libraries that you can reuse across all your apps."
Also, in general, for programming in the large, many experienced programmers, having worked on multiple large-scale software projects, tend to prefer statically-typed languages. We've found by experience that in such large-scale systems, major refactorings are much, much smoother and feasible in statically-typed languages (although unit and integration tests are certainly still needed). And a whole class of errors are eliminated.
I don't think RoR is trash, but if you were starting a large program that would be used across multiple departments whose in-house language was C#, it was probably the right call to suggest switching to C#.
Please note: I'm not a C# programmer, and have never done any work in C# (although I've worked many dynamically-typed and statically-typed languages, and have a clear preference for the latter for large-scale software projects). So this isn't something I have any personal investment in.
Finally, I do get it, working for these types of companies is misery for most programmers. I've worked in such companies. But in this case, the senior may have had a good point.
[...]
>I don't think RoR is trash, but if you were starting a large program that would be used across multiple departments whose in-house language was C#, it was probably the right call to suggest switching to C#.
According to the parent comment, he was working at a startup that was in the "Quicken led start-up space". My interpretation is that Quicken was acting like an incubator, and he isn't working in quicken, and so the engineering teams are separate. Therefore I don't think organizational inertia applies here.
I’ve also had to explain why logging is a good idea and how to use SSH. What frustrates me isn’t that people don’t know these basics (nobody is born an expert), but that people get hired to do a job for which they lack core competencies. If your job is to fix engines and you don’t know what a spark plug does, you probably shouldn’t be fixing engines. This was at least an issue for me. I know people at Quicken proper who have told me even more ridiculous stories.
It’s a shame. Detroit’s got a lot going for it, but I think most of the tech companies there have some connection to Gilbert and Quicken, and no amount of coneys will get that taste out of my mouth.
Did you end up working at another Fashion-Tech company? I'm curious how much of this is characteristic of Fashion-Tech industry in general.
My advice is to do what you gotta do to pay the bills, but don't delude yourself into thinking that a crazy sauce employer will change its ways because you try extra hard to change them when they resist your attempts to do so.
The serious tone of this article made me double check if this was April 1st when I read this paragraph. The stolen data is shoe sizes? MD5 hashing for passwords isn't ideal, especially combined with email addresses - that could lead to some email accounts being accessed if people use the same password for everything. The article seems to not really give much attention to this though, not clear if the author even realises this is the main problem.
StockX is a platform for trading shoes, among other things.
"The platform works by buyers undercutting each other in a fashion similar to the stock market, eventually causing limited items to lose all value. "
Can someone elucidate how this is like the stock market, because I don't get it.
It's basically eBay, but focused specifically on limited release/high value fashion - sneakers, bags, streetwear, eatches, etc. Since the market rate for these items changes over time, the gimick (hence the name) is to track them like stocks.
Stockx basically facilitates the sale and exchange of items, taking a cut of the sale and verifying the integrity of the items.
[0] https://csrc.nist.gov/projects/hash-functions/nist-policy-on...
This is absolutely atrocious if this is the case. MD5, even with a salt, can be cracked in a matter of seconds even with the most basic hardware. MD5 hasn't been an acceptable password hashing algorithm for at least a decade now, and StockX was created in 2015, long after the creators should have known better (sadly, despite this, an absurdly high number of companies still use MD5 for pass hashing).
These passwords, hashed with MD5, might as well be considered to have been stored in plaintext.
And when you have a unique salt per user, that’s basically game over.
The fact that the article says “believed to be” strongly suggests that things are not as simple as they’re “believed to be”, because if the passwords were easy to crack that’d be trivial to prove.
Usually the attacker will also know the salts in a breach of this type, unless the company did something clever with the salts (doubtful since they used MD5).
We don’t know how the passwords are hashed. All we have is a journo guessing.
Looking at phpass (one of the md5 algorithms), a high-end GPU can do 7M hashes per sec.
The bigger issue is if they were actually using raw MD5 (which is sadly quite common), which is benchmarked at 25 billion (with a B) hashes per sec per GPU.
That's 7.2 billion hashes per second per single Radeon GPU for md5. For 8 character password with numbers and letters that's 8.5 hours max and 4.25h on average.. The numbers only got better since then, I expect it's at least halved for this year's hardware. (edit: 1080ti does 32GHps, so yeah... make that 1h on average https://www.servethehome.com/password-cracking-with-8x-nvidi... )
Assumptions, not specified in the article:
- The salt is stored with the hash (this is a standard approach)
- It's only a trivial implementation with a single round of MD5. If they used multiple rounds or PBKDF1, then the hash choice doesn't matter as much.
5 years ago, a run of the mill gaming PC could crack an MD5 hash in a reasonable amount of time. Worst case, you'd have to let it run overnight.
One of the big issues is collisions. You might not find the original key, but you'd find something that hashed to the equivalent output.
AFAIK the best current preimage attack against MD5 gives you a complexity of 2^123.4. Even if you had every computer in the world working on this you'd never succeed.
>One of the big issues is collisions. You might not find the original key, but you'd find something that hashed to the equivalent output.
This is false. There does not exist a feasible preimage attack against MD5.
I don’t know anything about MD5 or what makes it vulnerable but if StockX has no reason to believe the hacker acquired the key they could certainly make the argument they had no notice obligation.
Reading the statute (and assuming MD5 is as weak as everyone here says) I would say it falls outside the definition of “encrypted” but it really kind of depends on how honest the security engineers were with the lawyers.
I’ve never thought to ask “how encrypted?” when dealing with a breach but I definitely will now.
Somewhere I used to work (now-defunct) stored DES-encrypted passwords with the key one column over in the database.
I don't see where you're coming from at all.
MD5 hashes is not good. But it also isn't catastrophe level security. If you aren't reusing passwords then the hashing choice doesn't matter since the system has already been breached. If you are reusing passwords you don't exactly want to rely on bcrypt hardness to keep you safe.
If I could make the web services I use switch to MD5 hashes and spend more time on other relevant security posture, I'd very seriously consider that.
We know people reuse passwords. This is a non-negotiable threat model for any user-facing system. Given this, one should make password-decryption as hard as possible. MD5 is just not good enough by any standard, in 2019.
A platform like StockX should be a continual breach, because the information will let you make advantageous trades and time series against the customers.
Its pretty dumb to even announce a past tense on this as if it was a single event.
Investors should be weary of people from the fashion industry. I say this as someone who has both a computer science degree and a fashion design degree, and 90% of my friends were in the fashion industry at some point. Coming from tech, you'll find people here are much flakier and just unreliable. In the NYC fashion scene in particular, people have huge egos and they don't always act out of pragmatism or logic. The tendency to keep up appearances manifests itself in many ways. Look at Barney's, it appears great on the outside, but recently considered bankruptcy before receiving a capital injection.
Recently, I went into one of the top streetwear brands in the world, a staff member tried to start a fist fight with me after a piece of paper fell out of a hat, and I refused to pick it up and told them to screw off after the guy tried to disrespect me in front of my girlfriend. I've never been to a retail store where a staff member told a customer "meet me outside p___y", but that is the nature of streetwear culture in NYC. In case you aren't aware, these streetwear stores in NYC have BOUNCERS. Let that soak in. They're just accustomed to bullying customers because people are so desperate to buy clothing that they are willing to put up with the nonsense. They particularly like to single out mainland Chinese who don't realize (or maybe don't care) when these staff are disrespecting them, and, since I'm Asian, the guy who picked the fight thought I would not stand up for myself. If you want more examples of ridiculousness of streetwear, search "ym bape compilation" on YouTube. This dude loves the clothing brand called "Bape" and goes around and assaults everyone he sees wearing the brand Supreme.
There's economic demand and money to be made here, but just know the demographic you're dealing with. The customers and investees are of the same thread. I'm looking to start a fashion tech company myself, and aren't intimidated by potential competitors considering how disjoint these two worlds are, both network-wise and culturally. The typical engineer won't see the value of all this vain-ness, and people in fashion business aren't always the most reasonable people. A UX designer I work with recently told me a story of how they were redesigning a website for a top fashion brand, and the brand requested they make the shopping experience as UN-usable as possible and difficult for people to actually make a purchase (but it works I guess). At any rate Investors, find a leader who can bridge that gap while still being able to attract engineering talent.
Don't get me wrong, you can fund leaders in the FashionTech who are completely unreasonable, and even incompetent, but the company will still do well since product-market fit and demand will outstrip all other factors, until something like this happens. One of my professors owns a set of retail stores in NYC that was acquired by one of the largest clothing manufacturers in the US, but they did not know a single thing about accounting, business operations, engineering, and non-artistic things. Super unreliable, super unprofessional professor but they had a really good intuition for branding.
Luxury brands, corporations (Adidas, Nike), and the LVMH conglomerate are a slightly different story
Did they ask you nicely (initially) to pick it up? What was the tone of your response? These are two missing clues on how this might have gotten started.
Nope. It was "Are you going to buy that?" and other rhetorical heckling questions. I said "yes" and didn't give them the attention they want. Eventually, looks over to my girlfriend, turned back to me, and commands me "go pick that up". If they simply asked "did you drop that?" I would have immediately said "oh, sorry, I didn't see that" and done so.
I think the sort of behavior you describe can happen with any sort of niche or exclusive retail. It's not particularly fair to single out streetwear here, which already has a negative stigma in many circles.
I'd assume the bouncers would be there primarily to prevent shrinkage. Seems reasonable to have people around to ensure running off with hundreds of dollars of clothing is less attractive of a proposition. Sorry you had to deal with that confrontation either way, though.
I'm curious how much of the flakiness attributed to fashion applies to other arts or arts-adjacent areas.