Scaleway: Incident SSD was stolen during a secure transport between datacenters | Hacker News Reader

Scaleway: Incident SSD was stolen during a secure transport between datacenters

blog.scaleway.com

99 pointsmot2ba27 comments

Scaleway's claim that customers were immediately informed («prévenu la clientèle potentiellement impactée») appears to be false. Affected customers were not notified until June 2021. [1]

Scaleway did not publish their blog post until after the 3 part video series by Micode, despite being aware of the incident since May 2021. [2]

[1] https://www.lowendtalk.com/discussion/172819/scaleway-ssd-wi...

[2] https://twitter.com/Micode/status/1395640486715662336

I’m having a little trouble following since I don’t speak French, but based on other comments here, it sounds like:

1. French YouTuber Micode bought a used SSD from leboncoin.

2. Micode wanted to demonstrate that data should always be properly wiped from used drives.

3. The SSD Micode obtained had been quick-formatted and was never encrypted, so it was trivial to recover the data on it.

4. Micode asked his followers on Twitter to try to identify the source of the drive.

5. It was eventually identified as belonging to Scaleway, and it contained important data from a Scaleway customer’s VM, including an SSH key, source code, and an S3 secret.

6. Scaleway threatened Micode, who now claims to have wiped the data.

7. Scaleway published a blog post claiming the drive was stolen while being transported between datacenters.

Unless I’m missing something that was lost in translation, I call bullshit on #7. They also seem to be claiming customers were notified immediately, but it that doesn’t appear to be the case. This just seems like they sold an old drive that should’ve been encrypted (it wasn’t) and wiped (it wasn’t). Whether that sale was authorized is a matter of debate—one former employee said they wouldn’t be surprised if someone just decided to walk out of the building with decommissioned hardware.

> This just seems like they sold an old drive that should’ve been encrypted (it wasn’t) and wiped (it wasn’t

I find that unlikely ( unless as you said it was a rogue employee) - they're certified HDS ( hosting healthcare data) so i find it improbable they aren't supposed to have a special disk decommissioning process to follow, including secure wiping.

Yeah, our procedure for decommissioned SSD and HDD (imposed to us by HDS requirement) where 3-pass shred on the disk (logical wiping), and either pierce the HDD or microwave the SSD, with photographic proof send to our client if asked.

The 3-pass shred was done on the virtual disk each time a client left, and was done on the physical during maintenance.

[edit] And concerning encryption, it was at a premium, and i think only one of our client asked for it (this had a few issue). "hot" logs where not encrypted, but during the logrotation we tried to implement this (this was not technically audited btw, i think it was OK for Deloitte not to see our code)

> one former employee said they wouldn’t be surprised if someone just decided to walk out of the building with decommissioned hardware.

Who here hasn't taken decommissioned or unneeded work computer crap home?

At work I recently replaced four 2.5" WD black 500GB spinning rust disks in Dell mini computers with m.2 SSDs. Since we are moving all of our systems to SSDs the WDs were trash but perfectly working with only a few months of time on them. I took the disks home and used them to play with FreeBSD and NetBSD on my Lenovo laptop. I didn't wipe them before taking them home either...

They collaborated so much with Micode that they threatened him when he disclosed the SSD was from a cloud provider (without giving away its name)

As for the other things, at the end of the 2nd video he did succeed on extracting and gaining access to the data, with full code source, AWS and Facebook (bot account) credentials (among others).

The exploration of the (redacted) data is in part 3

Google Translate link into English: https://translate.google.com/translate?sl=auto&tl=en&u=https...

Edit: seems like the blog itself can be read in English with the button at the bottom, yet the articles themselves aren't necessarily translated after clicking on it. Localization is hard.

And why exactly is the bare storage of these not encrypted? I would expect, at the very least, that the data is encrypted using customer-specific keys.

The article doesn't say whether the disk was encrypted or not. Where did you find this info?

> The article doesn't say whether the disk was encrypted or not.

So we can conclude that it was not. If the disk was encrypted (with decent passphrase/key) there is 0% chance that the article wouldn't mention it.

They also wouldn't have bothered to get the disk back from the YouTuber. Why would they if the data is unreadable anyway?

If it was encrypted, then the news wouldn't be about someone recovering a quick formatted partition. It would be about someone breaking encryption.

They ublish all their blog entries in English, but this one is in French. Why? I would understand if the information were addressed to French stakeholders, but this information is most relevant to (perhaps international, perhaps prospective) customers.

File Carving: https://resources.infosecinstitute.com/topic/file-carving/

So, did the YouTuber extract any data from that drive or not? If not, I suspect there wouldn't be a headline? + The text starts with: "Over a year ago, an SSD was stolen...".

I may be wrong of course. But would gladly know more info, if anyone has it.

You can find the first of 3 videos from this youtuber about this subject here: https://youtu.be/vt8PyQ2PGxI

It's in French though

Yep, the French vlogger Micode [0] asked their Twitter followers to identify the data on an SSD they purchased on the local classified ads website leboncoin.

[0]: https://web.archive.org/web/20210531091659/https://twitter.c...

[1]: https://www.youtube.com/watch?v=vt8PyQ2PGxI

[2]: https://www.youtube.com/watch?v=aOBVZUL1iBA

[3]: https://www.youtube.com/watch?v=xf_cKTlOYLo

For those who don't speak french, this Youtuber bought a second-handed SSD to do some forensics and raise awareness. He managed to recover the partition, inside he found qcow2 image of a company, he managed to mount it and recover the full data, including source code of their app, a ssh key and S3 credentials of the company

Thanks. Is there any info whether drive was formatted or not?

The drive had a quick format, so he managed to recover the partition with testdisk

These days every single drive should have full disk encryption. And then you can format by simply securely wiping the key part.

SSDs work in an odd way: they have more capacity than they claim, and when you overwrite some data you're really just writing to some of that spare space while the old version is just marked as free [1, 2]. Admittedly, at that point it's going to be very hard to retrieve the old data (you'd need to bypass the remapping logic presented by the controller circuitry), but it's theoretically possible. If you're paranoid, it's best to rewrite the whole thing even if it's encrypted, ideally several times (at that point a little original data could technically be left over but it's unlikely and it's not going to be feasible to get any of it if it was encrypted).

[1] https://databasearchitects.blogspot.com/2021/06/what-every-p...

[2] https://news.ycombinator.com/item?id=27572218

If you only ever write encrypted data to the disk you don‘t care about spare ssd cells. The data was always encrypted you wouldn‘t even need to format before giving away.

In general I'd agree, but the parent comment mentioned "wiping the key part" so I guess they had the assumption that the password is weak enough that its vulnerable while that data is accessible, so I was assuming the same thing. (As you're no doubt aware, the encryption key for full-disk encrypted disks are encrypted by the decryption password and stored on the disk itself, at least when not using a TPM.)

Yes a running VM from an a corporate customer containing all source code, SSH keys, access keys to github repos, AWS keys and much more.

Now this has utterly disappeared from the front page. wtf happened ?

Sounds like someone forgot to run 'dd if=/dev/zero of=/dev/sdX'

This raise a question, why weren't they using encryption on their qcow2 volumes ?

Reply on news.ycombinator.com