Comment by feelix | Hacker News Reader

Comment by feelix | Hacker News Reader

Parent Full thread

I wrote some really effective data recovery software a decade ago. Before I knew it governments around the world were buying it up in droves. I then got pushed (not by the governments, but by my partner at the company I had) to write a special forensics edition, that does a byte-by-byte scan of the source disk rather than a block-by-block scan, which would extract embedded files out of a hard disk and basically ensure that nothing would be missed.

That code eventually got a big fancy "forensics" UI wrapped around it with a bunch of other functionality such as logging the process, and it got sold for $1k / copy. It would sell upwards of 80 copies at a time in a single batch. It was very profitable (though I personally did not see much of that profit).

I always felt very uneasy about it and I stopped developing that line of software even though at this point I could get ~50% of the profit if I were to continue it. I had no idea what it was actually being used for all of the time and I had no way to find out. I do know however that some of the time it was used for good, to catch people distributing child pornography and so forth, however I don't know what percentage of the time it was used for that kind of thing, and I'm also aware that that is the justification that is used for a lot of surveillance. Philosophically I believe in counter-survelliance more than surveillance, because I'm pro privacy and pro citizen empowerment rather than the other way around, I think the balance of power has gotten out of whack as this article nicely illustrates.

If I got it correctly, unlike the software in the article - yours wasn't a malware that stealthily does something unbeknownst to the legitimate user, or something that even impacts the security measures user has set up. It was a genuine and honest data analysis tool, scanning the drive one already has full access to.

Tools like yours (cracked/warez - even a $100 bar was impossible for a schoolkid in late '90s/early '00s Russia, not to say about $1K+ price points) and articles on filesystem internals had helped me to recover my own data from drive failures a couple of times. So, thank you.

And I really believe that would've anyone taken a possession of my notes and inspected them with a magnifying glass - the problem would be anything (me leaving the notes, someone taking the notes, ...) but that someone made the glass.

In my view, much grief can be avoided if people that have a mind able to build things that can have an impact for better or for worse, will use that mind to also control how those things will be used.

When engineering and philosophy are divorced, bad things happen.

Unfortunatelly that may not even be enough. Intelligence doesn't have morals, and when it is employed with a predatory mindset, there may not be escape for the "prey" without a fight.. The arms race may be a mechanism of evolotion. You either accept it, or dissapear..

Just out of curiosity, if you don't mind, what is it called?

This is the data recovery app consumer version that kicked it all off (it has the same engine as the original one): http://macdaddy.io/mac-data-recovery/

You wont find any data recovery software that does a more thorough scan than that, even though it was written in 2004 or so and has been barely modified since. I spent years writing it, in C, so it ended up highly optimized. In forensics mode it scans a drive byte-by-byte, and it assumes each byte is the start of a file of the 100 or so filetypes that it supports, and it maintains that assumption until it is proven to be false as it goes deeper into that file's data. That way it really shouldn't miss anything. It also transparently UUDecodes, unzips, and otherwise decodes data on the fly as to extract file contents under as many conditions as possible. It uses linked lists to maintain its memory of what files are where until it dumps out its buffer as output files, and it uses binary searches and threads to speed up. All in all it's about 1000 lines of code, though I really don't think that anyone would ever be able to read it except for me.

I recently noticed that if you run it over your iTunes Music folder it'll extract and output the media that is contained inside your M4P ("MPEG 4 protected audio") files, and output it as unprotected MP3's. So it effectively strips all of the DRM off of all your media in seconds. Even if you didn't purchase that media (say your friend sent it to you) and couldn't play it before. That was not intentional. You can see that effect in this app: https://itunes.apple.com/app/file-extractor/id1129674765 which is the same codebase, but it is being applied to used space (files) instead of free space.

Point is it doesn't matter, as a creator, maker, investor, etc. -- you always have a choice, and for some the temptation to follow the money even if the path is one they would not take if given another choice of equal financial value.

My money is on test????/photo???.

Not that I care for the karma but why is it being downvoted ?

As the sister comment says, it's because your comment doesn't make sense. What does "test????/photo????" mean?

It's an obfuscated reference to a popular open source data recovery toolset. I think the parent was trying to avoid potentially "outing" GP against their wishes, by making a vague reference that some might get.

Well, it's a guessing game to know what company the parent parent is referring to and there are two well known recovery software but I didn't want to give out the name so others could keep on guessing. So I just wildcard masked the thing.

I reckon people find it uninformative. If not, then I'd guess the exact opposite. You should know which.

    testdisk/photorec

I think because the comment is confusing. Are the question marks a text encoding error?

With your skills, you could help with the balance. Out here and with your karma account ;)

I've read a story similar to yours on HN where someone explained that they made passive income off a data recovery tool. They also said that it wouldn't work on SSDs, so it was becoming less relevant with time.

Do the forensic techniques you implemented work on SSDs? Also, were you the person who posted that earlier?

Yes, that's me. Hacker News is the only place I feel I can talk about this kind of thing so it's cathartic. And no, it doesnt work on SSD's. They're black boxes. However I think data recovery on them is possible with hardware. It's the controller that returns 0's after it has been sent the command to TRIM some free space.

You shouldn't feel bad. If something like that is technically possible, and there's a demand, someone will write that code.

The only two ways to fight it (that I can think of) are:

1) Make it technically impossible (open firmware, open hardware)

2) Make it illegal

There's of course a third way - not give a crap, and just let them watch you.

> The only two ways to fight it

Actually, I think both technological and legal measures are always necessary. One just doesn't work well without the other.

(Personally, I'm leaning towards "but technical measures first", though - as tech is much more agile than legal stuff.)

> If something like that is technically possible, and there's a demand, someone will write that code.

That isn't necessarily true, though it's an easy fig leaf to hide behind when you're doing something that is unique and that helps the dark side.

I've heard it said by people that were actively developing malware and spam software.

Reply on news.ycombinator.com