DARPA: reconstruct shredded documents, win $50,000 | Hacker News Reader

DARPA: reconstruct shredded documents, win $50,000

shredderchallenge.com

103 pointsmef49 comments

Yes, this is a solved problem. That isn't the point of the competition. From the site:

"Today’s troops often confiscate the remnants of destroyed documents in war zones, but reconstructing them is a daunting task. DARPA’s Shredder Challenge calls upon computer scientists, puzzle enthusiasts and anyone else who likes solving complex problems to compete for up to $50,000 by piecing together a series of shredded documents. The goal is to identify and assess potential capabilities that could be used by our warfighters operating in war zones, but might also create vulnerabilities to sensitive information that is protected through our own shredding practices throughout the U.S. national security community.

Presently, a variety of techniques exist for reconstructing shredded documents including manual assembly, fully automated (computerized) algorithms and hybrid operator-assisted approaches. DARPA hopes to gain new insight into which of these or other innovative techniques are quicker and more efficient, and, whether the wide availability of high resolution photography, communication and crowd-sourcing strategies offer unexpected advances.

“The ability to reconstruct shredded documents will potentially yield information that may save lives or offer critical information about an adversary’s plans,” said Mr. Dan Kaufman, Director of DARPA’s Information Innovation Office. “Currently, this process is much too slow and too labor-intensive, particularly if the documents are handwritten. We are looking to the Shredder Challenge to generate some leap-ahead thinking in this area.”

The Shredder Challenge is composed of five separate problems in which the number of documents, subject matter and the method of shredding is varied to present challenges of increasing difficulty. To complete each problem, participants are called upon to provide answers to puzzles embedded in the content of reconstructed documents. The overall prize awarded depends on the number and difficulty of problems solved.

Registration is open to all eligible parties at www.shredderchallenge.com, which provides detailed rules and images of the shredded documents for the five problems. The twitter hashtag for this event is #shredderchallenge. Participants are encouraged to build teams using the event forum and to monitor www.twitter.com/darpa_news for the latest news updates."

Alternative hypothesis: the point of the competition is to make people think that reconstructing shredded documents is harder than it is.

Why? So that the enemy will keep on shredding documents instead of burning 'em.

If that's the case, it reminds of Enigma, and how the allies were sending spy teams trying to steal an enigma, while they already had one.... Hmmm....

Incidentally, when did soldiers become 'warfighters'?

Soldiers are a subset of warfighters.

For $50,000? Is that all this is worth to them?

I would say it is certainly technically possible with modern tech, but seriously...$50k?

Change the prize to a million dollars and you'd get a solution overnight.

They probably don't want an expensive solution that takes ages to develop and implement. What they're looking for is a cheap, easy-to-deploy-in-the-field solution. The prize attracts a different type of applicant with different methods and tools.

Edit: To clarify: A prize of $1 million may result in everyone spending $100,000 to find the solution quickly, when in fact DARPA only wants solutions that can be developed at a price well under $50,000.

I disagree. If they want cheap and easy to deploy, add it to the spec. As it is, a prize is offered to get the attention of talent. $50K won't get the attention of anyone even remotely talented enough to produce "best of show" solutions. I suggest $1million will get their attention.

It's more about generating publicity for DARPA than an actual solution. Just look at the website design and the language. I mean seriously, a forum?

Most public DARPA projects are awarded via grant application from serious labs, and the funding often goes out before you reach a solution, since they fund high-risk projects with a low enough likelihood of a solution that profit-seeking companies won't want to try it otherwise.

>I mean seriously, a forum?

it sounds like you somehow missed the Grand Challenge trilogy.

http://en.wikipedia.org/wiki/DARPA_Grand_Challenge

http://www.darpa.mil/Our_Work/DSO/Programs/23_Mathematical_C...

http://en.wikipedia.org/wiki/DARPA_Network_Challenge

they don't want a million dollar solution. Among other goals, they want to know what a solution can a 3rd party get at this market and technology state on a budget of $50K scale. Ie. how sound the DoD shredding practices is.

The finished product will likely be something that can automate photo taking and a laptop.

The amount of money DARPA pays to get the know-how has no connection at all to the price of the finished product.

germans already solved this to recover shredded stasi documents. take german solution, profit.

http://www.iiconservation.org/news/?p=55

http://www.schneier.com/blog/archives/2008/01/unshredding_1....

wow, fascinating stuff. thanks for posting

I'm probably in the upper range in terms of regular-folk-paranoia, but even after shredding my documents, I usually mix it in with food and regular garbage so that the information is destroyed. I also don't just shred important documents, I shred everything so that it's even harder to discern what is important and what isn't.

The irony, of course, is that despite my paranoia, my identity has been stolen because of a screw up with the credit agencies and I've been dealing with all 3 credit agencies for months trying to resolve this.

Today is a fail. First I got invited for a lunch at HUD HQ in DC for a meet and greet but instead of lunch they had cookies and water and now DARPA only offering $50,000 for document reconstruction?

I guess govt cuts really are strict.

Why don't "challenges" such as this get the same hate as "$300 for the best logo" contests?

It's purely spec work, and in this case requires much more time and effort in comparison the the shallow work done on 99designs or elance.

You can keep the work though from what I can tell - if you have a shredded documet reconstruction system then you can simply scan five docs and get $50k ... they don't keep the thing that's valuable, the system.

In a logo comp the only thing created of value is the output itself. The equivalent would be if the logo competition was to create a machine that created logos on demand and then only give you a small price for that machine.

I am not sure the window here is long enough to develop a method of reconstructing these documents that will scale to the what the DOD would need. Honestly this feels kind of like the old myth about carrots making your sight better(a ruse to keep RADAR a secret from the Germans). I am virtually certain the gov't already has this technology perfected, but this is a way to make people think we don't.

I find it mildly disconcerting that DARPA's server infrastructure is clearly not scaling well for this announcement.

My intuition suggests that all of these documents being on lined paper would make the problem quite a bit easier, by providing clear features that have to line up between fragments.

At any rate, it makes you wonder if most documents are on lined paper, or if it's more common to see things on, for example, white printer paper.

Or if paper in general has some sort of "grain" created during its manufacturing process?

While a neat puzzle doesn't anyone else think that the NSA was able to do this decades ago?

Of course, just like their SATAN project catalogs almost all US internet traffic.

Decades ago shredders weren't nearly as good as they are now either. It isn't a static problem that will remain solved.

by making all shredders to contain secret mini scanner

Do you have evidence that this has been done? I am reminded of the secret identifying marks that printers leave on their documents.

https://www.eff.org/wp/investigating-machine-identification-...

no it was a joke inspired by very popular Russian ATM crime :

http://www.upi.com/Odd_News/2011/10/07/Illegal-scanner-found...

Another - Russian security agencies use Western hardware ( i mean what other option do they have seriously ?). Before being put to use the hardware is disassembled as much as it is possible with specific goal of finding the "mini-scanners". Similar history - http://www.spybusters.com/Great_Seal_Bug.html

http://editinternational.com/read.php?id=47ddf19823b89

"Roy Zoppoth stands over a Xerox 914 copy machine, the world's first, which was used in soviet embassies all over the world. The machine was so complex that the CIA used a tiny camera designed by Zoppoth to capture documents copied on the machine by the soviets and retrieved them using a "Xerox repairman" right under the eyes of soviet security."

I found this paragraph even more interesting:

“From the numbers of cameras ordered, we later realized the CIA had placed secret cameras in every Xerox copy machine in all embassies in the world – friend and foe alike.”

Entries may only be submitted by a registered participant, who must be a permanent resident or citizen of the United States

Shame - I don't know why they would not want entries from people outside the US.

You don't know why our war department would not want to pay foreign nationals for code that they intend to use in a battle situation? Is it honestly that hard to figure out why that restriction might be in place?

DARPA has a ton of research done by foreign nationals. (I have worked in labs which have foreign nationals who get paid to write code for DARPA projects).

They will audit the code, no matter the nationality of its author. There are plenty of US residents who would sabotage the code if they could.

Would they rather want them to give the code to, say, the Syrian government?

It is not like they can't take the idea expressed in the code and pay somebody with the appropriate code to fix it.

And it terms of military expenditure $50k is a joke.

Netflix offers $1MM to improve their algorithm by 10% and $50,000 is all that DARPA can offer for a harder problem?

It doesn't appear you even need to provide an algorithm. You can solve it manually and still win the money.

Netflix is a much much harder problem.

If the shredded documents are of the same type as those in the picture, this isn't much of a challenge.

I was hoping to see some sort of ridiculous challenge to piece together documents that were destroyed by a high end unit that was specifically designed for sensitive documents. They basically spit out sawdust.

If this is for the "warfighter" is shredded document reconstruction really a commonly encountered problem in the US's current wars in Afghanistan and Iraq? Its seems that in Afghanistan, for example, they don't even have electricity in the majority of the country let alone paper shredders.

Seriously DARPA? 50K? That's no Netflix prize.

Your password policy (and error reporting) for registering is so stupid I don't want to work with you guys anymore, ok? Don't get in touch.

Personally, I shred documents to make sure they're destroyed. Why would I ever want to help DARPA come up with a way to reconstruct them?

to advance the state of technology in the document shredding industry, of course.

Because you can't read the first sentence of the damn article?

I find the first sentence to be inadequately motivating.

Besides, if DARPA does invent this who's to say they won't turn it on the American populace? And who's to say that folks won't go back to burning their documents?

I think that's what the $50,000 is for:)

You know will go "missing" once you have put it back together and seen the document.

Reply on news.ycombinator.com