The Strangest FOIA Redactions | Hacker News Reader

The Strangest FOIA Redactions

boingboing.net

56 pointspmoriarty15 comments

I've filed over 1,600 FOI requests for commercial ratepayer data in the UK over the last 4 years, and I've seen some truly pathetic reasons for rejection of my requests. The worst are the ones claiming that such data would cause terrorism.

I filed a FOI request to the UK Ministry of Defence for a classified report in my field. About a month later they gave me a copy. The only redactions were names of various researchers, including the names of the authors of the report that I requested. The odd part was that I already knew the authors' names and gave them in the request in the first place. Must have been some policy that is followed regardless of whether it makes sense.

What if you knew the names of some of them but took a guess at the rest? Including the names would confirm your guess.

Good point, the same way if someone phones you out of the blue and asks you to confirm your name and date of birth (or other personally identifiable information), you don't tell them anything.

I was amazed at how quickly they processed my FOIA requests. Asked customs for my travel so I could match up some dates on my heavily stamped passport. Within a month or so, they sent me a decade of travel all cleanly printed out. I'd already sent out the guestimate which is what inspired the request - but it was nice to sync up what I thought it was to what they thought it was.

Not surprising. Searching travel record is what they do in daily operation (applying visa, border control, etc..).

The person who can make an amazing, open source search engine (like the Google Appliance) that can take in Exchange Servers, File Shares, and any other data source and turn around and be able to do full text search on the data would become very rich- or very well liked.

Some of the examples here show that even in the government, information is not really centrally cataloged or searchable. One hand doesn't know about the other. Maybe some of it is malice, but I have to think part of it is just not having the ability to search Email, File Shares, Sharepoint, et cetera quickly and easily.

Like Relativity, Nuix or any of the existing eDiscovery processing tools?

Elasticsearch?

You need to add data to Elasticsearch for it to be useful. I wrote a search engine for Confluence using Elasticsearch- which is amazing, but you need to add components to get it functional.

I'm thinking a "index this site" or "Index this fileshare" and go.

Any chance of open sourcing it?

Confluence's search is appallingly bad, and tags barely help.

Maybe. I've PoC'ed it, but it's really rough. The biggest issue is UI. The indexer and query engine is primitive, but it works. I've been meaning to re-write it. If I do it after work I can opensource it.

Look for a Show HN Post or I'll fire off an email directly too you in about a week.

Think you can write a blog post on general purpose indexing with Elasticsearch?

(Also, thoughts on Mayan EDMS?)

that is a solved problem. the person who can do “AI” on the results to save all the human postprocessing hours stands to make a fortune.

Neat - one of my requests is listed!

https://www.eff.org/deeplinks/2019/03/foilies-2019#seattle

Reply on news.ycombinator.com