Comment by bpchaps | Hacker News Reader

Comment by bpchaps | Hacker News Reader

This stuff is becoming more and more maddening to hear after my recent FOIA adventures.

I requested this from Chicago's Mayor's office last week:

Please provide to me all communications or records of communication (phone, email, receipts of phone/email, etc) from those in the mayor's office for the below companies. Please limit the search to Oct, Nov, Dec of 2014.

Aura - Prestige Club (of verycoolrooms.com)

Kennealy & O Callaghan

Statewide Investigative Services

Azura Investigations

Their response, which reeks of complete bullshit:

In order to run an email search, the Mayor’s Office needs the names or email accounts that you wish searched. The email system’s tool set cannot identify the department where an email user works, and therefore, a search cannot be based on a department. Parameters that would assist the Mayor’s Office in conducting an email search include: (1) the e-mail address of the account you wish searched; and (2) the e-mail address of each individual’s mailbox, if you seek e-mail correspondence to and from two individuals.

Without email addresses, search terms, or a time frame, the Mayor’s Office would have to retrieve each and every email for every employee from October – December, 2014, which is both unduly burdensome and costly. After retrieving these e-mails, each and every e-mail message a Mayor’s Office employee received or sent would have to be downloaded and reviewed to determine if it is responsive to your request or is exempt from disclosure. If any exemptions were to apply, the Mayor’s Office would need to redact such information. The entire process of retrieval of the emails, finding responsive records, reviewing for exempt information, and then redacting the materials would be onerous.

How do you propose the mayor's office identify senders and recipients from e.g. that law firm without combing through every message? What if their email domain has changed? What if someone used their personal email address? If they left out any records, you could sue them.

The remaining reasons, mainly an inability to filter emails by department, sound normal. If you're designing a database, can you build an efficient index for every possible query? No. The mayor's office has not optimized their records for random people making FOIA requests. Nor should they. I'd rather my tax dollars go to potholes, rather than axe-grinders.

You'll have better luck with more legwork on your end.

I'm not asking for a complicated query system, really. An upgraded version of outlook would probably do the job. Or an email to their IT department who could do it. That's what I ended up doing last time which ended up turning into a year and a half battle to prove that the numbers were releasable under FOIA.

The judge ended up telling them to JFGI [0], since not googling it was costing the city thousands in court fees. That should address your concerns. It probably didn't get me every releasable number, but I'm sure as hell not going to sue and waste another six months. It takes too long to sue, they know it, and it's a huge factor in why this is all so difficult to do.

Please realize that paving the way for others to obtain this data is a huge part of the end goal. In some ways, I'm trying to fix the potholes of FOIA.

[0] "Next, DoIT and its counsel conducted online research on each remaining responsive phone number — in other words,they "googled" each number to determine whether that number was publically listed, and, if so, to whom it belonged."

Edit: Cleaned up (a couple times).

NB: I'm not expecting they'd actually do the below. The political downsides of being efficient in your FOI requests are too great to seriously expect somewhere like the Mayor's office would dedicate effort in to achieving it. But document/record management is not some mystical never before done thing.

Somewhere in the 9.5 million dollar budget of the Mayor of Chicago [1] they could squeeze out $20k for a HP Records Manager setup (or any of the dozen competing software packages) and just blindly catalogue all emails in to it. Even without setting up ANY of the business classification features it makes this kind of search trivial.

Perhaps it could come out of the 1.4 million dollars allocated to the "Innovation delivery team" who "Creates and implements new solutions to improve the efficiency of City government.".

[1] Page 59 Program and Budget Summaries by Department Finance and Administration Program Summary OFFICE OF THE MAYOR (http://www.cityofchicago.org/content/dam/city/depts/obm/supp...)

Obscurity by bureaucracy at its finest. We all do this to some extent in our day to day jobs. Remember that behind these emails are real human beings.

Keep pressuring, keep pursuing, for you, for us. Observe, pressure, and report. Rinse, repeat. There are plenty of us that appreciate your hard work in holding the system accountable. Keep up the good fight ++1

Totally agreed on the points in your first paragraph. I try my best to help them out and stay as friendly as possible. For example, the Department of Finance senior FOIA officer and I are on a "hey man" basis now. :)

Thanks! This all started in November of 2014 and I doubt there'll be an end for a while.

Not to derail too much, but this sort of work is one of the main reasons why I'm in favor of a basic income. It sucks that it's impossible to live freely while doing this stuff, without being a journalist (bad at writing, so my reliance is on other journalists) or without having a stable job. Not having either right now makes me think to go to patreon for "basic income", but that just feels dirty.

A comment from me from below:

"What is the new becomes the old in no time at all. Something will, and realistically has to, fill the void of high level domestic security. And unless the people that make up it are fundamentally different, or unless the structure itself and its system of rewards and punishments are different, we will only repeat history.

Laws and education are our only hope. Right now these two things are time consuming and are full of hardship so much so that the average person has a hard time taking on responsibility in these domains in their daily lives. If we could lesson these burdens for the average person in these areas then I think, I know, that we are really on to something."

If we could limit the burden for people like you then it would be a huge net positive. If we all figure out that basic income is our best bet then I am all for it. I remain skeptical though and think that there are better options in the medium and long term.

You should have given them an explicit list of domains to query for in the headers.

Based on history from previous and successful requests, I shouldn't have needed to. They've fulfilled requests as vague as that in the past.

Submitted two requests two days ago that might be a bit better than providing the domains, though. One for DNS resolution logs of the mayor's office and another for all domains and times for all emails sent from their office. If just one of those are successful, then I don't need to play any more FOIA games with mayor's office.

Sure, I could just submit the same request to their IT department like those two... but, no. I'm getting that information from the mayor's office directly, dammit. :)

DNS resolution logs...

Brilliant. I love you.

Thanks. :) & <3

Do a FOIA request for the staff directory?

That's actually all available at data.cityofchicago.org. My first thought was along that line. Turned out to be about 60 different people, so it was still a bit much.

Reply on news.ycombinator.com