Justice department 'uses aged computer system to frustrate Foia requests' | Hacker News Reader

Justice department 'uses aged computer system to frustrate Foia requests'

theguardian.com

161 pointsrmason50 comments

The unfortunate fact is that government officials, whether at the DOJ, the FBI, or any other agency have absolutely no incentive to follow either the letter or the spirit of the law. There are never any consequences for the individual government employees who break the law, so there is no reason for them to follow it. What's worse is that the only government officials who are ever held accountable for violations of law or policy are whistleblowers who shine light on wrongdoing.

When the law ceases to be a social contract under which we all agree to operate and be equally judged it loses its moral authority. Without moral authority law carries no more weight then the threats of violence wielded by those with the most power and society falls apart.

> When the law ceases to be a social contract

Law has never and will likely never be a social contract. It is always enforced with violence, which encourages compliance. Give up Hobbes' misguided concept already as it will never help you make predictions about or explanations for how people behave.

I've been on the receiving end of some of this at some point in my life and I'm happy to report that suing the government (to be specific, the Queen) in NL actually works and you can win.

It will cost you some money and some time but given a good lawyer it is definitely possible to hold government officials accountable.

I'm looking for a new FOIA lawyer - would you be willing to share the details for yours? (I'm in IL)

I'm in NL, so I don't see what good that would do.

Oh, derp. I'm stuck in travel exhaustion and read that wrong. :)

You should direct this question to the good people on the FOI-L listserve: https://listserv.syr.edu/scripts/wa.exe?A0=FOI-L

That being said, if anyone wants a New York FOIL attorney, particularly if you're dealing with a stubborn NYC bureaucracy like the NYPD, I can recommend the firm Rankin & Taylor: http://drmtlaw.com

False. Please stop spreading falsehoods without doing some basic research. It took me < 10 seconds of Googling to find many counterexamples to your claim.

https://en.wikipedia.org/wiki/List_of_American_state_and_loc...

https://en.wikipedia.org/wiki/List_of_American_federal_polit...

Heh. A few examples means jack. I can give you tons of personal examples where it's true. It's mostly just a matter of how patient you are in filing a suit or pursuing with an AG. That takes months, and more often than not it's not worth it. It's incredibly frustrating and extremely sad on so many levels.

I seriously doubt you've submitted a FOIA request or else you wouldn't have responded that way. Go submit one and you'll see.

Edit: The linked article is actually similar to a lawsuit I would like to raise. A lack of technical knowledge can stop a technical request in its tracks without resorting to escalating the issue. Calling, emailing, leaving voicemail very often leads nowhere in just explaining what a domain name and becomes very mind numbing when that obstinate ignorance leads to a rejection.

Find a single one of them who was convicted of breaking the applicable public records law. These laws can be a powerful tool for government transparency, but only when they are actually enforced.

The parent wasn't making claims about the public records law in particular. He said "There are never any consequences for the individual government employees who break the law".

Ok, but I think you are being kind of picky. We are having a discussion about an article which alleges willful obstruction of FOIA requests by government employees. I think it's clear in this context the comment is not talking about murder, securities fraud, or what have you.

I think in the context it was used, "the law" is referring to the FOIA rather than being used as the general term for all laws. (Not taking a stand on whether the statement is correct or not though, as I don't have any specific knowledge about that)

Logical fallacy. Just because there are counterexamples doesn't mean a claim is false. If I say "humans eat meat", the claim is not false because I found a vegetarian.

Err, you're the one making the logical fallacy. The parent said "There are never any consequences for the individual government employees who break the law" and the counterexamples disprove that. "X never happens" is false if I find examples of X happening, by definition of "never". QED.

That's just word play and purposefully misunderstanding the parent's likely meaning. It would make no sense for the meaning to be exacting like a mathematical proposition, in that there are literally NEVER any consequences. When speaking of an entity as large as heterogeneous as the government, you can't make such specific claims, so the obvious meaning was "for the most part" or "in general".

Yes, it actually does mean the claim is false.

In ordinary English discourse "humans eat meat" is an existential proposition, with some possible claim of frequency, commonality, or representativeness. It's not a universal proposition; that's conveyed by "all humans eat meat" or "humans always eat meat" or some such.

Do you really think the OP's claim was universal, when talking about something non-monolithic like the government? The implied meaning is one of frequency, not absolutes.

The OP wrote "never", which is an absolute, and only requires one counterexample to be proven false.

What was implied is just you making sh*t up.

Informal natural language is in most cases not literal. For example, I assume you don't mean that I'm actually creating feces.

Oh man, is this relevant to my interests. An organization I founded is currently in a state-level Freedom of Information law fight -- using the Missouri Sunshine Law -- against the Missouri Department of Health and Senior Services, asking for two database dumps as CSV files. The databases have millions of records covering decades of data, but only about four columns, so this should take a couple of hours to export, tops. Of course, we're willing to pay for the USB flash drive to put the files on, and cover any unusual labor costs to do the export.

The good news is that the Department has agreed that we should have the legal right to this data.

The bad news is that the Missouri Dept of Health is claiming with a straight face that they can only retrieve one day's worth of data at a time from their mainframe, and must re-run the work job for each and every subsequent day's data. They claim that it is impossible for their software to select for a date range.

And therefore, they have quoted my organization a price of about one million dollars and 2.6 years to select and deliver our data.

Shot: https://mobile.twitter.com/ReclaimTheRecs/status/74644736092...

Chaser: https://mobile.twitter.com/ReclaimTheRecs/status/75116107051...

(Don't worry, we have a good Sunshine Law lawyer and he is as outraged/amused at this bullshit as we are.)

Have them offline and copy the raw drive/tape to trusted 3rd party (as they must do regularly for backups anwyway) and have that 3rd party query the data. Or if the data is old enough, have them just copy the backup to the 3rd party.

This should eliminate any query expense they would have had (as it's been replaced by raw data copy expense.)

And the 3rd party is to protect the data you're not supposed to get.

I would spend the money to convince or "partner" with the paper of record in the region to run a story on how out of touch they are with technology.

Unfortunately most journalists won't do it for fear of losing their "access".

Properly embarrass the right politician near an election and you get results. Apart from changing the leadership in an actual election that's about your only hope.

To be fair, the government's computer systems really are just that crappy. Technical debt and obsolete systems that still kind-of-sorta work are always the last things to be addressed in budgets.

There is also no incentive for the government to do anything other than 'try' in the most perfunctory manner possible. One solution would be to make a separate entity responsible whose performance is judged based on what they can actually dig up (of course someone has to get the funds and resources for these people to work).

There is no incentive to build systems that are optimized for FOIA requests (ie enough hardware to search the data, or data indexed in ways that make it easily possible). I doubt servicing FOIA requests is super high up on the spec list when a system is designed.

There are also legitimate issues concerning cost recovery: think of it in terms of paying for someone to spin up new instances to hit multi-TB databases for a couple of days, then paying someone else to see if the results are responsive, if they're actually releasable in terms of security, privacy, etc. And then you have to deal with the less legitimate requests that come along. I think serving FOIA requests is a core part of the government (both ideally and by law) and resources should be spent accordingly.

Finally, there are government officials, especially at the state and local level, who use the above excuses to make it hard for people to find out what they're up to. The worst effect of the issues listed above is providing these people with cover and plausible excuses which are used to stonewall legitimate investigation by the people they serve.

This is, I think, the best way to frame it: the systems are actually that bad, and bad actors can misuse the bad systems.

That said, I'd be very curious to hear about people in govtech about how this plays out in your municipality.

I like how the replacement system costed $425,000,000

The tech company told them on site deploys would be $424,550,000 more expensive

Yeah I find it hilarious as a contract developer how just because it's government software that means you can charge like 200 times more for essentially the same software any other client could get for like 1 million at most.

I've been doing contract development for a little over 10 years now and even the most complex projects were never even remotely that expensive. And Im sure I've probably written at least 85% of the same functionality of whatever that software does. Hell, I'm sure there is probably open source software that does similar

So glad I pay taxes

Prices of that nature usually include 10-20 year service contracts and onsite maintainers, to include hardware. Not that the prices are not still inflated, but the reality of running a contractually 99.99% uptime, multi decade, and completely backwards compatible platform is not trivial, or anything south of multi millions of dollars at the federal scale especially.

At Slashdot someone pointed out that the student requested a search that should've pulled up his own earlier FOIA requests -- and didn't -- thus proving their system was inadequate.

https://yro.slashdot.org/story/16/07/17/0224251/is-the-doj-u...

This was also mentioned in the linked article.

They're using "aged people" to frustrate as well. If there was some more meritocracy, or better pay, we might be able to get some new, technically useful, blood in there. Personal accountability wouldn't hurt either. It works for more than business and families.

Don't blame the people, aged or not. This is being driven from the top down.

So they're just following orders?

You are implying that meritocracy or better pay would go towards better results for the FOIA requester.

An equally likely interpretation is that the meritocracy is delivering the desired results from the government's perspective of interests.

I'd say that most energetic people with skills would never waste their time with such nonsense. They can make real money in the private sector.

Not everyone is solely motivated by money.

The rest of us like having a roof over our heads, food in our bellies, and our student loan payments made on time.

It is possible to have both.

It is, but spending your energy on researching and implementing the most effective way to annoy and obstruct people that are trying to exercise their rights?

Generally people that could work wherever they want and choose to not chase after money will probably go for something that has a high moral value (or is a noble cause) to them, like helping the poor. But not something like this.

I'd imagine that the only thing they might get out if this is political power which should translate into money.

"In 2014 total federal compensation averaged $119,934, or 78 percent more than the private-sector average of $67,246."

http://www.downsizinggovernment.org/federal-worker-pay

We are talking about the best of the best, the cream of the crop when we are talking about Meritocracy, not the private sector average.

I'm saying these people wouldn't be interested in wasting their time doing such nonsense and be on top of that poorly compensated.

$119,934 is ridiculously low pay for top performers.

You are right, I forgot the adversarial nature of these requests. I am not sure less paid, less meritous people would work better though.

I generally agree, but when the hiring and management of the persons processing the requests is carried out by the same parties interested in delaying the process, the principle goes untested.

I think this is less about a deliberate attempt to frustrate FOIA, and more about lazy government workers who see no reason to fix something that isn't broken (in their opinion).

It is departmental policy to do the most limited search possible in an attempt to return as few results as possible. That is the very definition of a deliberate attempt to frustrate both the letter and the spirit of FOIA.

Reply on news.ycombinator.com