Hacker News Reader

Comment by patd | Hacker News Reader

For the ballpark estimate, I think they just wanted for the request to go away by quoting an insanely high price.

Long long term resident with a connection to local government

They don't have a choice. Seattle IT is so underfunded that hands are tied because there isn't any resourcing.

On one hand, you have to respond to all of these requests (and rightfully so, as it's the law.) On the other, you have no money for your department because it has no funding because the citizens didn't want to spend the money.

The person who did this isn't malicious. Just very overworked and did a data pull wrong. They probably didn't give a shit to check because their job kinda sucks and they have too much to do already.

A good chunk of this is caused by our city repeatedly choosing awful vendors that bilk the city for crazy amounts of money, and provide trash as the final product.

City Light and the new meters/new billing system are great examples, all the new power meters have no encryption, and use FSK for modulation. Asking City Light about this got me a response that FSK was the encryption, and the gal was dumbfounded when I pointed her to the Wikipedia article on FSK.

On the billing side, an Oracle salesman ran off with over $100 million in city funds for what is essentially a CRUD app, and the worst part is they didn't bother to customize this system, just forklifting this in place and letting the chips fall where they may. The prior billing system had quite a bit of data validation and business logic that has yet to be implemented or replicated on this new system. The same actions when you call customer service now take significantly longer.

Both these vendors fleeced the city for broken, insecure systems, and neither is having to face the music for it. Worst part is, eventually someone may attempt a fairly trivial exploit of either system, which could wreak havoc in our city.

The vendors may be horrible, but it's still on the shoulders of the city for choosing the vendors. This is really down to the fact that most decision makers have no idea how to understand or differentiate between options. It's been like this for decades. When a secretary of state doesn't know the risks in running a private independent email server and how to ensure those risks don't become issues, how do expect much lower level city governments to make any better decisions. Honestly, the IT workers are often underappreciated, underpaid, and underskilled. I usually don't blame the IT people. If they could work at FANG, they would. If they get into a rut where they end up not caring anymore, it sucks. But let's put blame where it belongs, it's on leadership to develop a culture where people care and on leadership to invest money and resources accordingly.

Edit: though I shall add that it unfortunately is sometimes not possible to build a new culture without firing incumbents. That one is a really unfortunate situation. It's not the fault of low-skilled IT workers if they are enabled and rewarded for poor skills and attitudes. But if an organization wants to make good cultures, it sometimes requires hard decisions. Still don't blame the workers, just like I don't blame factory workers that get automated. It's just an unfortunate thing that can happen that most people don't deserve.

I don't think it's good enough to say « it's still on the shoulders of the city for choosing the vendors ».

If I write a piece of software which is technically capable of meeting its requirements if you read the manual carefully enough, but in practice the intended users can't figure out how to do so, that piece of software is no good.

Similarly if the market is in principle providing IT vendors who are capable of providing a decent service, but in practice the purchasers can't figure out which ones they are or how to make them do so, the market has failed.

Do you have a viable method to fix the problems you describe? If not, then we're still left with it being on the shoulders of the city. Ultimately, it's really hard to police that the vendors don't make crap solutions. If the market fails to figure out which are crap, the market may have failed, but I don't have any ideas that would succeed better.

Edit: and let me say that I posited that the issue isn't market forces. The issue is lack of expertise at decision making levels. Even if there were zero market, it wouldn't stop people from doing it wrong.

Principally: start punishing the corrupt vendors; don't assume that reputation mechanisms will ensure that non-corrupt ones will eventually outcompete the corrupt ones.

Just as you have to build software for the users you have not the users you feel you deserve, we need a service industry that works for the service-commissioning agents we have.

Again... how would you do it though? Who would do the punishing? How would you enforce the punishment? Would clients build the punishment into the contract? Even if you could do that, the vendors have the advantage of how to construct the contract in their best interests and avoid punishment. Most ideas will go straight back to market forces. Again, the issue is expertise to choose good options, not the existence of market forces.

> Wikipedia article on FSK

https://en.m.wikipedia.org/wiki/Frequency-shift_keying

> The demodulation of a binary FSK signal can be done using the Goertzel algorithm very efficiently, even on low-power microcontrollers.

The source links to the TI page for the MSP430, named because it originally sold for $4.30. While the original link is dead, an internet search reveals many side and college projects using this technique.

>A good chunk of this is caused by our city repeatedly choosing awful vendors that bilk the city for crazy amounts of money, and provide trash as the final product.

How much of this is voters voting for politicians who are good at what is essentially a popularity contest instead of politicians who are capable of signing decent contracts? As long as voters don't care enough to change their voting habits this will continue to happen.

I heard from my city solicitor (whose office handles these FOIA requests) that one issue they have is that when people make requests of other city departments the default response is “go make a FOIA request” even in straightforward situations where they could easily give out the info directly (and used to, before the FOIA was passed). He said it’s annoying because it ends up putting a lot more workload on everyone.

It was a reasonable ballpark. Let's say a city prosecutor was working on a organized crime case that involves the FBI and other people.

Based on timing of emails this would leak the list of people working on the case, maybe informants and put them at risk.

We all lost our collective shit when NSA said they're only collecting metadata. Metadata is Data.

Since they revised the cost for the data they actually sent him down from $33M to $56 (90 days of data at $1.25 for 2 days data), was a ballpark estimate that's over 500,000 times higher than the actual cost really reasonable?

The ballpark estimate was for checking the full text of each and every email (which was a misunderstanding -- the author wanted metadata only).

The actual cost of servicing this request was much greater than the $56 estimate.

Especially after the city made a mistake!

$32M is not a cost the requestor bears, but an estimate of the cost the city bears.

IT estimated the requestor fees to be $21k/year assuming 10TB of data.

Clearly, IT were estimating the costs of releasing all email text, because FOIA mistakenly changed the words "please provide the following information:" to "including metadata:"

Once they figured out the request was for header information only, the city came back with an estimate of under $60.

> Metadata is Data

And in many cases metadata is just as useful as the payload, in some cases even more useful.

Reply on news.ycombinator.com