New York makes complaint records of 83,000 police officers available to public

We can police the police by tracking them with data. Complaint records are amazing to have, but we can also find dirty/racist cops with existing data from county public records. It just needs to be scraped and collected. Redditors and folks at HN started this project, and we are making progress, but it's a big task. With enough volunteers it will be possible to create a database and scraping infrastructure that will enable citizens to truly understand how individual police officers and departments are behaving. Context: https://www.reddit.com/r/privacy/comments/m59o2g/i_think_i_a...

Note that this does not actually make the complaints themselves public. It's honestly a pretty lame attempt at transparency. With these sorts of weak datasets being published, we shouldn't feel comfortable with this near-useless data until NYPD releases the actual complaints themselves (with proper pre-release review, of course). I'm not familiar enough with NY to know whether their public records law allows the release of complaint files, but they should be FOIA'd if possible, and if not, then things need to change.

Contrast this dataset to Chicago's COPA's release: https://data.cityofchicago.org/Public-Safety/COPA-Cases-Summ.... This dataset is also missing information, but at the very least it gives context to complaints. What it's not missing is the underlying information that led to the complaint, the depth of investigation, a written summary of findings, etc. Happy to show anyone who's interested how to FOIA for that info, though.

Are these sorts of datasets useful? Sure, but at the non-trivial cost of an off balanced trust model where many folk who are just beginning in this work will think is the end-all, be-all and won't push for more. In a lot of ways, these releases don't feel like progress to me anymore. Particularly considering that a lot of these sorts of datasets are filtered, while implicitly hiding a significant amount of information, often without detailing what's missing, leading to biased analysis from the beginning. Keep in mind that these cities treat those who want access to data largely as adversarials, with things like this happening: https://www.chicagoreporter.com/chicago-police-department-ar...

Relatedly, Chicago had a huge win on this front with the Green v Chicago lawsuit, which resulted in about four years' worth of complaint PDFs, video and audio between 2012 and 2016. The lawsuit itself is supposed to yield about 40 years worth of complaints, but it's a slow process that the City's fighting back on -- last year they attempted a $500k buyout last year, but it failed after public outcry and it's hard to say what's coming next, or if the remaining ~35 years will be provided. I have sincere doubts.

One of the only ways that we, as a public, can get access to deep information that leads to real understanding is to get access to the underlying records used to build these datasets. That means submitting FOIA requests and pushing back on bogus rejections through litigation where needed. Sadly, not many folks do this.