Hacker News Reader

Comment by pseudo0 | Hacker News Reader

Would results of discovery be public record? If so, I could also see:

3) They don't want their "secret sauce" made publicly available.

That would both open them up to commercial competition, and make their product less reliable in court, as anti-Shotspotter experts would start figuring out how to effectively argue against the product's methodology.

In general I think technology like this should have to be fully transparent if it is used as the basis for public entities' decisions, but so far the arguments against that practice seem to have won out. For example, good luck getting source code or design documents to contest an automated red light or speeding ticket.

View on HN

This is a reasonable argument, but I'm not sure you're aware of allegations of this company altering evidence to better fit a police narrative: https://www.vice.com/en/article/qj8xbq/police-are-telling-sh...

I sincerely doubt there's any secret sauce to protect when they have "analysts" sitting around regularly "correcting" evidence.

I agree that there is definitely moral hazard when it comes to after the fact "corrections", particularly if the analyst knows the outcome desired by police. The company's track record looks pretty poor in this respect.

In general though, I think there are legitimate reasons for manual analysis. Any machine learning approach to a problem like this is going to have to balance false positives and false negatives, and there is necessarily going to be a somewhat arbitrary cutoff. Detecting bang-like noises in a large city is likely going to have a pretty conservative cutoff to avoid DDOSing the police with calls. For example say the company sets their cutoff for automatic reporting at 90% confidence, but when the police ask them to review a specific time period, it turns out there was a shot detected with 89% confidence, and manual analysis indicates a false negative. That is probably still useful information, and it would definitely be useful to include this data point with the correct classification in future iterations of the model.

They could ask for the information to be under seal to protect their commercial interests.

The actual technology is not all that complex. Transient detection and triangulation by decorrelation from a microphone (edit: not microwave!) array. You could probably prototype it in a week if you leaned on an established FFT library. Obviously they would not want to reveal their source code and so on, but if you know digital audio you know this isn't that hard.

I think what they really want to conceal is not the technology but its use in operation. I don't know what Chicago pays for the service, but Sacramento CA spends about $1m a year on it and Sac is a fairly small city. https://www.sacbee.com/news/local/article231997777.html

Chicago Aldermen are no doubt getting a generous kickback on whatever they are paying.

While true the tech is not complex, I believe the discrepancy is "how accurate is it and is the data being manipulated to fit a crime" versus "the data is accurate and leading to fair justice outcomes."

Wouldn't "red teaming" ShotSpotter solve for this to see how accurate they are? A team would perform controlled firings in known locations under various auditory scenes with the city's approval, and then request the data from ShotSpotter to see how accurate they are (without revealing you are performing an exercise). ShotSpotter then doesn't have to reveal their methodology, as their data provided from a blind test will reveal the accuracy regardless of how they arrive at conclusions. The system is either accurate or it isn't (with a measured level of confidence), and this can be tested for without ShotSpotter's consent (as long as they continue to provide data to the customer per their contract).

That would tell you if it correctly detected gunshots... but it wouldn't tell you if it falsely reported certain sounds as gunshots, if it works in all weather, etc.

This would work to give some evidence for the accuracy levels of the automated parts of the system, at least for true positives vs false negatives, but doesnt test for how the system generates false positives.

A bigger problem is that this doesnt approach any of the process problems with the human moderated parts of the system. For that you would need cooperation from the law enforcement customer. The customer would need to make a request like "are you sure you didnt find any gun shots around this time and place?" and see how the company responds.

> and make their product less reliable in court, as anti-Shotspotter experts would start figuring out how to effectively argue against the product's methodology.

If their methodology has weaknesses which allow those arguments, then its kind of a travesty of justice not to allow them.

There’s no way they have a legitimate secret sauce though… right? This doesn’t seem like a hard problem to do. Maybe some of this

https://en.m.wikipedia.org/wiki/Independent_component_analys...

Plus basic triangulation logic synced across a bunch of microphones?

There is also the matter of correctly identifying gunshots.

They get lots of false positives, from band practice to backfiring vehicle engines and (of course) fireworks.

It's phenomenally hard. Echos, indirect routing (bouncing off buildings), sound waves changing speed as they go through various materials (like a lens). It's a very hard problem.

Heck, a truly calibrated surround sound system requires using a microphone to measure the volume in place.

So can't you just do that? Install the system and then drive a calibration car that emits sounds and logs gps coordinates?

Map every city in the US and world?

Now that's a lot of work.

Google was collecting point cloud data of US cities 15 years ago, I assume some GIS vendor somewhere is selling a copycat dataset to self driving car startups.

I doubt you could do it just based on point cloud data.

Could just do it for the areas that sign up for the system if it worked.

I would definitely object to any system that required a car to creep along every block and regularly make a simulated gunshot sound. It sounds prohibitively annoying.

And how often would it have to be repeated?

> It's phenomenally hard. Echos, indirect routing (bouncing off buildings), sound waves changing speed as they go through various materials (like a lens). It's a very hard problem.

I doubt they've solved (or even have non trivial mitigations for) any of those subproblems, based on stories of how often they apply human "corrections". The portion they've implemented may be very easy to replicate.

Sorry I meant to the level of accuracy they seem to have achieved. Which seems to be loosely flagging the location of bangs.

If the real concern was secrecy, there are protective orders and other techniques that could go pretty far to assure this (but nothing is impossible).

I think the most reasonable inference is that they are outright trying to hide. And probably because the truth is totally damning.

I'm sensitive to the idea their methodology could be used to create a competing product, and think the court could get the experts to agree to event more stringent than normal NDAs to examine it. But I think if a computer system is going to be used to convict someone, experts effectively arguing against it's methodology is good. In fact, I think experts should have access to various machines that could convict someone.

I don't really care about a red light camera (as the image shows the license plate and the red light at the same time), but there are tons of other technical solutions that need to be examined.

If your business is to help uphold the public rule of law it’s a stretch to want to keep that process a business secret.

You can’t make rulers while also trying to keep the definition of an inch secret.

I think this company is probably more successful the less potential buyers know about it, the product sounds like total trash to me.

For real. I wonder how active phone microphones and geolocation are generally -- I know there are a host of location tracking (marketing) companies that use app geolocated data as their sources.

Please report to the nearest processing center for your attempted libel against the police-industrial complex.

Reply on news.ycombinator.com