Show HN: Play with an interactive heatmap of SF crime (and other cities) | Hacker News Reader

Show HN: Play with an interactive heatmap of SF crime (and other cities)

safemap.io

You need to divide crime counts by population density. Otherwise you're just plotting population density.

Depends on how the map is being used. If I'm looking to avoid traveling through neighborhoods with a high robbery risk (e.g. as a tourist), I'd prefer absolute numbers vs. per capita.

That's probably not the correct analysis, because the number you're trying to estimate is not P(a crime occurs near you), it's P(you will personally be the victim of a crime). Crime rate over density won't necessarily do that either (since the population density of a primarily commercial+touristy area, e.g. Fisherman's Wharf, is << the number of people who are present there on a given day), but neither does raw count.

I mean, I'd also be pretty interested in avoiding P(a crime occurs near you), especially as a tourist.

Yeah there are only 24 hours in a day, so the right metric might be density of crimes in an area per hour. Even if I’m not the victim of a robbery, I don’t really want to be watching it from across the street.

> I'd also be pretty interested in avoiding P(a crime occurs near you), especially as a tourist

What counts as nearby also scales with density. Down the street is nearby in the suburbs. In New York the next avenue block may as well be another town.

As a tourist in SF recently I was interested in seeing the whole city, warts and all, and I did.

I don't really understand why as a tourist you'd want to use this map for a short trip, unless you were going somewhere that's a literal war zone. Most cities just aren't statistically dangerous for a tourist unless you go out looking for trouble. You're much better off applying common sense than digging through digital maps of (reported) crimes.

If I was looking for somewhere to live then it would be more interesting I think.

I dunno, I'd think car break-ins would at least be a very important statistic.

My wife and I will be spending a day in the Fisherman's Wharf area, and the statistics on car break-ins has me concerned, though a co-worker who lives in the Bay told me that if I'm parked in a parking garage, it's less likely I'll be a victim, as it's harder to smash-and-grab-and-go when you have to deal with a parking gate.

if you’re really worried, don’t leave anything in your car and don’t lock it

I don't understand this advice. Are you suggesting that if you make it easier for thieves to break into your car, they are less likely to damage it?

yes and also do not leave anything in your car

Yeah we parked across from the Exploratorium near there for several hours and it was fine, I didn’t think to consult crime heat maps before visiting, just followed standard common sense of not leaving visible ipads etc in the car.

What is the actual car break in statistic there?

This seems like the best use-case, finding a place to live in a city. As a SF-area resident I know the Tenderloin is a no-go zone, but besides that not much. This is a good tool to see which neighborhoods to focus my search on in the apartment-finding process.

>> If I was looking for somewhere to live then it would be more interesting I think.

When I was looking at places to live in South Carolina, Florida and places in the Northwest like Oregon and Idaho, it was interesting to see some of the crime maps and grades these sites gave for crime.

In small coastal towns in Florida, they had small populations and were essentially tourist towns or inhabited by "snow birds" people who live in Northern states who live there during the Winter. Several of the places got D's or F's for crime. I was just shocked, but reading the footnotes, they made it clear that the numbers could be skewed because of the transit population or other factors like Spring Break revelers who come down, get in fights, people get assaulted, or sometimes killed and suddenly a small town's crime rate goes through the roof because of an outlier event that year.

It just made me leery about any statistics you see online with cool heat maps or grades they give to certain areas.

As an aside, the same thing was evident in small rural towns in the mountains in Idaho as well which also got poor grades for crime. It aligns more with your comment, it you go looking for trouble, chances are, you're going to find it.

Yes, data and statistics aren't a substitute for careful thought. You shouldn't throw them out but you definitely can't outsource your thinking to them either.

That's part of why online forums can be nice: a well moderated one can be great at pointing out blind spots in your thinking.

I, as a tourist, would very much prefer being able to walk around looking at random things and relaxing instead of being on the lookout for people trying to steal my things or worse.

Also, when on a trip, getting my phone stolen, for example, is much more of a hassle than having it stolen on the street where I live. Bonus points for thieves actually going for bags and such, so now it's likely you'll need to pay a visit to your embassy or whatever to have your passport replaced, too. In the city where I live, if my ID gets stolen, I'm likely to have at least a different form at home, and I also pretty much know where and when to go to the authorities to try to obtain a replacement.

Speaking of SF, I have no idea if my country even has any kind of representation there, where it would be, how I would get there, and how long it would take to have temporary papers made up, especially without a phone or similar device.

Thanks for checking out the site and for the feedback. It's definitely something we're looking into.

It's kind of an interesting question though because it somewhat varies by crime category. Take car break ins, for instance - this is more of a constant factor than a population density scalar because the number of parking spots is generally fixed on a per-block basis rather than scaling with population density.

I think we might try to give users the option of dividing by population density but allow them to toggle this on/off.

Parking turnover probably scales with population density. One of the reasons parking is more expensive in big cities is to encourage people to move their cars sooner. Smaller cities will have more cars that never move, where larger ones will have fewer.

If it's about movement, then population density might not be the best factor either. You might need some other way to see the number of humans moving through a given area. Stuff like office buildings have higher populations during office hours, but wouldn't be caught in "population" metrics.

Yeah, I think having it as an option is the way to go. Though I’d personally have it default to per-capita, as I do think that correlates closer with actual risk. To illustrate with an extreme example, an area with 10 burglaries per 100 people is clearly a way higher burglary risk place to live than an area with 20 burglaries per 10,000 people.

The number of parking spots may be static, but the number of potential thieves scales with population density.

Does it really? Some classes of crime have been narrowed down to the same 50 individuals. You can't model car break-ins as everybody in the area rolling the dice on getting a taste for breaking glass and grabbing things.

it scales with a factor of less than 1.0 but yes it scales. For instance the number of thieves could be given by the ratio .001 * population.

yes, And this is why it's so frustrating the police don't just setup a few honeypots and catch those 50 people. Sooooooo many citizens would be helped, neighborhoods too, ...

As a toggle would be useful because depending on how you're using the map you might want to see per capita or absolute numbers. (Personally I think it's more useful per capita.)

Car break ins still depend on thievs that scale at some rate in the population. There’s finite time to break into a car.

That depends if people care more about crime happening to them or crime happening around them.

this is not a map of population density it’s pretty clear just the SRO strip/TL and then the two mission bart stations

But it's interesting to see that this is not true for all crime categories. E.g. car break-in hotspots are by Fisherman's Wharf and the side of Alamo Square with the Painted Ladies -- i.e. prime tourist locations.

However, I suspect there are some basic data errors upstream of this visualization tool that also limits its usefulness. E.g. 'rape' has a giant hotspot centered at the south side of the intersection of 8th and Mission ... which is site of a city Adult General Assistance office. I am guessing these are being tagged with the location where the victim gave a report, not the location where the crime happened.

Is it actually true that crime in an area increases proportionally with population? Why would it?

Extremely high density but near zero crime would be inside a large office building.

there's such a thing as a "target rich environment" but I always look at these maps as "where people are", college campuses, tourist attractions, crime is high because the number of marks is high, its a coincidence if many people happen to live there too

It's an important truth that walking a mile in a high population density area puts you at a higher crime risk than walking a mile in a medium or low population density area.

Not really, since your chance of being a victim is divided by the number of people in the area.

> Not really, since your chance of being a victim is divided by the number of people in the area.

Depends on behavior. If I stay in my room all day, I don't contribute to your divisor. If Jack Dorsey doesn't walk around alone past midnight, he's not absorbing the same risk quotient either.

> If Jack Dorsey doesn't walk around alone past midnight, he's not absorbing the same risk quotient either.

Jack Dorsey is a weird motherfucker and a real outlier for this example. He used to walk to the Twitter office every day and generally looks and dresses like someone well outside his economic strata. I would not be surprised to find “midnight solo perambulations through the tenderloin” among his hobbies.

It's the opposite - right? There are more gross crimes in the high-density area, but your chance of suffering one is much lower (all else being equal). Trying to apply statistics directly like this is generally inaccurate, but to the degree the numbers say anything it's the opposite of what you say.

Fair point, it's more complicated than I originally thought.

Density increases the number of potential criminals you pass but also increases the number of alternative victims they could target.

If you're a uniquely attractive/visibly wealthy personal walking home from a night out then the risk of passing more criminals dominates the advantage of alterative targets. After all, there's nothing really competing with you.

However, if you look like normal person but happen to have a million dollar watch in your pocket then you're conceivably safer in a city. Because there'd be so many alternative victims who may be more attractive targets than you. Compared to the countryside robber who only gets a single target per night.

There's also a lot of evidence that a given individuals' chance of being the victim of a particular crime derives from their social situation. Most people who regularly do crime are used to doing crime in a way they can repeat, and those crimes most often target locals and people from the same social sphere. There are also plenty of factors that lead to the targeting of tourists or wealthy people, but most crime victims are poor and live in similar conditions to the person committing a crime against them.

Obviously this isn't every crime! People do have unlikely crimes happen to them all the time! Just that they're statistical anomalies and the population level statistics will mislead you about how high your risk is.

The population density in Singapore is much higher than in Detroit. Guess where you are more likely to become the victim of a crime..

Not necessarily, as high population density generally means lots of police and lots of cameras (both security cameras and individuals with smartphones)

And you have to use daytime, not resident, population estimates, or a combination.

Yes and no.

Not dividing by population density ~ Probability I will become a crime victim by travelling through that area

Dividing by population densitiy ~ Probability that I would be a crime victim if I live in that area

Not necessarily, people may (or may not? maybe it’s regional) be more likely to victimize strangers.

> Probability I will become a crime victim by travelling through that area

Still not that. P(you’re the victim) = crime count/potential victim count.

Population density may not be the right number for the denominator, but you need something in there.

I could travel through a rural area with an extremely high per capita crime rate but my chances of encountering ANY person (good or bad) is low. I effectively avoid crime by avoiding people.

If you live somewhere, you are forced to be a part of the community, which subjects you to the per capita rate.

And the whole WWII plane meme concern... this is where crimes were caught/reported, not where they occurred.

Divided by population one of highest crime rate areas in the UK is Westminster because this are attracts a lot of tourists (some are targeted by pickpockets) but not much people live there (denominator is small). A better metric will be crime count divided by man-hours spent in the area per day. Such data unlikely exists in open access but mobile operators should be able to estimate it based on base station logs.

I don’t think there’s anything particularly wrong with it. What is it that you think is misleading from this metric?

It's not wrong but what people usually want to know looking at crime stats map is a probability to become a victim if they visit a particular place of will more to live there. Crime count divided by number of residents only relevant for crime targeting residents like home burglaries. For other cases these number can be no-representative.

It should really be crime count by number of people out and about a day, but you will never have that number without some orwellian citizen surveillance program.

As always there is a relevant xkcd: https://xkcd.com/1138/ So many geographic statistics just boil down to population density maps instead.

the issue is that the the specific population density is the density of workers, tourists, and people commuting through the area not the density of people who live there.

Can anyone speak to whether public statistics are accurate or not? Can anyone reassure me that these statistics are impossible to manipulate?

I have a suspicion these statistics are only accurate and available when it is convenient for the people responsible for providing those statistics.

Isn't it the case that the police are responsible for providing statistics to the public? During the pandemic and during the George Floyd protests, there were political reasons for gathering information and using to show that A) "crime is up!" or B) "crime is actually still historically low!"

A few years ago, I was frustrated about the reporting on these statistics, and went to a variety of police web sites.

Some of the websites said: "Because of the pandemic, we are so overwhelmed by crime and everything, we cannot provide current statistics!" And, other sites said "Because of the pandemic, we are so overwhelmed by crime and everything, we cannot provide past statistics!" And, some websites just completely won't load. If a police department does not want to pay their IT person, or the company that provides it can't get their act together, the website will be down and you cannot get information directly from the police.

If you have a category of crime that is high, the police need to either fix that problem, or fix the numbers. Right?

I know that some sources of information come from the FBI. But, aren't those numbers often provided by the police?

And, in certain parts of the country were in favor of defunding the police, and certain parts of the country are filled with "Blue Lives Matters" people. Won't the police in the defund areas want to show crime is up? Won't those in regions with different sensibilities want to show their policing works? Can anyone tell me why this isn't a problem?

It is anecdotal, but aren't there areas of certain cities where the citizens have stopped reporting crimes because the police don't come anymore? How is that reflected in the numbers, if at all?

Can anyone else see a big gaping hole in how these stats are collected and used? Is there standardization across the country about these statistics?

I'm sure, like anything, there is incentive and opportunity to manipulate the numbers. but I don't think it would be easy to do without being found out. Police precincts do a lot of paperwork and behind any statistics, they are going to need the paperwork for individual cases to back up the numbers they submit. and if they are going to under report, they will need to destroy a lot of paperwork. but my intuition is that while leadership may want to under report crime, cops will usually want to report as much as possible to prove that they are busy and necessary. so the balance likely gets us acceptably accurate numbers.

I just don't buy that it evens out.

For example, go here:

https://cde.ucr.cjis.gov/LATEST/webapp/#/pages/explorer/crim...

Look at Florida: "The 2022 estimated Crime statistics for Florida are based on data received from 403 of 758 participating law enforcement agencies in the state that year."

That's 54% participation.

Look at Oregon: "The 2022 estimated Crime statistics for Oregon are based on data received from 215 of 236 participating law enforcement agencies in the state that year."

That's 91% participation.

Why don't police departments in Florida report? Why aren't Floridians demanding that reporting? Florida has incarceration rates double of Oregon by the way. Do Floridians not want to look at that uncomfortable fact?

Portland has been pilloried in the news the last few years as the example of liberalism gone wrong. But, IMHO, liberals are asking questions about law enforcement, and when you report fully, there are going to be examples of wrong doing. If you just don't report, then it is easy to hide.

A few days ago I had to collect the body of my cat from the Polk County Florida sheriff's department so I called to make sure I had the hours of operation right. I was on hold and told over and over "Crime is at a 50 year low!" It felt like propaganda, and it feels even more so like propaganda after seeing the above.

That might be a problem for state-level reporting but I'm not sure it really maps on to the city of San Francisco very well, tbf

Good point. Thanks.

This map makes me think of the New Inquiry's White Collar Crime map [1]. That project creates a strong image which for me disrupts the stereotype of what crime looks like and where we can find it.

I realize it has a very different aim than safemap, but I frequently use it as an example of how the selection of data sources and its visualization is crucial in communicating a particular story. This communication already starts with the name: calling a map of reported crime a 'safemap' implicitly suggests that red colored places are 'unsafe' and should be avoided. (The about popover literally states: "Learn which parts of San Francisco are safest and which parts are best to avoid.") Which, as the discussion here suggests might be more complicated than that.

[1]: https://whitecollar.thenewinquiry.com/

Thanks for making this, it would be helpful if the city boundaries weren't so closely observed, most people don't care where the exact city lines are but have more of a feeling for the general metropolitan area as a blob

I wish that too but it seems their data sources are the cities themselves so those cities don't report things outside their jurisdiction.

Pretty wild that you can intuit the city boundary by using the car break-in stat, it’s the most evenly and widely distributed one I saw.

Zillow used to have this years ago, and they dropped it. I remember using it in 2016 when house hunting. Later in 2022 I noticed they had dropped it; I'm guessing it reduced the number of home buyers.

It's interesting to see how certain crimes (e.g. Assault and Robbery) have hotspots that map perfectly onto BART stops (16th & Mission, 24th & Mission).

it’s not surprising if you’ve ever lived in that area

It is somewhat surprising actually. Presumably if there's an actual hotspot that police would be ever-present in the area until it dilutes out

> police would be ever-present in the area

you live in the area? because your comment doesn’t sound like it

Stopping crime means arresting marginalized people, and SF leadership won't let that happen.

If police were ever present dealing with crimes, that’s actually how we would see hotspots. No police, none of that gets reported, and bart stops fade into the background.

Possibly because the crimes are occurring aboard the train, so the exact location of the crime is unclear?

probably not - the areas outside these bart stations are known as crime hotspots

made that mistake once and never again, not fun when it's pitch black out, you're waiting for a bus, and there are drug addicts crawling around doing weird shit

The rest of the hotspots are just 2 more stations, Civic Center and Powell, too.

Also serves as a map of where to find prostitutes. Apparently, that's the area around South Van Ness & 20th St in SF.

Capp Street is the center of the issue - so much so that permanent bollards have been added to cut down on customers circling the blocks looking for people selling themselves.

https://missionlocal.org/2024/04/barricaded-capp-street-park...

I am not a fan of the kernel density estimators with a continuous color ramp (at least this map the KDE does not recalculate on zoom, that is pretty much worthless).

Currently the UI on the left returns all empty images for me, https://www.dropbox.com/scl/fi/re068bj1vnaitsoh59x27/NoImage...

I rather enjoy using either points directly (and for dense point maps use leaflet's cluster markers), or actual outlined areas (e.g. using DBSCAN). For a few examples I have put together

- Dallas crime dashboard (using wasm + python), https://crimede-coder.com/graphs/Dallas_Dashboard

- Durham NC hotspots (leaflet), https://crimede-coder.com/graphs/DurhamHotspots

@SafemapTecnolgs, we (newscatcherapi.com, YC S22) are helping one of the largest NGOs build an interactive map of all reported corruption cases.

We collect over 1M national and hyper-local news in near real-time. Happy to help with collecting crime info from public news.

I'm happy to help for free; we'll also pay to develop & host information extraction using LLM!

Anyone, feel free to email me artem@newscatcherapi.com

The TL is high-crime and the Mission bart stations have crime. Not exactly shocking to anyone who has lived here

I was interested to see that there are only really prostitution arrests in the middle of the mission!

yes, shotwell and capp are a pretty obvious corridor if you want to witness it ever

Nicely done. Longer term historical data would be nice too, for comparison or trends. It should be available as these sorts of crime maps were all the rage 10-15 years ago.

Personally, I would prefer that the crimes filter panel automatically hide itself when I click on the map after selecting my filter.

Kudos.

Thanks! Yeah, we definitely plan to add long term historical data and give users the option of diving deeper into the trends. Should be in the very near future. Thanks for the idea of auto-hiding the sidebar on a map click - that's very intuitive. Will probably push an update later today to add this.

This is very cool.

I turned on all the types of crime and a weird set of diagonal darker regions appeared in the avenues. Either this is some weird artifact of the geocoding or there's some really strange magnetic fields or something affecting crime.

Thanks. Yeah, we noticed this too. It seems to be just a strange quirk of the SF dataset, and only in the Richmond / Sunset districts. Or who knows, maybe it is true.

Great work!

- The maps are pretty and scale nicely. The clarity and granularity of text is appreciated.

- The controls are excellent. I might put the copy and help in the same single title/menu panel (without "viewing:" as obvious)

Some other features I found myself wanting:

- Visually bound the map so you can see when it can't be moved in a direction. Think about what to do when the control panel is open and the user is trying to drag the map to see what's under the panel (instead of closing the panel).

- ability to sort data drill-down by number, to look at most prevalent

- using gradients for data starts to break down at low numbers. I wonder if you can start putting dots under some threshold

- time-range delta heat-maps: 2023 car break-ins vs 2022 (where green is improvement, red is worsening, and intensity is size of delta)

- surfacing qualitative aspects of data sources: one or many (only police?); and perhaps user feedback on quality of data

- curate super-categories, e.g., capturing all cocaine-related offenses, or distinguishing the stacked charges (loitering in the area of ...) from the operative ones (dealing), or crimes against (visiting?) strangers vs locals or friends. The data sources have their own arcane categorizing logic, and it's rarely what users actually want.

- color-code user-specified combinations: prostitution + drugs vs either alone

- Most of these relate to the kinds of questions users can answer with the maps, so if you have a (user-contributed?) panel with questions and their associated configuration sets, it would be very powerful and sticky.

- Then it would be nice to apply the same configuration-set to different cities, and perhaps to see some comparison across cities

Some things I wouldn't do:

- All-caps, tm? ok, but...

- I think putting safety under quality of life makes the product a bit less polarizing and hence more popular - which means adding other quality-of-life measures (e.g., zoning (residential/retail/manufacturing), cost/sf for housing, walkability scoring) E.g., it would be interesting to know which retail districts are super-safe.

Thank you for the great suggestions. You've given us a lot of great ideas to think about. Much appreciated!!!

Awesome job, any chance of extending this to Europe? (Like Paris)

Thank you! We are definitely interested in expanding to as many cities and countries as possible. We focused on large US cities with good datasets for this MVP, but depending on how things go, we will hopefully be expanding to cities across the world very soon.

Location based danger zone notifications would be helpful. "Warning: You are entering an area with a large number of reported muggings."

or a warning every time you get into your car "Warning: locations like these have a significantly higher fatality rate"

“Warning: its 8pm on a friday night and half the cars you see from here on out are drunk”

Stay tuned for some exciting feature launches in the near future :)

Please let me configure it so that I can play Kenny Loggins' Danger Zone as my notification.

There is an app called Citizen that notifies you about crime happening near you in real time

Damn, the police should start using that app!

It's vise versa. Citizen app listens to the police radio and puts the gps coordinates on the map with the descriptions taken from the police report. They have social network on top of that where people can submit videos and live videos

Takes phone out of pocket to see what the notification was

https://www.cnet.com/culture/the-joy-of-microsofts-avoid-ghe...

Hey, remember me? We met randomly on a bus from Machu Picchu to Cusco a couple weeks ago! First "orange site" people I've met IRL lol.

Great work on this - very slick UX and super quick.

Be cool to expand it to support the UK or maybe develop some open standard for the way data can be reported. I'm sure businesses would pay for a consolidated data api.

Hey Josh! I was wondering whether you'd see this! Thanks for checking out the site - we'd love to expand to the UK and to add an API, thanks for the ideas.

It was great meeting you and chatting with you on that bus ride, hope you folks had a nice rest of your trip! And hope things are going well with LoginLlama and the other cloud storage related project you were talking about. Let me know if you ever need some beta testing done or something like that :)

This is really cool. I see your using open street map, is that heat map you're using open source, or something you've built internally?

Looking forward to my business trip to Portland in a few weeks... The place where I am going is right in the middle of the big red blob, when the "Assault" is selected. Is it really that bad?

Do you have a version of this, but with positive data? For example, spots with great weather, or where acts of kindness frequently happen, or lots of photos get taken (so excellent views). Might be fun!

The location of the police stations seems influential. I know in Berkeley crimes without a firm location get geocoded to police HQ. Do you think that might be affecting these maps?

Interesting, thanks for the info. I think it hopefully should be OK for the currently supported cities. Lots of cities have 0,0 as the geo point for crimes without points, which we filter out. For others such as SF, it looks like the police HQ is in Mission Bay which doesn't seem to have a high crime rate for any category. This info would also hopefully be included in the dataset disclaimers, and we didn't see this in any of the datasets we use.

But thanks for the info, this is great to know for any potential future expansions. If you see any cases where this might be happening for the currently supported cities, please don't hesitate to get into contact with us :)

SF has several police districts and district stations - may be worth checking against those, too: https://www.sanfranciscopolice.org/your-sfpd/sfpd-stations/s...

Filter by prostitution - Red light district guide ?

Add more countries please

FYI none of your images are loading for me, they all return 301s repeatedly until the browser gives up.

Hmm, thanks. Hopefully this doesn't include the map itself. Would you kindly provide the browser / OS combo you are operating? Will try and look into this.

Firefox, Windows. The map loads fine, just the icons don't work.

Damn, wtf is going on at Little Saigon and that Park hotel?

It's right smack in the Tenderloin

Very cool! Car Break-In useful for “safe to park here?”

Thanks! Yeah unfortunately with cities like SF nowhere is truly "safe to park" but hopefully this helps people avoid the major hotspots!

It's actually quite safe to park in SF with two exceptions:

- With anything visible that might contain a valuable. Any devices in the trunk should be in Airplane Mode. If you're a tourist, don't park by a tourist trap.

- Any model with an oft-stolen catalytic converter, esp. Prius.

SF itself has higher-than-average tourist density, so it's a burgling destination too. But it's possible to park for decades and never have your car broken into.

This is cool! You should add selfishly Raleigh NC

Looks like a GTA map

where did you source your data?

Public datasets. You can view the sources for each city by expanding the "Data Sources" tab in the sidebar menu.

@dang or other HN mods - can you help me understand why this post was so rapidly downranked? It's now on the 6th page... it's got to be the highest upvoted yet lowest ranked post submitted today. As far as I can tell there were no flags or anything. Am I missing something here?

Two things: (1) it set off the flamewar detector a.k.a. the overheated discussion detector; (2) users flagged it.

https://news.ycombinator.com/newsfaq.html

this is probably the least flame-y thread about crime in SF I've ever seen on here

Thank you for helping to resolve the situation! Please disregard the email reply I sent as the issue was clearly rectified.

Reply on news.ycombinator.com