josh-pdap

https://raheem.ai/ is an interesting project. One idea someone in one of their community calls had was scraping dispatch data to figure out where social services might be useful, and they're creating a dispatch app which lets people ask for non-police help.

I think if a project like ours ever did use blockchain, it would be behind the scenes as a part of the product; decentralization and transparency are key parts of our ethos! However, it's too unstable and as you can see by the comments, mentioning "web3" costs a lot of credibility in many, or maybe most, communities online.

https://knish.io/ is a good example of people using blockchain to make real products. I don't know that it's ready though.

Hello! I'm the executive director. I have a design background, have done product management in the past, and aside from keeping the lights on at PDAP and making sure we're tax-compliant I am in a product role. I talk to people using police data, and figure out where we can add value to make the data more accessible.

TL:DR; If you want to write scrapers: go for it! Run your scraper, share the results in Discord and with your friends, and talk about the process. We'll be listening, and it will help us build tools to support this important work.

A few things to clarify:

a. The source of truth for "what are we doing right now" and "how can I contribute" is https://docs.pdap.io/.

b. Empowering people who write scrapers is a part of our broad mission of "police data accessibility", but we have some foundational work to do first! Right now our primary project is creating a database of police agencies and data sources. This will help people understand what kinds of data are available, at which agencies, with which steps to access it. It will also help us create archives of the primary sources, so that if they get taken offline we can still go back and scrape them.

c. What we have realized in the past few years: there are already a ton of people writing and using web scrapers for their day to day work. They are as decentralized as our police system. Our scrapers repo will reflect that. We shouldn't all rely on one library, or even one language. The people who need the data are most motivated to maintain scrapers, and we expect that maintenance will be ad-hoc and as-needed for the immediate future. In most cases, data already published on the internet is useful to local users as-is.

d. If you have a question you'd like to answer about the police, here's the investigation process:

1. Determine whether public data exists to answer your question. Use google to find the appropriate agency, and see what they're publishing. 2. Determine how it can be accessed; do you need to make a FOIA request? Is there a URL? 3. If there's a URL, determine whether you need to write a scraper to access the records. Often, the records can simply be downloaded. 4. Write and run a scraper, if you need one! 5. If there's not a URL, make a records request for the public information. This is a long and complicated process. 6. Share the data with your friends.

This means that scrapers are helpful and necessary some of the time; but not always, and not as the first step. We're trying to help with steps 1, 2, 3, 5, and 6. The theory is that writing scrapers is something people can easily slot in and help with; and that, depending on what question you're trying to answer, two scrapers for the same data source might look wildly different.

Scrapers are an important part of the ecosystem, but they're one piece of the puzzle.

This was an early experiment, we don't have a published app right now.

Yep! It's really helpful to see where people find problems and want to jump in.

That's one of the many reasons why we abandoned the crypto idea after some initial optimism and research.

Thanks for taking a look! DM me on Discord if you'd like to chat about this.

The post you quoted lists potential avenues for using web3 tools. You may notice we have not gone down any of those avenues. We aren't doing crypto stuff.

We did indeed spend some time looking into web3 and peer-to-peer, and ultimately decided it's not for us. Most of the energy there is spent trying to make money / scam people. Peer-to-peer is cool too but has its own risks.

Thanks for checking out the discord, though!