4,642 karma · joined September 11, 2013
I'm curious about how you worked out the inner dimensions and part placements so accurately. Thank you for providing this marvelous free resource!
My advice would be to first set up an automation for yt-dlp to pull the media, then use Whisper to build a transcription pipeline. Chunk the transcript based on desired result granularity, then store embeddings in local Qdrant. It helps to use an orchestrator to handle all of this - my current recommendation is Dagster (dagster.io).
This one is about Dolton, IL - the series is called "Dolton Documentaries" on YouTube.
I did 90-95% of the work on this with a local GPU. I still ran Claude and Codex, but they were just writing code to orchestrate API calls against local models, local Whisper, etc. Doing the same with the public cloud would have cost thousands of dollars, instead it cost a few hundred dollars for a frontier lab subscription and tens of dollars for electricity.
I recently got into the DGX Spark and just a month ago, got an AMD Ryzen developer platform. Both are incredibly useful tools for some upcoming projects I'm working on (also related to government corruption), but in my limited experience of trying to run opencode on them, the "good models" are still completely unusable because the system prompt alone requires a minute of thinking.
I asked a SOC2 auditor a while back about why they don't just make their own compliance automation dashboard, so we can pre-fill all their specific requests. They mentioned that certain rules are in place to separate evidence preparation from the actual auditing firms. However, after doing some armchair ChatGPT research, it seems like an audit firm could potentially be permitted to provide evidence management software, as long as you're not guaranteeing an audit result or preventing people from taking the evidence elsewhere. Perhaps you could shed light on the legal aspects and AICPA guidance here for providing evidence collection as AI skills to the companies you audit?
I don't know where you get this expectation that people should anticipate that a crawler might try every possible combination of query parameters, thereby missing the cache on each one. Most people consider it a bitter and arrogant perspective, which is why this got downvoted.
When I looked at the logs after getting the billing alert, 99.99% of the requests were to the "/search" endpoint with virtually every permutation of ~10-12 facets in the query parameters. There was only one scraper, but it triggered an enormous amount of network egress since it ended up missing the cache on the majority of queries.
Despite the "download all data" link being prominently placed on the front page, the AI scrapers decided it would be more efficient to download terabytes upon terabytes of raw HTML by paginating through every possible facet on the search endpoint. Even with CloudFront caching results and a fairly efficient backend setup, the monthly bill ended up with around $1k just going toward network ingress/egress, so I shut down the site the following month.
Just take a look at the recent Epic vs. Health Gorilla lawsuit to see how nonexistent the protection is around exchanging your medical records, one of the most sensitive types of PII.