My other consideration is that I'm running on CPU and don't love depending on cloud services, so I've also been mostly getting stuff out of a DOM where I can... but it's occurred to me that there are scoped reasons to consider this, like deeper parsing from job descriptions, and I occasionally toy with it on stuff like HN posts. The prompt format here is a lot more thorough than what I've tried, and I might have to go back and experiment with this some more. I haven't gotten great consistency with this yet myself, and I'm not sure how much that's my prompting and how much it's that I'm using smaller (mostly ~7B) models. Which LLMs are you using for this -- ChatGPT throughout, or are there others?
I've been trying to avoid too much prior art while working on mine, but I'm definitely interested in hearing more about what you've been building around this.
[0] bhmt.dev/blog/scraping (warning: this is 10000 words + code samples, as it goes from browser console to browser automation and covers a few different side projects)
[1] github.com/chaosharmonic/escape-rope
[2] github.com/chaosharmonic/escape-rope-ui