Show HN: Archivy-HN – automatically save and download your upvoted HN posts | Hacker News Reader

Show HN: Archivy-HN – automatically save and download your upvoted HN posts

github.com

144 pointsetherio21 comments

Cool. If you want an smaller alternative, just getting upvoted ids is easy enough in python

    import requests
    from bs4 import BeautifulSoup


    with requests.Session() as session:
        page = 1
        while True:
            resp = session.get(
                f"https://news.ycombinator.com/upvoted?id=ashish01&p={page}",
                cookies={"user": "get this from your browser"},
            )
            tree = BeautifulSoup(resp.text)
            links = [x["href"] for x in tree.select(".subtext .age a")]
            if len(links) == 0:
                break
            for i, link in enumerate(links):
                print(page, i, link.split("=")[1])
            page = page + 1

You could also use the haxor Python library that wraps the HN API:

https://github.com/avinassh/haxor

Interesting. But for me upvoted links are already stored on Hacker News. What I wanted to download / archive is others' comments. Some comments are so valuable that I saved in a personal note (HN Wisdoms)

Is there a way to download all your HN comments?

I'm thinking about doing something like that - I'll implement it because it'd also be quite useful.

Do you think it would be worthwhile to be able to save upvoted / favorited comments also? I also have to think about what context (ie parent / child comments) it would be useful to gather.

Actually, the thing I would be interested in most is a better way to search my comments, my upvoted comments, etc.

HN has a search box, but afaik there's no way to filter by user, or to restrict the search to my upvoted comments, etc.

Downloading would be a good second-best option, because at least it would allow me to "grep" through the archives.

Hmm I see that could be interesting.

The software this script is built around [0] comes with a developing [1] search functionality.

The idea is that using these plugins and the main software one can group / save / search their digital identity - and that's exactly what saving comments would work towards.

[0]: https://archivy.github.io [1]: https://github.com/archivy/archivy/pulls/159

You can restrict "by:user".

I do not see any rate limiting, this could lock people out if they have more then a couple pages of posts.

The API (https://hn.algolia.com/api) being used limits to 10000 requests per hour so I doubt there would be a problem.

Upvoted comments is the one thing I've always wanted (I don't care for the rest). I often upvote things that are useful to refer back to, and without this there's no easy way to find them again.

Why not favorite them?

They're public? I don't necessarily feel a desire to make a public announcement about every comment I think might be interesting or worth referring back to.

Are favorited comments really any more discoverable?

(I use them a lot: https://news.ycombinator.com/favorites?id=akkartik&comments=...)

The username endpoint in the HN API contains ids for all of that user's comments - you could take that and iterate the ids and download them from the API.

It could probably be done with a simple script and you would end up with a folder full of JSON files. Your ISP might hate you, though.

This would be very useful.

I built a node library that does this 6 years ago and I just verified that it still works. It uses the algolia API.

https://github.com/jaredsohn/hnuserdownload

There was also a website which is not working. Going to turn that off now.

All HN comments are available with Google's BigQuery public datasets: https://console.cloud.google.com/marketplace/details/y-combi...

I was going to write a script using haxor to convert my HN submissions and comments to Maildir, didn't get around to it though.

https://github.com/avinassh/haxor

https://github.com/jwilk/hackerlates

oooo meta

Reply on news.ycombinator.com