Hacker News Reader

Comment by _lpa_ | Hacker News Reader

I did something pretty similar over christmas, though I used named entity recognition to extract book titles rather than looking for amazon links, and (so far) also limited it to specific "Ask HN" threads about books. You can find it here: http://www.hnreads.com/. It is interesting to see how little overlap there is between the two, though that may be due to my using far fewer (and also newer) threads!

View on HN

Surprised to see Permutation City in that list. Given that the book is written in 1994, Gregg displays admirable prescience about how computing would develop. Honestly you would think it was written in the last 5 years or so. His vision of cloud computing is absolutely outstanding. It blew me away when I checked when the book was written after reading the first few chapters.

I'd read Schild's Ladder prior to reading Permutation city, which is also a good read. It does seem to get bogged down in the technical and descriptive side of things at times, however, it's a fantastic idea for a story. The main premise of the film would make a great movie.

Whilst I'm on the subject of good "Hard sci-fi" novels, Tau Zero is also worth reading.

Edit - I'll also throw this in: http://www.amazon.co.uk/gp/product/0814703259

Magic :-)

If you like Permutation City, you might also enjoy another of Egan's works, Diaspora. http://www.amazon.com/Diaspora-Novel-Greg-Egan/dp/1597805424...

"Since the Introdus in the twenty-first century, humanity has reconfigured itself drastically. Most chose immortality, joining the polises to become conscious software. Others opted for gleisners: disposable, renewable robotic bodies that remain in contact with the physical world of force and friction. Many of these have left the solar system forever in fusion-drive starships.

And there are the holdouts: the fleshers left behind in the muck and jungle of Earth—some devolved into dream apes, others cavorting in the seas or the air—while the statics and bridgers try to shape out a roughly human destiny."

Egan's books have been some of the most thought-provoking I've ever read as far as science fiction technology. A lot of the works were out of print until recently; I'm glad to see there's been a resurgence of interest in his writing, and the availability of his works.

Another Egan book well worth reading is Schild's Ladder, which reminds me a lot of Diaspora and was recently reissued in the US:

http://www.amazon.com/Schilds-Ladder-Novel-Greg-Egan/dp/1597...

Those are all great, in fact I'd recommend reading pretty much all of his books. I've got a shelf full of them. :)

Cloud computing isn't really anything new. Back in the day they just called them "servers". And clients were "dumb terminals".

Read the Orthogonal trilogy. I think it may be Egan's best work.

Although it's more like Schild's Ladder than Permutation City, I suppose. (Physics ideas rather than computation.)

Many thanks for sharing this. One usability nitpick though: please consider fixing links so that they could be opened in new tabs/windows.

Opened 7 new tabs from the interesting books to me before I realized this didn't work too. It seems the URLs are not quite "pages" -- opening one and refreshing it redirects home as well.

Nonetheless, this is an awesome project.

Yeah this is my mistake - sorry :(. I'll fix it as soon as I have time.

If it's not too much bother, any chance you'd be willing to put this on a git repository, like GitHub or Gitlab? It would be cool to be able to contribute to: for instance, being able to filter the list to fiction and non-fiction would be pretty great :-)

I might be a bit embarrassed with the code quality - this was one of my first attempts at front end development. I was planning to expand the features of the site a bit, including genre filters, so perhaps I could clean it up a bit and release it when I've done that.

Oh, I wouldn't worry about that :-) if you are concerned, just say it's rough and ready in the readme. I doubt anyone will be critical, and if they are then tell them to piss off!

The beauty of open code is others can help improve it

Interesting that your list yields mainly fiction, while OP's list is mainly non-fiction.

One thing that struct me about your site (apart from being a good list, well done!) is how blazingly fast (close to HN, which I find funny) the page loads. Could you fill us mere mortals in on how the fuck you got it so fast?

Looks like fastly:

    $ host www.hnreads.com
    www.hnreads.com is an alias for global-nossl.fastly.net.
    global-nossl.fastly.net has address 23.235.44.204

it's a simple js view for a 1.4mb json file, so "you got it so fast" really just depends on how quickly you will download that 1.4mb file.

I mean the initial http response, not even the speed of the subsequent ajax.

That's not really much to do with me - the credit there lies with fastly! This was the first time I've used a CDN, and I had a great experience using them. Highly recommended.

It looks like it loads all the data on the initial page load. After that, it's a SPA app and all the links are just drawing local data.

This is really cool. I started reading HPMoR after finding it.

I'm curious: after performing NET on the corpus, how did you filter to find books only? Did you just search on Amazon's catalogue for exact matches, or was more tweaking required?

I built a little tool to help with that, but pretty much the first result on amazon was always right - for books with more than one match, at least.

Thanks. How did you deal with, e.g. 'Steve Jobs'? I guess a generic NER tool would extract that as a single entity. Then you'd search Amazon and find an exact title match (the biography by Isaacson). But you probably don't want this in your list, as the references were to Steve Jobs the person, not Steve Jobs the book.

So did you use some manual review step to remove these?

Well, the nature of NER is that it can extract 'Steve Jobs' as either a PERSON or a BOOK entity (in my case). You can in principal extract whatever kind of entity you want, provided you have appropriate training data. The idea (loosely) is that it learns the structure of text around, say, a book. You can then label a piece of text as a book, despite not previously knowing that title. It doesn't solve the problem of two books having identical titles, however :).

(Note: I'm not an expert in NER/NLP, so please correct me if I have got something wrong!)

Aha, I see. Thanks. I thought you were using some generic pre-trained NER, but I understand now that you used either one which was pre-trained to find books, or you trained your own.

As a non-programmer I find your list more interesting.

On the system browser on my phone (a Kyocera Rise running Android 4.0.4) all I see on that page is the header and footer, no content. :-( I get the same result if I hit it with Firefox with Javascript disabled, but Javascript is very much enabled on my phone browser.

BlackBerry Passport with Javascript enabled here, I get the same thing.

Damn, sorry about that. Do you see a loading spinner on the page at all? I don't have an Android or BlackBerry, so wasn't able to test with either.

Nope, nothing.

Cool project! Do you mind telling which library or service you used for the named entity recognition?

I actually wrote it myself, based on conditional random fields! I would not recommend doing this - it was mostly an exercise to learn how they work. I should probably have mentioned that I manually cleaned up the results a bit. I understand nltk has NER capabilities and python bindings, though.

Thanks for your reply! Going to look into the NER capabilities of NLTK.

Thanks for sharing! This list is a list I actually expected the OP's list to be. Probably because I'm also more likely to view the Ask HN threads about books.

It would be great if you or OP could do the analysis again on HN Reads, but with the larger dataset.

Good result. Like a curated list, but automated. I saved here for my reading list.

Reply on news.ycombinator.com