Open Sourced: Newscombinator Front End | Hacker News Reader

Open Sourced: Newscombinator Front End

github.com

71 pointstomw180825 comments

What happens when you are on the HN top site (beside you open a beer and enjoy that little tiny moment after months of hard work):

a live snapshot http://i.imgur.com/0KbPjcx.png

Thank you everyone. Hard to describe the feelings, somewhere between moonwalk and break-dance.

another snapshot: http://i.imgur.com/CBYRvV3.png

server load peaking @2.00 seems to be stable

After a short break, still on the HN/top. Seems like america is sleeping now and europe is not yet quite awake :)

another snapshot: http://i.imgur.com/LcgXdFy.png

still everything running stable.

Cool, I like it. One request: Start all stories collapsed. Right now, the Ellen Pao story is at the top, with about two dozen related headlines beneath it - that takes up a lot of screen space (every time I refresh I have to collapse that space).

It would also be good to be able to collapse/expand by clicking on the headline count.

I'm not sure how I feel about the continuous scroll. Often I like that, but here I am unsure, but it may be because of my particular use case: I use LinkVisitor to mark all links as visited, so that next time I know which I've already read or decided to ignore. Having a fixed number of headlines makes it easier for me to recognize that it is time to hit cmd-fn-f2. If others get a lot of value from the continuous scroll, don't worry about it, mine is likely quite the edge case.

Hey there, thanks for the feedback.

Its kind of a funny thing here with that Ellen Pao case. I didn't see similar stories for quite a while on the very top and exactly now there is one. Shouldn't be an excuse, rather something I literally just learned about my app. And if everything is collapsed at the very beginning, it also looks kind of odd (for me at least).

Thanks for the great feedback. Considering its an angular app, it shouldn't be that hard to have a switch to choose between endless scroll / normal pagination. But its a long way to go. I'll put it on the list :).

Seems really cool, it gets the great honour of being a pinned tab in my browser (ńext to HN).

I think a little icon that indicates the source next to the title (on the left) would be really useful to quickly scan what media ranks where and maybe only select the top stuff from your most trusted source etc.

That is, indeed, a great honor. I didn't expect my Saturday ending at home, monitoring the servers - in a good way.

The icon is a great idea, now the problem: What happens if a link was on 8 sites (2 x reddit, 4 x ycombinator, datatau, makernews). What would you expect next to the title?

I also have been exploring what it may mean to automagically categorize news content.. I took an approach on http://www.rivyr.com of following RSS feeds.. then reverse searching the titles against my product database to attempt to autoclassify news by product. Wish I could see your backend as that's where the secret sauce is but definitely understand not doing that ;)

That is super interesting. I'm actually working more closely with Apache OpenNLP for feature extraction and LibLINEAR to classify the news according to their category. Its not too far from coming out of alpha to an open beta version. Why not subscribe to the mailinglist and I'll shoot you more infos over in the upcoming weeks?

This was something i was planning to build soon for myself. Great job on the UI.One thing kinda bugged me was when you select the source from the sidepanel it collapses for each selection not allowing to select another source without collapsing. I am on OSX Yosemite/Chrome-Version 41.0.2272.104. Also is the backend Elasticsearch? Is the backend elasticsearch?

Hi. First: thank you for the kind words. Second: I'm going to fix that, I know, it already annoyed me today and I just implemented the collapsed sources last week for better readability.

The backend is Solr in combination with a custom scraper in java and a lamp stack. The REST interface is wrapped around a php for better auth handling - and mainly because I'm a big fan of the Zend Framework and Pimcore [1] - one of the best CMS I've come across in my life (I'm not working for them).

Although I cannot give you access to the full source of the scraper/backend (at the moment), why not get in touch and/or follow on twitter/mailinglist. I'm going to send out updates what problems I was struggling on and how I solved it from time to time.

[1] https://www.pimcore.org/

The related stories feature's cool. I made my own tool for reading HN (and some other sites) it's kind of different as it pulls the stories into a linear feed. I'll need to have a go at adding related stories. My site's http://www.serializer.io if you're interested.

I saw it today in the HN Notifications Thread (I think) and was already impressed, its great work, imho.

Cheers! Nice to know I'm not the only one that thinks it's cool.

Btw, your "Btw, JSON anyone?" link is broken:

    {
      "error": {
        "msg": "undefined field: \"log\"",
        "code": 400
      }
    }

Otherwise, great work! Looks very nice!

Interesting, as it works normally for me. Which browser you're on, if I may ask?

It should just open a new tab with the same request as sent to the REST interface anyways in the background...

Like that one http://goo.gl/eLiUT1

this is interesting - could you talk a bit about your solr schema ? In a very naive way I would be simply storing it all as a indexable text blob. Did you do anything more, just so that you can avoid a database ?

Sure. Well, basically Solr is just a flat index, although you can store now mutliple-values in one field and even make joins. Here is the clou: Everything is preprocessed and stored in a mysql-db and solr just "caches" everything. Its some kind of the flattened mysql schema. So I have a combination of a relational db and all the benefits with the huge bonus of the speed and reliability of solr in between. I started of with just a MySQL db, but real-time scoring is really a pain in the "back" with mysql and I couldn't figure a nice way to do it. On the other hand, if you expand a story, you see that the comments/upvotes from the sources are fetched, because joins like that are no possible in solr (read: I couldn't find a way to do it).

So, all the information is preprocessed at index time and solr is basically a text-blob spread across multiple fields, yes.

Cool, thanks a lot :)

why is it written in angular?

why not?

overt complexity for no real gains

Let me comment more on my short "why not" - here is why yes:

First off: I don't choose a language because its 2014, 1999 or 1985. Or because its hip, cool or just "in". I choose a language for me, because it can help me achieve what I want to do in a reasonable amount of time. If the outcome is great, even the language is shit, its still great outcome. If the outcome is aweful, no matter how great the language is, the outcome is still not good. And I think the outcome is okay in this case and the framework is in my opinion pretty cool (considering I started JS first on IE5.5), so I cannot be blamed for being happy with a framework like this :).

And, based on my opinion, I choose angular, because I have learned it first and to love it last year. Great community and good documentation imho. I think its the wrong move to change the unterlying API with the new verison. If that really happens, then I maybe switch, learn something new, better - maybe. React is awesome, plain Javascript too imho.

Thoughts?

I'm really intrigued. Why not Angular?

Angular is so 2014... now everything should be React based.

Reply on news.ycombinator.com