Tell HN: Check out HNTrends.com | Hacker News Reader

Tell HN: Check out HNTrends.com

hntrends.com

126 pointsmatt155 comments

The idea for HNTrends has been bouncing around in my head for about a year as one of those man-I-wish-I-had-more-free-time-type side projects. I recently decided to take up learning Flash and ActionScript and this seemed like the perfect opportunity to build it.

It's fairly straight forward: A Python script scrapes HackerNews every 15 minutes and saves the data in a MySQL database. A PHP script queries the database and outputs the the last five hours worth headlines as neatly-formatted JSON data, which the chart grabs and plots.

The older, less trendy news items are rendered with a slight transparency in order to reduce the chart's business. You can hover over any of the lines to see it in greater detail. Also, if you click any of the points it'll take you to the comments. The links on the right take you to the actual news articles.

Hope you like it. Let me know if you have any suggestions -- Matt

Any plans of releasing the source code?

Suggestion, if you keep on keeping it up: release a simple API?

What would be most interesting is to extend the list past the page with the first 30 articles. That's what would really give you a chance to deal with HNs front-page feedback loop.

There is lots of stuff that does not make it to the front page during busy times.

Any chance of increasing the length of the list to 60 or maybe even 100 ?

That would really add value.

Very nice. It'd be very cool to have a slider at the bottom that you could use to go back to previous days.

I totally agree. This is mostly meant to be a Minimum Viable Product. The feedback I get here will determine what to add in the future, if anything.

This is pretty fascinating. The most important thing I notice almost immediately is that there are generally two classes of articles:

1. those which reach the front page by virtue of quickly getting a few votes while they are young and then dwindle

2. those which barely reach the front page and have some kind of resurgence (discernible by a large jump in front page rank).

It'd be interesting to group articles based on how they perform like this and see if there are any other correlated factors. My feeling is that the ones with resurgence are either long and require some amount of karma "activation energy" before people are willing to dive into them or that they're ones with long conversation chains which have a spike for similar reasons.

Cute - excellent chart and interaction. Just be careful you don't offend the dons with your scraping.

One criticism: Flash?

Who cares if it's flash or not?

well, I'm browsing on my phone right now, so flash means it's 100% unusable to me.

Blame Apple if you have an iPhone. This sort of presentation also couldn't work on a small phone, though a text list version should be provided for mobile users (is it?).

I hope the author checks out something like Raphael (JS SVG library) and converts it though, because Flash is definitely tacky.

You can visit this site for the text version: http://news.ycombinator.com

Enjoy :)

In all seriousness though: I simply wanted to learn Flash. As an iPhone user myself, I realize its frustrating that it doesn't show up. That being said, even if it did show up you wouldn't be able to make much of it because of the screen size and the sheer business of the chart. It's really meant to be viewed in a normal browser.

No, the other one is sort of a horse race, which is neat. Maybe show the top sites and their ranks over the past five half hours/full hours in a narrow table that would work in any browser.

Try the SkyFire browser if you have a Windows Mobile or Symbian phone

http://www.skyfire.com/

I am, thanks for the link.

[edit] That's absolutely amazing - youtube videos now load faster in my browser than they do in windows media player. Thanks again!

Flash means that it takes up 25% of my CPU simply by having the virtue of being opened in a tab.

Seriously? That sounds odd I'm running the latest version of Flash in Firefox and it is using between 0-6% CPU with 8 tabs open including this Flash chart.

Flash only works well in windows, it's crap on all other operating systems.

you're already looking at HN, it's not like you're really gonna need that 25 for anything else right?

If he's using a laptop, he's now burning significantly more power. 4 hours of battery life at idle can easily become 30 minutes at a constant 25% CPU load.

you're already looking at HN, it's not like you're really gonna need that 25 for anything else right?

Actually, I did need that 25% for something else. Was waiting for code to finish compiling (well, for Clang's Static Analyzer to finish running). It takes more then a few minutes to run on a sizable project.

Plus, as jrockway pointed out, it absolutely kills battery life, and I'm on a laptop.

Not for me, or anyone at my office.

Probably only me - but to my mind that's a shame. It's only my opinion, but I think Flash is nuts for anything new except video at the moment. I think Flash is:

- slow

- fairly unstable, even on Windows

- a privacy nightmare (cookie system)

- being left behind by browser vendors, where there is competition

Excanvas on IE and the canvas tag are really well placed for visualisations like these. I built a donut chart in Canvas recently for something I'm working on, and I'm glad I did, because I was able to do an impromptu demo with someone just the other day on my phone.

I know it's not a critical piece of engineering, and it doesn't actually matter, but I do personally find Flash dulls my enthusiasm for otherwise very nice things!

I asked and was given the thumbs up.

Yeah, I have click to flash installed, so my first reaction was "what, flash?". It would be so nice to see it done with http://www.rgraph.net/ or something similar.

How about a link to the comments after each article title? I usually read those first and then click on the link (otherwise I'd have to come to HN front page anyway to see the comments on it). Works great though - good job.

If you click on a data point it goes to the comments. (I'd personally rather see it as a parenthetical link on the right of the story)

Agreed. I've been using the site all morning, and it's really growing on me. But I didn't see how to get directly to the comments until you pointed it out.

Ah, thanks. Agreed on the parenthetical link or perhaps just a note at the bottom explaining how to reach comments directly.

Yes.

"<Article title name> (comments)" is far easier than hovering to find the graph, then finding a node on the graph on which I can click.

Fullscreen version http://www.hntrends.com/Plotter2.swf

Incidentally, I thought the lines looked blurry... in fullscreen it's clear that they are.

The pop-up only appears for the datapoints - not if your mouse is over a line segment between data points. In fact, it was a while before the any popup come up, for me. Maybe make it attach to the closest data point? (google analytics does it this way).

It's not blur, it's overlapping line segments with some transparency effect. Not great for visualization, I agree.

That's odd - most of the data is fairly linear, but as of right now, there's a very sharp spike for 5 stories at 10:30pm (with a corresponding drop in 5 other stores), which is reverted back to "normal" at 10:45.

Strange voting patterns? Or a bug in your script?

Not a bug as far as I know.

Sometimes the numbers just works out that way.

Why would you plot rank over time instead of actual votes?

Rank shows strictly less information.

Less information, true. However, you can't downvote submissions, so the graph will be rather boring. Just a ton of upwardly sloping lines, some sloping upwards faster than others

You could show the rate of voting instead of number of votes.

I'd guess that wouldn't be much different from the rank.

I think rank is actually more interesting. It shows how much "visible" is an item, i.e. its potential/relative popularity along with its current one.

You could draw a window around the visible items. It would show off relative popularity and rate of change as well.

Nice. I have trouble finding the data points instantly. Not a huge gripe, it's just precarious moving the mouse along the line.

Maybe make circles there? Make a vertical line for each 15min interval where they show up?

I am actually mostly interested in finding a data point quickly in order to get to the comments. Maybe just break out the comment link feature into a "(comments)" link next to the article titles?

Quick update: There was a problem with the script which caused it to go down for a few hours. Should be fixed now. Graph might look a little off for a few hours until the logging catches up.

You should overlay the last N stories that reached #1.

Maybe a little less blur. Cool though.

Very nice.

Can you link to the article's comments instead of the article itself?

If you click on a data point it goes to the comments. (I'd personally rather see it as a parenthetical link on the right of the story)

cool. yet I find it hard to understand why is it so important to see a trend of a link. in other words, who can benefit from this information and how?

It's a great way to see how articles develop 'traction'.

It can be useful in a macro level, that is one might find out the best time in a day (or else, the best day(s) in the week) to post a link in social media site or an article in a blog.

Really nifty! Really fascinating!

Very cool indeed. May be add a link to the comments?

Edit: thanks guys, I must be blind :)

Already there - you just click on a data point in the chart. (The titles on the side take you to the article, which is also the comments for Ask HN / Review my App type things).

Hover over the lines -- click any of the points to go to the comments.

can you please post the source to the FLA? cheers

Reply on news.ycombinator.com