Ask HN: Review My Doohicky | Hacker News Reader

Ask HN: Review My Doohicky

19 pointsajkirwin13 comments

I got bored this weekend and decided I wanted to data mine twitter and over the course of a day and a half, I made this:

http://malicelabs.com/linkalyzer/

and I would like suggestions from people as to how to improve it and what they think of it. :)

Edit: Intermittently broke things and blew away some data intentionally. Add(ing|ed) in tinyurl/is.gd/etc through-finding of real URLs and page title grabbing.

View on HN

Would be good to expand the tinyurls or at least provide the domain they are pointing too. This may even help you consolidate tinyurls if multiple services are used for the same original url. In Python I sometimes do the following to get the finalurl (not fool-proof):

  finalUrl = urllib.urlopen("http://www.tinyurl.com/5w5wod").geturl()

I added this. A bunch of people demanded it. :)

As it stands, it has a few things that twitturly doesn't have, but also they have things I don't.

But, it's a beta product and hopefully soon, I can get a slicehost/linode server and get it off of dreamhost and I can spruce it up and add more features.

Maybe try and reverse engineer the TinyURLs so that users can see where they link to?

This. Or maybe find a way to grab the page <title> from the target URL and create a separate column with the appropriate information.

Otherwise, nifty little app.

Nice project. It would be interesting to see maybe a "popularity tree". Like for example, it would be fun to see if Scoble or Kawasaki put out the link first and see how it was distributed. Or if someone unknown sent out the link and someone popular picked it up. Not sure if its possible to trace though.

Also, I just started a topic on a data mining related note. If you have any thoughts, I'd love to read them. http://news.ycombinator.com/item?id=406705

Nice little project, good for learning the API.

You may want to check out http://twitturly.com/ if you haven't yet.

view more information gives an warning.

Warning: mysql_fetch_array(): supplied argument is not a valid MySQL result resource in /home/.dodi/jamesgarrett/malicelabs.com/linkalyzer/get_urlstats3.php on line 7

more warnings follow

Also getting various MySQL warnings.

That was my bad! I added some fields to a table and forgot that'd crap all over my UNION.

Always always always turn off error display_errors on a production server :)

Also, every image in your TwitPic section appears once as a png and again as a jpg. Some of these are broken but they still link back to the same TwitPic page.

Twitpic Hot 30 gives is missing about half of the images. There's spaces for them, but they just show the alt text.

Neat analyzer! I think there is so much data that can be gathered from Twitter that we have no idea exists.

Reply on news.ycombinator.com