beagledude

it's on the list of things to do, however one of the challenges is that those sites are always changing or updating their templates. So it's nice to have the most up to date links to hit against.

Goose also pulls all the images down as well to inspect them to try and find the most likely main image for the page.

It will probably be split into an online/offline test. If that sounds like something someone wants to help hack on, the more the merrier!

orange county, represent!

list of all the unit tests with the extraction results: https://github.com/jiminoc/goose/blob/master/src/test/java/c...

chartbeat?

Vertica is legit, we tested it against a billion rows on a single server and it didn't blink and eye. It reads compressed data, and optimizes compression based on the column's data type for that ec2 price you could get a few years of vertica licenses

I've been standing for about 3 hours right now for the first time ever since reading this article and I haven't skipped a beat actually. It's been a nice change.

say what you want about Facebook but I give them props for open sourcing so much code to the community. Cassandra, Thrift, Scribe,Hive, etc...

The article was almost unreadable, agreed

I listen to podcasts 2 hours a day on my drive to/from work. Looking forward to you expanding on the site, keyword searches, tags, descriptions would be nice

Well that didn't last long

Goose article extractor has a full suite of unit tests and also does pure text and image extractions: https://github.com/jiminoc/goose

I've watched a few of these, that professor is awesome. He teaches in a very positive way that's clearly understandable. He's the best one I've found on itunes U

I've worked at the library quite a few times until I found a co-working space.

+1 on the distraction comment.

hope you get better Andrew

that's a good point

depends on concurrent connections vs spread out, cpu, traffic of other customers on your box, apache configurations, etc...

It's just good practice to cache when you can. There is no reason to ask the database for the same article 1000 times a second if it's not changing. I didn't feel the server was going to crash at that moment but I did notice MySQL was doing plenty of work it didn't need to.

The concern was if it got picked up by another site like digg or reddit then it could have been more of an issue.

Looks like I have the Linode 512 plan, so far so good :)

I'm just amazed AOL still has money

I'm hoping open sourcing these stats from yesterday's article will help someone out there. I've learned a ton from reading HN so here's something back.

uhhh those prices have to be a joke? I'm running several 16 GIG redis instances, I'd be in the poor house. haha

100 mb for $25??? ha

based on the open source Goose article extractor from Gravity Labs: https://github.com/jiminoc/goose

curl -i -H "Accept: application/json" -X POST -d '{"uri": "http://threecrickets.com/prudence/rest/} http://prudence.textmethod.com:8080/proose/page/

WikiPage: https://github.com/jiminoc/goose/wiki

I'd be interested in the Debian one if it had useful info.

Redis is one of the best pieces of software I've used in the past few years. I use it so many ways in production it's loco. Having flexible data structures besides (string)k, (string)v is a huge boost.

I was more talking about someone posting a thread on HN earlier in the week that said someone was going to post something to fuck with HN users to get their personal info.

is this the hacker target the guy earlier in the week was talking about? put in your gmail creds, they get swiped up and you lose your gmail account? This would be the exact thing that would get lots of hacker news accts :)

sometimes that's all you need, broseph

+1 I've used this for a few projects already :)

I'd love to see some more articles on the subject out there, definitely take the time to post something. Entity or sentiment would be my first choices :)

It was actually a method of using wikipedia to build your corpus for any ambiguous word to automatically build some word sense disambiguation in your application. One word was just a simple example of using that data.