Hacker News Reader

Comment by akavlie | Hacker News Reader

One of the Stremor devs here. I'm helping to build Liquid Helium, and wrote the cited README. Never expected that it would make its way to a public blog post :-).

Yes, language processing is hard. There are two challenges here:

1) Understanding the ambiguities of language, when every word in the sentence can be 2+ parts of speech.

2) Making it fast, and making it fit within the relatively constrained RAM limits of App Engine instances.

We're wrestling with both while we greatly expand on what Liquid Helium can do. It's not easy, but some of the things we're able to do with it are pretty magical.

View on HN

To akavlie's Ram limit comment: We run on Google AppEngine. It gives us great scalability, but it limits us to about 512 Megs of ram.

We made this choice because with a small team it gave us the most freedom to focus on the code, rather than the scalability and the infrastructure. We don't manage servers, or routers, or load balancers.

Well, technically we are now allowed a limit of 1GB per instance, as of a couple of months ago:

http://googleappengine.blogspot.com/2013/02/app-engine-175-r...

I was going to ask about that, seemed strange to feel that a self imposed constraint was a challenge you had to work around.

There are other options outside of app engine that could help you achieve the same thing. I outsource most of my processing to picloud (specifically because I need more ram than I have).

I don't work in your field but I think it's a really interesting problem you're working on.

If you are going to compete with Google you might as well step in to the arena running on the same hardware. :-)

Also the best targets for acquisition put Google high on the list so building things the way they would makes sense.

To the ambiguity comment:

Can is really annoying. Can is a verb. Can is a Noun. It is part of a Noun Adjunct.

The rules to determine "The chicken soup can help when you are sick" and "The chicken soup can landed on my foot" and "We can peaches to eat later" is very hard. More so if you put them in a complex compound sentence.

Reply on news.ycombinator.com