355 karma · joined May 24, 2011
The stack which is required for the Streamified API is really quite large, too. Between node, mongo, elasticsearch, and a few other pieces of software... plus a few dozen node modules and even some custom linux configuration, it's really not feasible to expect devs to install it on their own machines ;)
There's a discussion below this one why it makes sense to have a single API for this, instead of open-sourcing the code. The benefits are numerous (we're hooking into many paid APIs and covering the costs, for example).
Would you mind shooting me an email to Zane@Streamified.com so we can discuss?
And yes, you could easily manage many users' RSS feeds :)
We should cover all major news sources. We have at least 250,000 different sources we're scraping, plus we also "crowdsource" news discovery. That is: organically (stream-discovered) content gets indexed as well.
Hope to hear from you! Zane@Streamified.com
For social networks, we're proxying the APIs (essentially). For articles and news, we both aggregate organically discovered articles from streams, as well as scrape 250k+ RSS feeds and other news sources. So, articles are stored local to the server (which is why we have such a large collection of them), but sensitive social information is a pass-thru to the social network.
However, in both cases, we're doing lots of analysis on top of the data per our "normalization" philosophy.
Yes, our searching capacities include everything from categories to sentiment to tags to media... you name it. ElasticSearch powers our backend, which is powered by Lucene, so anything in an article can be searched. Check out the articles/search endpoint: https://streamified.me/developer/#endpoint=GET%20articles%2F...
To your questions: one of the major benefits of the service is the "crowdsourced" aspect of it. That is: the more people using the same API, the more news articles get found and indexed. Furthermore, the API actually hooks into other paid APIs, so it actually wouldn't work very well for you to run it on your own servers unless you wanted to also pay for them. For all these reasons and more, we think that a hosted API is the best solution.
That said, I do acknowledge that WPM is not necessarily the best gauge of efficiency. I'm a programmer, and I'm not sure exactly how WPM relates to my work (I'd be curious to see a study) since much of what I type is "nonsense" words to begin with, and WPM is largely a function of your ability to habitually reproduce the same character sets (aka, common words in your given language).
The original study used only a timer, and had the same frustration which you speak of here.