Hacker News Reader

Comment by hehheh | Hacker News Reader

Neat. Any chance of adding JSON processing where you can pick a key (ala `jq`[0]) and then combine it with RSS or other JSON or what have you?

[0] https://stedolan.github.io/jq/

View on HN

I've made something[1] that used "jq for everything" in the past.

RequestHub was initially a way to connect webhooks from one service to API calls to another services, using jq scripting for all the customization.

Later I realized it could be used for processing anything (even text!), not only webhooks, but it was too late. Maybe someday I'll do a Yahoo Pipes revival that will just use jq for scripting inside the boxes.

[1]: http://archive.is/nGyH3, https://github.com/fiatjaf/requesthub.xyz

Why did you discontinue requesthub?

I couldn't imagine a way to monetize it, and it was using all my free Heroku hours.

What do you think? Do you think it had a bright future?

I think Zapier and the likes fill this need pretty well. I wonder if there's still a niche for more lower-level hosted service for plumbing webhooks though. It might be a rather small niche unless you figure out a way to offer better value than those guys.

One thing that definitely makes it hard is the chicken&egg situation. I can't imagine many people paying for a service (or even using it for free), unless it has a solid reputation and is highly available. Otherwise, they'll either host something on their own, or use one of the bigger players.

If I understand you correctly, you want to be able to join the results of multiple sources? You can do something like that with a library I wrote, riko [1]. A simple example would look something like this:

    from riko.modules import join, fetch, fetchdata

    rss_url = 'http://site.com/rss'
    json_url = 'http://site.com/json'
    json_path = 'path.to.data'

    fetch_conf = {'url': json_url, 'path': json_path}

    rss = fetch.pipe(conf={'url': rss_url})
    json = fetchdata.pipe(conf=fetch_conf)

    joined = join.pipe(rss, other=json)
    next(joined)

You can see the docs for the `join` pipe here [2].

[1] https://github.com/nerevu/riko

[2] https://github.com/nerevu/riko/blob/master/riko/modules/join...

I plan on enabling filtering via xpath, and in parts jq is like xpath for json. But I'm not sure whether you mean something more with "then combine it with RSS or other JSON or what have you"? Merging generic input into a feed at a chosen element?

Side remark: Pipes uses https://github.com/feedparser/feedparser to normalize atom and rss feeds, and that gem just got support for the jsonfeed format. Untested, but worth a shot if you are looking for json input.

Should you feel like, there's also JMESPath for JSON http://jmespath.org

Reply on news.ycombinator.com