Show HN: Pushpin – A new way to build realtime HTTP and WebSocket services | Hacker News Reader

Show HN: Pushpin – A new way to build realtime HTTP and WebSocket services

pushpin.org

114 points650REDHAIR29 comments

Great idea, but all the marketing speak on the landing page just didn't make sense to me. Things immediately fell into places when I saw the diagram in the"About" page.

Pushpin is a proxy server that adds websockets to existing request-response apis. Why don't you just say that straight away?

Thanks for the feedback.

One thing I've observed over time (the website is new, but the project is not) is that Pushpin's value isn't immediately obvious from a one-sentence description. So instead of starting out by saying what it does, I often like to start out by discussing the problems that it is trying to solve.

But yeah, you're right. I, too, prefer when projects plainly say what they do. I'll see about reworking the text.

Yea. Front page looks nice, but does not explain to me in the terms i understand (technical terms) what it does. Assuming your target audience are developers/CTOs/etc I would re-think what text you have on your homepage. Possibly even get some kind of diagram on the front page.

What is the intention of licensing the core server under AGPL?

Does the REST API represent enough of a boundary behind the Pushpin server to separate the web application from the AGPL's requirement to give the source code to public users?

Edit: Are there any widely-used proxies this tightly coupled with applications that are released under a similar license? (Translation: is this normal?)

Yes, the intent is that the Pushpin<->backend interface is enough of a boundary that you don't need to provide the source code of your backend applications.

The AGPL is used to protect Pushpin itself. Open sourcing it all was a big step for us. :)

I saw a presentation on this last month, and found myself wanting to invent a problem just so that I could solve it with this type of scaling. Great presentation and great product.

Can you provide a real world example of an interaction, and how it would be improved with pushpin in front of it?

Sure. For example, we made a tool called WebhookInbox that collects HTTP requests. It has a realtime API that lets you know when requests have arrived:

http://webhookinbox.com/docs/api.html#live-updates

The above API has the quality you'd expect from a long-polling or streaming API produced by companies like Dropbox or Twitter, except that the backend is just a basic Django app. There's no crazy stateful event-driven code in WebhookInbox.

So basically it makes realtime API development like this much easier.

Trying not to offensive. Two years ago I would find this useful. But I'm so glad I learned Node.js where this can be accomplished with high level sockjs module and nginx in less then five minutes.

https://github.com/sockjs

No worries! Pushpin isn't really about providing the easiest answer. It's about radically changing how we architect and operate realtime web services. In fact, Pushpin and Node are complementary, and Pushpin has a Node library.

If you're already using a language/framework capable of WebSockets or long-lived HTTP connections like Node (or Tornado, or Go, or countless others) then Pushpin's value may not be immediately apparent. It just seems like yet-another-realtime-server thing, right?

The long answer is that Pushpin restores the division between API designer, engineer, and operations that we all enjoy in the RESTful API world but that is currently non-existent in the realtime world. But, until you have many servers, a multi-tiered architecture, and a team, this may be hard to appreciate. ;)

Hey folks. The idea with Pushpin is to make realtime APIs easier to create. Think of it like Socket.io, but aimed at API creators instead of application developers.

Most realtime APIs today are implemented using low-level network code, because until now that was the only sophisticated way to do it.

I found Pushpin interesting when I heard about it a couple years ago. Haven't had the chance to use it yet, but I'm curious if you could give a summary of what's changed.

Aside from a host of fixes/refactorings, the biggest change is that it now supports WebSockets, with either a WebSocket origin server or HTTP origin server.

Also, the backend protocol (GRIP) was refined to work via HTTP headers instead of JSON instructions in the response body. This change helps simplify backend code and makes it more readable. The response body mechanism is still supported though, since it's more extensible and may be needed in rare cases (such as status code 304 with Apache).

I absolutely love the API design here - running a realtime layer as a proxy in front of a traditional request/response webapp (which controls the realtime proxy through custom HTTP headers) is really elegant.

This is fantastic! Thank you :)

For my use case, each client needs a separate channel which is short lived (they are generating a report which is unique and takes some time). Would pushpin have any problems with such a setup?

Shouldn't be a problem with Pushpin. There's no major resource overhead in having lots of channels, and since channels exist on demand there's no extra administrative work either. Just assert unique channels at hold time, and publish to those unique channels.

I notice that you can have Pushpin talk directly via ZMQ.

I would love to see an example ZeroMQ-based Ruby "web server" that can be used a backend for Pushpin to avoid the HTTP stack on the application layer.

The communication protocol is called "ZHTTP" which works in either REQ/REP or PUSH/SUB modes. It basically wraps HTTP in ZMQ messages. We submitted a draft here: http://rfc.zeromq.org/spec:33

I'm not much of a Ruby developer, but I just committed a Python example using the REQ/REP interface. It should be readable enough so you can get an idea of what's involved: https://github.com/fanout/pushpin/blob/master/tools/zhttpreq...

To activate the realtime stuff, you send the same Grip-* headers as you would with normal HTTP.

A word of caution: while this path is more optimized, it is super bare bones. As soon as you start doing anything advanced, you may find yourself wishing you were using a real web framework (I know, because this happened to me once already! haha).

Thanks for this, probably enough to get started.

The plan was to create a Rack-compatible injection point that uses ZMQ instead of HTTP which could be used with any of the standard Ruby frameworks, rather than getting too bare-bones.

I'm not sure whether there are any real wins here though.

Although my real use case is to take queries from Pushpin and put them directly into a persistent queue for processing by workers before them pushing the responses directly back at Pushpin so that we can do a zero-downtime deploy, even if the workers are down for up to 55 seconds (with long-polling).

https://github.com/rack-amqp 's Jackalope might be of interest to you. It uses AMPQ rather than ZMQ though.

Looks interesting. May be a good starting point, although it's ultimately Pushpin that makes this model interesting to me.

I'm curious as to how much Pushpin is part of the Fanout.io stack?

Is Fanout.io just a (admittedly very affordable) hosted version of Pushpin?

Yup, Fanout.io is basically a hosted Pushpin service, among other things (like Webhooks, Bayeux, etc).

Meta question: is the use of "realtime" for this kind of thing already accepted or is there some discussion about it? I ask because the other meaning[1] is more natural for me and once in a while I get confused about which meaning is intended in articles' headlines.

[1] http://en.wikipedia.org/wiki/Real-time_computing

For anything web development it's generally acceptable to use the term realtime/real-time, but it's really "near real-time".

Whoever is responsible for your site's design has done an excellent job.

That's pretty exciting. Good to see new players in this space :)

Is this similar to Faye?

Sort of. Faye and Pushpin are both publish-subscribe servers. The difference is that Faye speaks the Bayeux protocol with clients, whereas Pushpin is client-protocol-agnostic.

Use Faye for simple JSON messaging to applications. Use Pushpin for developing APIs.

Reply on news.ycombinator.com