Bridge is a new RPC framework for building modular services
getbridge.com
getbridge.com
If you are interested in some of the existing discussion, check out (They're PDFs. Sorry.):
* [1988] A Critique of the Remote Procedure Call Paradigm http://www.cs.vu.nl/~ast/publications/euteco-1988.pdf
* [1994] A note on distributed computing. http://research.sun.com/techrep/1994/abstract-29.html
* [2005] RPC under fire. Steve Vinoski. http://www.iona.com/hyplan/vinoski/pdfs/IEEE-RPC_Under_Fire....
RPC is widely used. It is mature and well understood. "RPC is bad because network calls look like local calls" is a terrible overused argument.
That said, most of the faults people have placed on RPC have been products of individual implementations and not the concept itself, and there really is much more momentum behind RPC engines than anything else. For my Master's thesis, I implemented a generative communication [1] engine. While I still feel like it's a "beautiful" model, it turns out no one cares. Developers understand RPC calls. Not that many get tuples / tuplespaces / pattern matching and why that can be nice.
1: http://theory.sci.univr.it/papers/Isa-orig/Varie/Linda2.pdf
Message queues are basically the modern tuple spaces, and I'll bet all the companies listed above use message queues too.
RPC has its place but it's overused. It's a low level primitive, not a distributed systems architecture. Some people's architecture is basically just RPC spaghetti, with a naive handling of errors.
The argument is confused by the fact that you can use an RPC with messaging semantics (async, a single argument data payload), and you can use a messaging system with RPC semantics (i.e. tunneling RPC over HTTP, using HTTP as transport).
I think we basically have to stop using the word RPC as a catch-all for so many systems, and identify which choices are good and which ones lead to design mistakes.
Client:
out("rpc-request", uuid, "say_hello", "m0th")
in("rpc-request", uuid, ?response)
in("rpc-request", ?uuid, ?method, ?arg)
response = call(method, arg)
out("rpc-request", uuid, response)
Message queues and tuplespaces are very different. A tuplespace allows pattern matching on tuples. And many tuplespace implementations do not provide monotonicity.
RabbitMQ supports reciving messages based on a pattern:
http://www.rabbitmq.com/tutorials/tutorial-five-python.html
It also says it supports RPC:
http://www.rabbitmq.com/tutorials/tutorial-six-python.html
That doesn't match my definition of RPC, but OK. I consider RPC server-to-server, not server -> "smart" intermediary -> server. But I guess they want to to define RPC as request/reply. So again it's a terminology issue.
What deployed, modern systems use tuple spaces and are substantively different from a message queue? My impression is that "tuple spaces" were the historical name, from Gelertner's papers, but everyone just calls them message queues. Message queues have lots of different properties but the defining one, as in tuple spaces, is that there's an intermediary between 2 servers. The sender and receiver don't have to be up at the same time.
None that I know of, but that's because no one uses generative communication nowadays :)
> Message queues have lots of different properties but the defining one, as in tuple spaces, is that there's an intermediary between 2 servers.
That's a very loose definition that could classify a lot of things as message queues. Generative communication says nothing about how the semantics are implemented; they may use an intermediary, or they may not. The early version of C-Linda did not use intermediaries, and was intended for in-process communication. Of course, a practical, distributed implementation of generative communication uses intermediaries. But then again, depending on the implementation, it may be distributed across multiple intermediaries.
I would say a message queue would necessarily have monotonicity (the name surely implies it!), which generative communication does not have.
> The sender and receiver don't have to be up at the same time.
In academic parlance this is called time decoupling, and it is an awesome property. ZeroRPC has this as well thanks to 0mq.
I don't really know anything about RabbitMQ to comment. I think I've heard that some engineers at dotCloud tried it out for our distributed communication needs, and it just couldn't scale to our load. But that would've been a while ago, and it has probably improved substantially since.
If your only concept is RPC, that's just too low level and limited, and you're going to end up with a mess. You're right that "developers understand RPC" (in a naive way). But that's just because we are still in the early stages and knowledge hasn't had time to propagate. Some RPC systems have more of the naive properties that the OP was pointing to; some have learned from those mistakes.
Perhaps it was because of how XMLRPC and related nonsense poisoned the well.
Times are different now. Developers are embracing asynchronous methods on platforms that used to be strictly synchronous and they're learning to adapt to the fact that your method calls take time to complete, or may not complete at all if you're not careful in your design.
jQuery and related methods are RPC in a very primitive sense. Backbone is a step towards a more Meteor-like paradigm that should serve as a more solid foundation for "modern" RPC.
[Edit: Oh, and tuple-spaces are cool.]
From a naive perspective of someone greater than 25, bridge looks interesting. I would expect some (not all) others who have "been there" would feel the same. I don't think the hyperbolic condescending attitude is really called for. I apologize if I'm inferring incorrectly the tone of your post.
EDIT: corrected a typo
RPC is not a "leaky abstraction"; it is semantically invalid. It makes network messaging look like function calls of some style. But they aren't. (At some point, you will find yourself asking what happens when the remote procedure completes successfully, but the calling side gets an error. Or spend some time wondering why some simple code is so slow before you realize that it's spending a millisecond on network transmission to do a tiny fraction of that time's computation.)
If you ever see the claim that you can write your code normally and then decide how to distribute it, run away screaming. Or picture me doing it for you. It doesn't work. When you build a distributed system, the fact that it is distributed, and how it is distributed, are an order of magnitude more important than everything else.
If you're using it to make communicating between languages easier, then yes, that is a big win. But there are other ways to do that. REST and JSON are relatively new, but textual protocols like HTTP and its figurative ancestor, SMTP, aren't. And if that's not good, there's always IETF block diagrams and network byte order.
But, you're saying, I'm doing RPC right, structuring my systems around the communications and relying only on large-grained, asynchronous calls. That's great, but what exactly is RPC buying you at that point? 'Cause it's costing you at least visibility into the communications.
My first question for anyone using bridge is this: what happens when you have 3 clients calling code on a server, which in turn is calling a couple of other servers, and those servers are making calls back to the original server.
As someone who makes architectural decisions on a regular basis, I need to know that if a company that we're a dependency on dies, we can rip it out and replace it with a) an open source version of what we were paying for or b) our own alternative implementation (which sounds like something on top of RabbitMQ in this case). It's frustrating to see companies like this positioning themselves in a way that makes them difficult to evaluate.
We have open sourced the clients but Bridge Server is not open source.[2]
How do we make money? We're building this type of software for much larger organizations (think enterprise & government). We're not in the business of charging hackers and students $5.
We're likely going to charge some money for Bridge Cloud as a lot of the startups currently using Bridge prefer to use & pay for Bridge Cloud. It's the peace of mind of not having to deal with ops on a messaging server. Bridge server itself will also cost some money.
But the point is that we're only going to charge the big guys who can easily afford such systems. Their alternative today is to spend tens of millions on an engineering team to build systems in house.
There will always be a generous free tier for Bridge. You can download Bridge right now and pump 40,000 messages per minute through it for FREE. Do you need more? Just call me, I'm happy to bump you up.[3]
We have a setup for companies paying us and need to see the source code. If it's really important to you, we're happy to show you the code.
What can I do to help you evaluate Bridge? Can you shoot me an email? I'm at darshan@getbridge.com.
[1]http://techcrunch.com/2012/01/05/andreessen-horowitz-salesfo...
I expect to add a pricing page to the website next week.
It's been super useful for all types of projects -- I've seen it used in tons of small 24 hour hackathon projects, and have used it to scale my own app and port it over to mobile with very little friction. Definitely recommend this piece of technology.
Planning on selling licenses I suppose? I would have loved to have a poke around in the erlang server code. Cowboy/rabbit/messaging - sounds fun.
As I said in comments above, if the source is really important to you, I can walk you through it and give you access so that you feel comfortable about what you're adopting. We're just not ready to open source the server yet.
You don't even need to pay for a server license until you're doing thousands of messages per second! At those scales, you should be able to afford cheap licenses. We're not focused on making small amounts of money from ramen startups or hungry students. We do have a good business in enterprise software to fund the free Bridge Cloud servers.
[1]: darshan@getbridge.com
No offence to you, but I have sen too many companies go bankcrupt, or get bought out and shut down, to trust such important infrastructure to you!
Bridge allows you to securely expose APIs to "untrusted" clients such as browsers and mobile clients. ZeroRPC does not do this because 0MQ doesn't allow such access control.
Question for you: on your roadmap[1] you write:
> Efficient binary data transfer
> This would allow for some interesting applications such as video streaming, file transfer, image processing and more. Binary data support was one of the key reasons Github chose to roll its own serialization format (BERT) instead of using existing options.
Why is BERT > MessagePack for this?
Github's BERT is also good for binary serialization but we decided to go with MessagePack because of its wide language support.
I don't think BERT-RPC is necessarily _better_ for binary. But we referenced it in the roadmap as a good idea that we derived inspiration from.
* ZeroRPC has been open-sourced by dotCloud with no plans for monetization. It's just a fundamental piece of infrastructure which we think is worth sharing. Licensing fundamental infrastructure software is not our business (also: it's very hard).
* I see very interesting scenarios where ZeroRPC can be integrated with Bridge. There is already work on integrating it with systems very similar to Bridge, for example Stack.io (http://github.com/dotcloud/stack.io).
* Bridge currently uses a central broker, ZeroRPC is broker-less.
* ZeroRPC gives you streaming responses (similar to python generators). Bridge instead provides pub/sub.
* Bridge uses callbacks everywhere, whereas it varies for ZeroRPC based on language implementation. Our node.js version obviously uses callbacks, but the python version doesn't, thanks to gevent.
Message queues aren't a silver bullet of course, they introduce a lot of potential problems which need to be solved on a case-by-case basis. Such as, what happens if a receiver dies before handling a message? Do senders need to retry sending messages or can they fire-and-forget? Is retrying built in to the queue layer? What happens if a receiver is bogged down with too many messages? Is there a limit on the incoming queue size and what is the limit? Can we start dropping messages if we're overloaded; which messages can we drop? If you make a request that expects a reply, should you timeout and what timeout duration should you use? When a new handler joins the system, should they instantly receive messages that have been waiting for them, or should they dump the existing queue? And so on..
In terms of a web api vs. glue code, Bridge requires very little if any glue code. Check out our roadmap. We do have a REST endpoint planned. https://www.getbridge.com/about/roadmap
here's a video of our blimp: http://photos.zyron.me/Other/2012-04-29/22712358_zSZWhh#!i=1...
Bridge looks like a good alternative. Stuff like client-server, binary data serialization, and no IDLs makes Bridge very appealing.
Take a look at Evernote[1]. They needed something like this, and it sounded like they grudgingly went with Thrift. Bridge might be what they needed.
http://blog.evernote.com/tech/2011/05/26/evernote-and-thrift...
I read the blog post you linked and I don't see any real evidence that their choice of Thrift was "grudging". They did say that there are differing requirements that might lead to others to make a different choice -- but AFAICT there is no indication in the article that they regret their choice.
WRT IDLs: YMMV of course, but the post you linked provides the following reasons why an IDL might be a good idea: "A more formal IDL and native code generation may help provide better long-term client stability, so the initial Thrift overhead for developers might be justified for some services that are concerned about client stability and longevity."
From what I've seen mongrel2 is designed as a HTTP/REST -> 0MQ gateway. We also have HTTP/REST endpoints in the works but we're designed with persistent WebSockets/TCP in mind.
It looks like it is built on top of rabbitmq.
I will be keeping an eye on these types of frameworks because I want to see how far the framework can go in the stack. Specifically I am waiting for frameworks that help me manage my web UI in an easy manner.
// js
var Bridge = require('bridge-js')
// python
from BridgePython.bridge import Bridge
// ruby
require 'bridge-ruby'
---
First of all you don't really need language name in the name of each of the bindings. It's kind of obvious that i'm using ruby or python at the moment, isn't it? Python version looks especially ugly it should have been much simpler:
import bridge
Sorry about that :(
"Write each component in the language best suited for the job. "
Or is this supposed to be a subtle stab?
Anyone else notice that the pip install is broke though? Complains that 'README.md' is missing from the tarball.
I'm guessing it does something similar to socket.io, but I haven't tested yet.
To enable encryption right now, please see https://www.getbridge.com/docs/gettingstarted/js/advanced
"The default is to send your private key in plaintext to the redirector."
What?
Not sure about this, but I think it's what you mean. I googled "node broke", and didn't come up with meaningful results, so I assume it's "Your". Correct me if I'm wrong!