DeepstreamHub: a faster Firebase with an open core
deepstreamhub.com
deepstreamhub.com
From the Firebase JavaScript SDK, I can do one thousand individual gets with a 3.4k payload at 1-2ms each. http://jsfiddle.net/katowulf/8wkon62c/
Mileage varies, of course, but I'd love to see a benchmark I can run side by side that can beat those numbers. I can't even get close with a simple GET, which returns in 86ms.
direct benchmarks are something that Firebase' Terms and Conditions make hard to do
Two different engines doing two different things.
Benches are for recording "personal best" stats, not "laps at the public pool". I.e. Testing against self and not others.
With firebase since we were not running a node or java app, we ended up relying heavily on the rest interface. Which was sort of the not well taken care of. We would see crazy spikes and we could not get support to fix it. Multi tenancy would occasionally cause other performance issues.
We eneded up running a bit of a monkey patched system for awhile using kineses to pipe updates to a java/node app which then would write updates via the web socket interface.
Firebase also lacked support for pub/sub events, which can be sort of faked, but not entirely reliably.
Deepstream model is a bit different as you said, and it does have some weaknesses. I have submitted a number of patches already to fix some of the edge cases i've run into and will likely submit more.
Unlike firebase, we can actually directly improve the deepstream application core as well as build options into the platform that better support our use cases.
Dsh allows you to store arbitrary data, but breaks it down into smaller chunks a large Mongo or Couch db.
As these smaller chunks have their own life cycle lists make it easy to use them within iterators, e.g. For each in Vue etc, where every component manages bits own granular document
We use it in production. The use case is actually Server <-> Server communication and so far it works good.
The core differences are in the way deepstreamHub structures its data (small, related documents with independent lifecycles rather than one big data tree), its speed and open core
(Still looks good!)
* a better presence story * a better story around adding data to a list and subscribing to changes to specific parts of it
Architecturally, Realm and DeepstreamHub are quite different. Realm at its core is a distributed object database that updates changes to objects in realtime. DeepstreamHub, as far as I can tell, relies on a pub/sub and request/response architecture. They are two fundamentally different approaches to the challenge of realtime.
One of the biggest advantages with Realm is that you work with your objects in your native programming language, there's no JSON or ORMs to deal with, just your live objects that update automatically in the local datastore as changes are made on another sync'd client or on the server.
I completely agree! One of the reasons we started is because we believe we have an interesting way of getting devices of all types ( IoT / browsers / phones.. etc ) talking together to generate new ways of displaying and moving data.
At the end of the day the selection of choices such as the eclipse iot project https://projects.eclipse.org/projects/iot, https://relayr.io/, aws IoT gateways, (shameless plug) deepstream.io and many others allow developers to stop having to think about how to get devices to communicate, but instead think about what they want it to do. I'm slightly overly passionate about this because my bachelor project a decade ago was around smart homes and I spent the majority of the time trying to get protocols to collaborate happily, where now that's just considered part of boiler plate setup! Really excited to see the potential outcome due to this general movement.
Here is my thoughts/arguments around it, dealing with the RethinkDB and Parse shut down as well: https://hackernoon.com/the-implications-of-rethinkdb-and-par...
Plus, was invited to speak about it on the Changelog Podcast: https://changelog.com/podcast/236
Please stay truly open :) that way we can still compete. ;) Or else we'll be the only company that is fully Open Source (replication, auto failover, etc.)!
It isn't the right approach, and we ( http://gun.js.org ) aren't going to take it. More than happy to be the only one left standing, devoted to true open values - but I hope others join us.
But at the end of the day any compromising on open-source ideals is going to piss someone off - heck even just choosing one OS license over another is going to piss people off. It doesn't matter. In business you need to have a thick skin to stick to your conviction about what is best for both customers and the company - whatever that may be.
And @yasserf's reply was fantastic, too. Sounds like they are more providing premium hosted services than anything else, which I think is definitely a good way to go.
https://blogs.gnome.org/bolsh/2010/07/19/rotten-to-the-open-...
A purely services based model is difficult, but so is open core. With open core you compete against yourself. Look no further than Nginx as an example and how lousy it is to do things like say a http healthcheck of HEAD /status. In haproxy it is trivial, it can be done in nginx plus, but not the oss version (at least it wasn't supported in any of the versions I tried a year ago sans the commercial version). When you compete against yourself you get put in a difficult situation "Does this valuable user contribution to my OSS project compete with my commercial fork?" and that is just all around a bad situation to be in for the community and for the company.
deepstreamHub literally runs the deepstream.io servers to provide all of the fundamental functionality. All of the client sdks are also Apache 2 based which means that they can be changed in whatever way suits people most, and written in any language.
We also have some really interesting forks by community members that have diverged a bit from how we do things, and has resulted in collaborative forums and in person meetings as well as even talks at meetups about the pros and cons which resulted in the divergence.
The non open aspects are all around the extra services ontop. These all work by hooking into deepstreams plugin architecture, meaning that as stated in following comments, open source plugins are encouraged and are documented in as much detail to help people get started. Offering all the non-core/multi-tenancy/usage monitoring/gui components aspects as open source is in that sense extremely counter intuitive, since we will instead be providing very specific functionality for the purpose of creating another PaaS system. That won't be stopping us from releasing detailed articles/give talks about how things work and run under the hood on AWS and the challenges we met hosting at large scale and across regions, it just means we won't be providing the actual code.
In terms of auto-failover, replication, etc, it would be great if you look at the current open-source aspects on deepstream.io that we have in place for that. deepstream is built with availability and clustering at its core, and its truly because of the involvement of the open-source community that we have managed to make it run in multiple production environments to date.
Would you be willing to call it "open source with premium hosted services" or something rather than "open core"? I feel like that is more accurate and would be better for marketing, and avoid the whole "open core / crippleware" debacle (see the other comments about InfluxDB). Just my 2 cents!
Cheers! (Thanks for the reply)
major humble-bragging. In bad taste to do that in an announcement for a "competitor"?