Horizon 1.0: a realtime, open-source JavaScript back end from RethinkDB
horizon.io
horizon.io
I've been up for about twenty-four hours, but I'll be around to answer any questions for the rest of the day.
What do you guys think about the approach taken here, and what are the major differences in approach theat you took with Horizon?
https://github.com/mikemintz/react-rethinkdb
I've had really good luck with that whitelisting approach, so I'm wondering what the biggest wins would be from potentially switching to Horizon.
On the other hand, Horizon exposes a simple API/protocol that's easy to learn and easy to secure. If you need more functionality, you can import Horizon as a Node.js module and start writing backend code that accesses RethinkDB directly. The learning curve is much easier, it provides an upgrade path for sophisticated apps, and feels like a more robust product.
(I don't mean to disparage `react-rethinkdb`, I think the project is awesome, it's just a philosophical difference in approach)
Could you give me a quick comparison of Horizon and Meteor? I've used Meteor in the past (and loved it), and it looks like Horizon might be fun to learn.
I'll watch Horizon but I'm taking it all with a grain of salt. Standard new javascript framework protocol people.
But Meteor isn't completely prescriptive. There are several pieces that can be swapped out: You can use Angular, React, or Blaze on the client, for instance.
From what I've read, it doesn't look like Horizon is a complete framework, just a database client/server component that can be used with any framework.
Except we:
1) optimized our database operations
2) relegated cpu-bound work to micro-services that could process work from a queue at their own pace
3) Leveraged the ecosystem to writer smarter and more concise code
Know your enemy and know yourself and you need not fear the result of a thousand battles (Sun Tzu). Meteor is an amazingly productive ecosystem for writing real-time applications. No it will not solve every problem you throw at it.
Just to be clear I'm not hating on Horizon. I definitely will try it in the future.
I don't actually know what is supposed to "not scale" about Meteor. I've just read it multiple times. But because of the nature of Meteor (Really easy for anyone to set up! Anyone can create a real time app in ten minutes or less!), it probably attracts a lot of junior developers. It therefore should have occurred to me to translate "It doesn't scale!" to "I don't know how to scale it and my app is slow!!!". :)
* Node as a runtime is deeply mediocre, in my experience. You can usually solve this by throwing more (usually virtualized) hardware at the problem.
* Meteor's architecture is (at present) tied into MongoDB's oplog tailing. You can scale it up through sharding, through alternate libraries, and DDP itself is not tied to MongoDB at all. IIRC fixing this was a goal for Meteor 2.0.
Both of these can be worked around. If nothing else, the Meteor web side will talk to anything doing DDP, and DDP is not that hard to deal with.
1) real-time view layer
2) diverse and highly integrated ecosystem (libs are written just for meteor)
3) productive development environment (es2015 modules on client/server, no fuss build pipline for sass, jade, angular-templates, angular dependency injection)
There are a ton of secondary benefits, but thats the crux of it. Yes we do sacrifice some of the benefits of Meteor as a monolithic (homogenous?) full-stack solution, but I would argue that they are in fact small, and if you dive into Meteor you'll find it to be pretty modular.
Horizon looks to be more like the Minimongo/DDP/Accounts parts of Meteor, and doesn't help with any more of the stack like Meteor does (eg: In Horizon its more up to you to wire it into Angular/React/etc). Meteor is more of a platform and Horizon is more of a real-time client DB and account framework.
If I had to choose between the two, Horizon would be good for if you had a strong opinion/technical need for very specific coupling between UI and DB, and Meteor for when you don't or need to get something out very quickly (eg: this weeks internal CRUD app for a few dozen users).
But without optimistic transactions, what sets Horizon apart from just running the transaction-code on the server? If I understand it correctly, the latency will be the same in both cases, i.e., one network round-trip.
Lack of optimistic updates is a temporary state, though. The vision for Horizon definitely includes this functionality, and we'll make sure it lands ASAP.
Make sure you don't regret about your answers later :) Answering questions on HN after being 24 hours awake is generally not a good idea :)
1. Do you have something analogous to https://www.firebase.com/docs/web/api/ondisconnect? Real-time apps often have some manner of presence indicator. Firebase allows you to say "when this client disconnects, set the value of this key to $BLAH". I noticed that Horizon has http://horizon.io/api/horizon/#ondisconnected, but this is just a client-side indication of when you've disconnected.
2. What are Horizon's transaction properties? How about ordering of appends to a collection?
Since Horizon is backed by RethinkDB, it's transaction properties are the same as that of RethinkDB. Check out this document for the specifics -- http://rethinkdb.com/docs/consistency/.
Also, there doesn't seem to be a callback arg in Collection#store: http://horizon.io/api/collection/#store. This is worrying. How do I know if a write succeeded?
Edit: found http://horizon.io/api/collection/#subscribe - you chain subscribe to writes when you care about their success
I opened an issue against our docs to add information on this, as I'm a bit fuzzy on the detailed behavior myself: https://github.com/rethinkdb/horizon-docs/issues/50
Can I use Horizon on mobile devices?
Horizon ships with a JavaScript client library that can be used for building web and mobile apps with frameworks like React, Angular, or React Native. We’re also working on specific platform client libraries that support the Horizon protocol. Client libraries for iOS and Android are coming soon.
Im really stoked for this and thanks a lot for your hard work.
This is key. Firebase seems great for quick prototypes, but I never built against it because it was unclear (to me at least) how to deal with cases where you want to leave the sandbox and add custom backend functionality that they haven't built out for you. Horizon seems ideal -- it gives you the ease of use of a turnkey system starting out, with the flexibility to customize backend logic/validation later when you need to. That, plus the fact that it's put together by the awesome RethinkDB team, makes it very interesting to me!
Regardless, I'm very excited to play around w/ Horizon as well.
That said, at first glance there doesn't seem to be anything stopping anyone from building a caching layer on top of Horizon to enable offline-first functionality. Definitely going to be keeping an eye on this as well!
EDIT: It looks like they are in fact considering offline-first support already [2]!
Does Horizon also solve "optimistic updates"? If so I'd love to learn more details. For comparison, Meteor keeps a local datastore that updates immediately when data is mutated and then reconciled with the real database.
// ex
this.props.horizon.order('datetime', 'descending').limit(8).watch()Security is the same as the clients DB writes are rejected if they are modifying fields in ways that are not permitted (eg: changing a trouble ticket from "submitted" to "approved"). This can make some of the security easier to audit as most of the business logic security is in one place rather than scattered about in dozens of API calls.
While the exact technical details for these real-time DB based front ends are different, the basic thinking is the same of never trust the client and its costly to transmit info to the client.
The nice thing about having the DB directly on the client is that it makes CRUD apps much easier to make as you have to build less bespoke framework to get data to/from the client and verifying things conform to business logic. If you are not building a CRUD app you will want to evaluate other methods for a better fit.
The sibling comment by ubercore has the right link. And your concern is important, since without security, why not just expose your db to the frontend directly?
At a high level, the permission system:
1. Is a whitelist approach (you can't do anything by default)
2. Allows you to specify query shapes that are allowed to be run by the current user. This prevents queries of the wrong shape from ever hitting the database, helping with scalability.
3. Has a fallback to full javascript rules which can depend on the data on disk. A little slower than the query rules, but allows much more flexibility where you need it. And you can combine them with a query rule to do a rough first pass of what is acceptable.
Ultimately, the permissions system is designed to prevent unfettered database access, but it also has an eye towards performance in the case of DOS attacks. Horizon overall attempts to use only indexed queries, low resource usage operations etc, so you don't accidentally invite DOS attacks .
Now, I'm not claiming we're DOS proof or anything, but I just intend to mention we have an eye towards it when we talk about security.
So while the user can change JavaScript as they please, that's ok; if they try to do something they're not supposed to do, the server will just reject the query.
> Horizon isn’t prescriptive or particularly opinionated–it’s designed to work well with the JavaScript frameworks that you already know and love.
It is seriously a breath of fresh air to read something like that. Bravo, Horizon/RethinkDB team.
This guarantees that things will interop nicely with eachother and we'll be able to use Rx all around.
It's an awesome database for a variety of use-cases.
Personally, I think their query language + support for joins makes it much more useable than MongoDB (which I've used quite a bit).
But man was I sad to leave RethinkDB for that!
Have there been any performance benchmarks or reviews of RethinkDB yet? Especially at this scale?
https://aphyr.com/posts/330-jepsen-rethinkdb-2-2-3-reconfigu...
We have a performance benchmark report we did a couple months ago basically locked and loaded to publish with accompanying scripts and data for transparent review of our methodology. We just have been so busy with Horizon, we had put this on the back burner. Expect to see something within the next couple weeks when we can put some serious attention to it to ensure we do everything just right.
In the meantime, you should check out the whitepaper by BigchainDB which uses RethinkDB for the purposes of consistency and automatic change notifications [1].
Also: kudos to whoever is handling the artwork/marketing side of things. Love the vector art and website designs.
A websocket connection between client and server is great for real-time apps, but for non real-time app, the unnecessary websocket connections create more work for devops -- specifically DNS and load balancer. Does/will Horizon support that clients only use the Collections API to get/set data without websocket (for real-time sync)?
I'm looking forward start prototyping some apps using Horizon. On a first look seems very promising.
Congrats!
This isn't false humility either -- I've been super busy with some other aspects of RethinkDB over the past few months, so while I did have a hand in designing and shipping Horizon, maybe less than 5% of it was my doing.
One thing that I always found a bit lacking with Meteor was pagination. The problem in a nutshell is that the client doesn't know how much data is potentially available on the server unless it requests it, making things like showing "page 1 of 12" type result counts tricky (I talk more about it here: https://www.discovermeteor.com/blog/pagination-problems-mete... )
I'd be curious to know how Horizon deals with that problem?
Sadly, Horizon doesn't seem to support aggregate queries yet, reactive or not, without loading the records with findAll. Theoretically, Rethink could be even more scalable at this than Mongo, since it could use its index to only "wake up" relevant count subscribers. But this is still on the roadmap:
https://github.com/rethinkdb/rethinkdb/issues/3735 https://github.com/rethinkdb/rethinkdb/issues/1118
Generally, it seems Horizon is very limited compared to Meteor, but it's a great fresh start, with a much more well-thought-out and scalable architecture, and I'm looking forward to seeing where it goes.
For convenience, this is the section taken from it that addresses this exact question! (I imagine other people may be thinking this)
How is Horizon different from Meteor?
Horizon has philosophical and technical differences with Meteor that will result in vastly different developer experiences.
Horizon is a small layer on top of RethinkDB with a narrow, purpose-built API designed to make it very easy to get started building realtime apps without backend code. Horizon isn’t prescriptive – you can use any front-end framework without any magic/customizations, and once your app outgrows the Horizon API you can use any backend technology (e.g. Node.js, Python, Ruby) and any backend framework (e.g. Rails, Express, Koa, etc.)
By contrast, Meteor is a much more prescriptive framework. Horizon is a reasonably small component that has a very clean separation with the database and the frontend, while Meteor is a much more monolithic experience.
Another major difference is architectural – Meteor uses a LiveQuery component built by tailing MongoDB’s oplog. This approach is fundamentally limited – it’s impossible to do many operations efficiently, and even the basic functionality is extremely difficult to scale.
Horizon is built on RethinkDB, so the LiveQuery functionality is in the database. This allows for much more sophisticated streaming operations, and scalability is dramatically simpler because the database has all the necessary information to allow for a scalable feeds implementation.
Are there any dockerized versions of this yet?
It would be even awesomer if you could include the articles in the feed so I don't have to leave my cozy RSS reader :)
The big piece remaining that will help is write timestamps in the database, which we're going to try to land in RethinkDB 2.4. If we were controlling a traditional MVC frontend model and had control over the objects themselves, it would be easier to do optimistic updates. Since we're aiming to be compatible with a lot of frontend frameworks where immutability of the model is important, we have to carefully consider how to do it.
That being said, it's super important, so it's near the top of our priorities
I couldn't find it in the docs, but what is the equivalent to Firebase.ServerValue.TIMESTAMP?
Why should I use it over my current approach ? Why is this compelling?
You may want to make your homepage clearer, before navigating to the FAQ I did not have a clue of what Horizon really does.
Amazing work Horizon team!
We pushed a hotfix a few minutes after launch that resolved this issue [1] which you may be experiencing. You should try doing `npm update -g` if you're still on `horizon 1.0.0`. You can check with `hz version`.
If this doesn't fix it for you, please open up an issue [2] and I will help you out.
Kind of being able to have the same logic in my spreadsheet as well as my api ?
This could be huge!
Looks cool!