Realm 1.0 | Hacker News Reader

Realm 1.0

realm.io

261 pointsporsager101 comments

Realm is awesome! I have been using it for a year now in a large production iOS/swift app with great success.

Great job on the recent additions to the more fine grain notifications, with this I was recently able to reduce my own code, simplify and increase performance!

Core data is good and a powerful weapon, and certainly educational to learn its approach and how to use it effectively. I sometimes ponder if going through the that first has made me a more competent iOS developer.

Realm is the right tool for the job for me in most use cases. Don't see myself switching back anytime soon, keep up the good work!

Do you use your own data synchronization solution? It seems to difficult to do it right. We love Realm but we eventually choose couchbase because of its out of the box sync solution. I hope Realm consider this feature as most mobile developers all have to think about this problem.

Do you mean from server to client?

this is handled by the applications code, which inserts data primarily from a rest api into realm in a background thread. Not all data comes from rest however, some from apns or xmpp, depending on application state and the particular data.

Syncing is hard to get right. If realm offered it I would consider but I would not want them to do that at the sacrifice of making their product as solid as it is.

How does couch handle it? Does it have push and poll type of hooks?

(Adam from Realm) thanks for the kind words! Fine grained notifications were very important to us to support to enable a more reactive architecture. Glad it has benefited your code base!

What's the benefit of using realm instead of Sqlite + an ORM ?

Here are a couple we highlight:

1. Since Realm is not a relational database with an ORM, we don't run into the impedance mismatch. Relationships are first-class citizens.

2. Realm uses a zero-copy architecture. Objects and collections are lazy loaded offering benefits to both memory use and speed.

Thank you.

For me it's:

- simplicity, which in turns creates less complex code and less crashes

- performance, which for my use case was faster than sqlite

- slightly opinionated, in that it forces you to not access objects from different threads, which can eliminate a whole class of bugs (you could and should do this or a similar approach to prevent these with whatever solution, but Realm forces it)

Congrats to the team for expanding their platform support. I'm still interested to see what sync looks like.

If you are interested in mobile databases, you'd do well to evaluate Cocuhbase Mobile, which is 100% open source, available for most mobile platforms, and includes bulletproof synchronization with its open source secure cloud server.

More info here http://developer.couchbase.com/mobile/

I do not think Realm has sync. Using couchbase for mobile development definitely makes it easy to get data from the device into the cloud (and back). The only complaint is it feels more geared to enterprise to have the best infrastructure for large set of users, but I guess that goes with the data size.

Do you have any comparison metrics of Couchbase and Realm or other mobile Databases?

Can someone elaborate how this differs from JPA/Hibernate style ORMs ?

The examples look suspiciously similar.

EDIT: So i easily believe that it's nicer than CoreData, and I see that there is lots of potential to reduce an ORMs complexity by assuming a single Writer (Process of App), but the core Problem seems to be exactly the same. - And the solution looks also very familiar. -> Proxy Objects, Bytecode manipulation to intercept field access etc.

Does it handle the uncool cases of ORMs better? - Unloading of state when thousands/millions of objects are touched once

- Collection fetching of large collections, iterators/chunks

- OrderedList O(N) index update for non-append inserts

- What happens to invalidated (deleted) objects? are collections updated live?

- Does is handle cascades explicitly/implicitly/not ?

I'm quite interested in this, since it may solve real pain problems.

Realm is not a relational database, but an Object database.

This means that we totally avoid the impedance mismatch when you try to map Java references to tables. In Realm, relationships are first-class citizens.

This is also one of the reasons that we can provide lazy-loading for everything. With ORM's the API might look the same on the surface, but you usually run into problems when trying to access foreign collections.

API wise we try to stay as close to normal Java as possible, which is probably why it looks similar to what Hibernate offers.

Regarding specific features.

- Realm uses zero-copy as much as possible, so collections in Java point directly to the underlying data (i.e takes almost no space on the Java heap).

- Collections are live updated yes.

- Deleted objects could potentially be accessed after they are deleted, but there is API methods for avoiding that.

- Cascading deletes is not supported yet, but really high on our TODO and definitely a feature we want to add.

Sorry my edit was after your answer.

Yeah well, literally ORM has relational in it, but you probably know what I mean.

So it behaves more like a graph store? - Can I query bottom up / through collections ?

I'm really interested in the technical details and their implications.

Yes, thinking about the internal structure as a graph will give you the right ideas.

We don't have support for collecting objects bottom up yet, bu t it is tracked here: https://github.com/realm/realm-java/issues/2232

Realm Swift/Objective C has support for bi-directional references and we are adding that to Realm Java shortly as well: https://github.com/realm/realm-java/issues/607 ..That will also help when doing bottom-up queries.

If you want to know more about the internal details. There is some talks/articles here that might interest you:

- https://realm.io/news/threading-deep-dive/

- https://realm.io/news/jp-simard-realm-core-database-engine/

- https://www.youtube.com/watch?v=vGFInG96KH4

Thanks !

I'm reading through the docs and it seems very appealing. Especially the Observability of changes. I assume this works well because it is contained in one Process.

No, the change notifications will work across processes as well. Realm Java still need to add multi-process support though (https://github.com/realm/realm-java/issues/1091), but it is working in Realm Swift/Objective C.

Thanks for your answer.

Realm has excellent marketing, but being a closed source third party SDK disqualifies it as anything I would recommend for my company to use.

I've had to deal with enough pain from closed source libraries from the likes of Google and others. Closed-source prevents me from adopting a new platform until the vendor gets around to porting it, and only if it fits with their strategy. Also a middleware-only company is liable to be bought out, and then closed. Example: Parse.

What part of it is closed source?

https://github.com/realm/realm-cocoa https://github.com/realm/realm-java https://github.com/realm/realm-js https://github.com/realm/realm-dotnet

Edit: Apparently not the guts of it, my bad :(

The core C++ part. They plan to open source it later, but who knows what will happen in the meantime.

Where did you read that they plan to open-source it later?

FAQ time:

I see references to a “core” in the code, what is that? The core is referring to our internal C++ storage engine. It is not currently open-source but we do plan on open-sourcing it also under the Apache 2.0 license once we’ve had a chance to clean it up, rename it, and finalize major features inside of it. In the meantime, its binary releases are made available under the Realm Core (TightDB) Binary License.

https://realm.io/docs/objc/latest/#faq

Ah, OK. I always assumed that the core was closed-source for business reasons.

It's a LOT of work to open-source something which has been evolving at a huge pace and is as complex as the Realm core.

Even just announcing the work of the Xamarin C# team that I'm on got some complaints from the public about this representing a "loss of focus". So yes, you could say there was a "business reason" of protecting the core team whilst they got to 1.0.

If you look at the C++ portions included in most of the Realm open source you can see we have moved code gradually into the open source codebase. There's not yet an official C++ API either for the same reason - it would lock in too many decisions and tie up the team. (disclaimer - these are opinions and observations from a remote contractor, not Realm management!)

I don't believe that complexity or size of code really affects the difficulty of open sourcing a project. Its more likely the culture and daily development practices that are difficult to change, but probably not as difficult as the business strategy we're all wondering about.

To add on to this, it's also a pure tech services play, which IMO is generally a bit harder to monetize than a product company, but generally a lot easier to sell in an acquisition. Which doesn't make this sitatuion any better.

What exactly is Realm's business model? The core is closed-source, but as far as I can tell, it's free for application developers to use.

Congrats on 1.0!

+1 for not just telling us what Realm is, but for showing us (with inline code), and for supporting React Native :)

What about using realm Java outside of Android? The readme says:

> This repository holds the source code for the Java version of Realm, which currently runs only on Android.

I'm interested in trying this on the desktop (windows/linux/mac), but that doesn't seem possible? Wonder what makes it android only.

FWIW you can use Realm Objective-C or Realm Swift on Mac OS X.

I'll let someone else clarify why Java outside Android isn't supported.

The core of Realm is closed-source, and Realm only provides builds for iOS, OS X, tvOS, and Android.

don't forget watchOS ;)

Oh, missed that bit.

We're focused more on mobile for now, JVM support is something we want to build. For now we're relying on some Android specific components like the Looper (to deliver notifications) & the Android Context, otherwise most of the API uses pure Java API.

VP Product at Realm here. Thanks to everyone on HN for the support over the years! We came out of YC in Summer 11 and it’s been a fantastic ride growing over the past few years. Would love to hear y’alls thoughts on what we should add next. Is there any reason why you’d still use SQLite?

I've been trying Realm for Xamarin recently and I've got to say I'm pretty disappointed with it, compared to my previous experiences with EF.

My complaints are as follows:

- No enum support

- No inheritance support

- Many LINQ operations aren't implemented, no select, no aggregate, etc. I have to do a ToList or ToArray to use these operations.

And that was just what came up in the first day of usage...

Why would I use Realm if EF7 gets SQLite Code First+Xamarin support and can do all of these things? Sure, it's faster, but faster doesn't matter if I'm not speed critical on my DB (most mobile apps aren't) and I can write cleaner code. It might be better than manually writing SQL queries, but it's a lot worse than existing ORM solutions.

The Xamarin product is the youngest in the family (and, not officially 1.0 like the others) so we obviously have a lot to do still. :-)

Your complaints are noted and are more or less all in the pipeline. Our goal is definitely that you should be able to write cleaner as well as faster code than you would with EF. Thank you for your feedback!

Xamarin has been highly requested, so we have tried to make it available as quickly as possible as a pre-1.0 version to get the proper input from developers like yourself as to what should be prioritized. Love to keep getting your feedback on github as well. Thanks!

1. "Realm is not an ORM, and is not built on top of SQLite. Instead we’ve built a full database for mobile app developers, one that uses native objects that are dynamically mapped to a full, custom database engine (not just a key-value store)."

=> Why not use the terms "object database" or "OODBMS", then ?

2. "Realm supports Java, Objective-C, React Native, Swift and Xamarin."

=> "React Native" is not a language, why not just state that you support "JavaScript" ?

3. What about Python ?

> 3. What about Python ?

Is there any practical way to use Python for mobile apps? Realm is only targeting mobile platforms AFAIK.

Some people are active in this domain, see: https://www.python.org/community/sigs/current/mobile-sig/ and https://mail.python.org/pipermail/mobile-sig/

Two approaches that seem to work (or to be promising) are Kivy (https://kivy.org/) for cross-platform and PyBee / VOC (http://pybee.org/voc/) for Android.

1. We are close to an object database, but with some subtle distinctions from the mainline expectations I believe — data is serialized in a language-independent binary format, that is actually column-oriented. We also have full indexing available of all fields. It’s not that object databases can’t do that, it’s just that most people probably don’t expect that level of access. And to be honest, there are marketing considerations too, in the OODBMS feels very dated as a term.

2. We don’t support JavaScript in general at the moment, only JavaScript that runs inside JavaScriptCore inside React Native. Otherwise, we lack the C++ APIs that we rely on to power the underlying storage engine. We are trying to add support for more JavaScript runtimes though.

3. Python support is doable, but we are focusing 100% on mobile front-ends right now. We don’t even support using Realm from Java outside of Android, for example. Eventually we do want to add support for Python, Ruby, node.js, etc.

Looking forward to Cordova/Ionic support, our community would love this (I see it's slated already, cool).

React Native isn't a language, but it is an application platform. JavaScript just happens to be the language for that platform. The Realm bindings for JS are specific to that platform and they won't work with traditional hybrid apps.

R-tree :)

Also: text index and a way to "unmigrate" a database on iOS would be cool when developers are working on different branches at once, so they can restore their data when they have a different migration version in some container

(JP from Realm here)

Unless I'm missing something, what you're describing could easily be done by having the version number or branch in the Realm path, or whatever identifier you want. You should open an issue[0] so we can explore that more.

Strings are indexed, but I think you're referring to something like "full text search"? That's on the roadmap[1].

[0]: https://github.com/realm/realm-cocoa/issues/new [1]: https://github.com/realm/realm-cocoa/issues/3650

> Is there any reason why you’d still use SQLite?

Windows. Specifically, my current project is targeting Windows desktop (not WinRT/UWP) in addition to mobile platforms and Mac.

Also Unity3d works really well with it using the SimpleSQL plugin.

Is unsigned integer support coming any time soon? Right now I'm using an integer and a computed property (Swift) to reinterpret the bit pattern as an unsigned value, which is... okay, I guess, as the field doesn't ever need to be queried. Filesize really matters in my case, so I don't want to go up an integer size.

It just seems like an odd limitation - it's all bits!

(JP from Realm here)

Yes, it's all bits, but those bits matter :). The internal query engine is extremely optimized based on the layout of those bits, on a per-architecture basis.

Also, integers are adaptively bit-packed, so you shouldn't worry about the size of your integers, as Realm will intelligently use the appropriate widths.

I'd suggest continuing to use this computed property pattern and to keep track of #651[0] where we're tracking better support for this.

[0]: https://github.com/realm/realm-cocoa/issues/651

Oh, given the packing, is there any reason to use the property instead of just moving up a size?

I'm not sure what you mean here. If you mean "why would I ever use Int8 or Int32, etc. if Int64 won't take up more space in the file" the answer would be memory considerations when you take that data out of the Realm... you should just use the size you need and you'll be ok.

I'd like inheritance / polymorphism. For example, would love to subclass `AppUser` from `User`. I realize it's on your backlog, but just figure I'd give it another +1.

To answer your question, Realm blows away Core Data & straight up SQLite. The context management (or lack thereof) is wonderful. Going further, I have found the errors self-explanatory and easy to fix. Finally, nice job throwing an exception trying to start a write block in a write block -- this forces client programmer to really think about where their writes happen instead of blindly writing all the time (and inevitably running into a nested write).

I'm looking for a good geospatial data store that works on mobile devices. Ran into an issue where Spatialite used to corrupt databases larger than 2GB on Android (as in, run md5sum on the database, open it read-only, run another md5sum, notice they've changed.) At the time when I investigated this, the Android support was scattered across both the initial project and a secondary build with a vendored Spatialite, it wasn't clear whose issue it was, and none of the projects seemed interested in claiming ownership or suggesting fixes. The whole experience left me a bit soured on Spatialite as a mobile geospatial data store if your intent was to, say, load large areas for offline navigation. Maybe it'd work better in the case where you're syncing up specific areas, but my interest was in prepopulating with specific datasets for large regions since I was often off-network and couldn't predict whether a dataset would run out in the middle of a dead zone.

Any chance Realm will or does support geospatial data? Couldn't find any indication that it does currently, so if I ever returned to this project then I'd likely suck it up and try to figure out why Spatialite corrupts larger datasets. But having an alternative would be nice.

(Adam from Realm) we don't offer support for geospatial data yet. An issue to track this was created for Realm Cocoa: https://github.com/realm/realm-cocoa/issues/2569

We don't have a similar issue open for Realm Java, so it would be great if you could create one and detail your needs so we can work to support it.

This isn't Realm related, but Lucene has a spatial extension. And Lucene has been ported to Android. So maybe you could use that.

Huh, will definitely do that, and will also file an issue for Java support in Realm. Thanks.

Per property/per field listeners would make Realm perfect for us (especially w/ RxJava support). The product as it is currently is fantastic, but with the ability to do realmObject.observeProperty("propertyName").subscribe(propertyValue -> {}); my life would be complete.

I tried Realm for an OS X desktop app a few months ago. I initially liked it, but was surprised when I discovered the threading requirements (one realm instance per thread, cannot pass realm-based objects across threads).

1) When I see that a library has a note about threading, I generally expect that it will either be "its thread safe" or "these operations are not thread safe and you need your own synchronization." I think having no ability to share realms/objects across threads is really unusual, and it would be better if the docs made that requirement clearer at the beginning.

2) My use case was reasonably simple - background thread A listens for messages about some expensive work to do, it updates the UI thread with the work, and then it spawns thread B to do that expensive work and keeps listening, thread B updates the worklist on the UI thread on completion. Based on my understanding of realm, I was going to have 3 copies of each object, one on each thread. Traditionally, I would expect one in-memory copy, with the threads using synchronized methods to modify the data. Then, when thread B completes, it just updates its work item and then notifies the UI thread to re-render.

Maybe I misunderstood it, or there is some really simple architecture to handle this issue, but that is how it seemed to me from the documentation.

(JP from Realm here)

Realm's transactions are entirely ACID, and all transactions are thread safe. This means you can't get "faults" like you would in Core Data if one thread deletes an object being accessed in another.

On the flip side of this, it makes it difficult to share mutable object graphs across threads, mostly because then you'd need to know which Realm and which object is at what version of the db. Realm's underlying db engine supports this, but it's difficult to expose/enforce at the API level.

We have some designs in mind to allow scoping around passing objects between threads (see #3136[0]), and some other ideas for passing immutable Realms across threads, so you can expect to see some improvements there in the future.

About improving our documentation around threading, we're always open to feedback. I'd appreciate if you could read through the "Threading" section of our docs[1] and share how you think it could be improved.

[0]: https://github.com/realm/realm-cocoa/issues/3136 [1]: https://realm.io/docs/swift/latest#threading

> having no ability to share realms/objects across threads is really unusual, and it would be better if the docs made that requirement clearer at the beginning.

Good point that we should be more explicit about threading model of Realm and its benefits.

Realms and their objects can be shared across threads (and processes), you just have to be explicit about it (by having each thread get its own instances of the given objects/realms). The huge benefit you get from this, is that you never have to worry about locking or any other concurrency issues. All threads will always have a consistent view of the data, it will only update to reflect changes done by other threads at safe points in your code (like between events, where you are not in the process of using the data).

> Based on my understanding of realm, I was going to have 3 copies of each object, one on each thread.

You don't need 3 copies, only three instances. That way each thread will have its own instance (and as such not be affected by changes on other threads), but they will all point to the same underlying data.

> I would expect one in-memory copy, with the threads using synchronized methods to modify the data. Then, when thread B completes, it just updates its work item and then notifies the UI thread to re-render.

With realm you don't even have to do that. You can let thread B modify the object, and then the UI thread can just watch its own instance of that same object and be notified when it changes, it can then update the UI. All perfectly safe without any need for manual synchronization between the threads.

Congrats on the release, glad you made it to 1.0. And well done sticking to semantic versioning and going to 0.100 before doing the 1.0.

Yes. I've been using Realm for my work app. In my opinion it's easier than Core Data, and certainly much easier than bare SQLite.

I've expected it to cross 1.0 after 0.99 but it keeps going on to more than 0.101. So, congrats on finishing the milestone.

One request though: what about grouping functionality?

(JP from Realm here) Yes, that's something we're hoping to add! We're tracking that feature as #3384[0]. Still needs some design work though.

[0]: https://github.com/realm/realm-cocoa/issues/3384

Please don't say it is faster than SQLite anymore. http://kpgalligan.tumblr.com/post/133281929963/my-talk-at-dr...

With all respect to Kevins benchmarks. There are many ways to measure performance and most of them are wrong. If you see the talk he even admit that his numbers changed quite a bit between each benchmark run.

From what we see, we are generally faster than raw SQLite on all metrics except when doing batch inserts. This is primarily an API problem, because our low-level access is 200-300% faster than SQLite (which Kevin also mentions). It is something we are well aware of and plan to address quickly after 1.0: https://github.com/realm/realm-java/issues/1684

Our own numbers are here: http://www.slideshare.net/ChristianMelchior/realm-building-a... including a link to the code

It's worse than just that. They claim it's a "replacement" (!) for SQLite. That is an incredibly bold claim. From what I've gathered about Realm it isn't the same type of database as SQLite at all: Object store versus relational tables.

Great link, and the presentation slides are great. Thanks for sharing.

Realm is still pretty fast. A simple comparison of Realm with Swift libraries for SQLite: https://github.com/groue/GRDB.swift/wiki/Performance

my main frustration with realm is that the design for the rxjava support is not that great. I can't use custom schedulers when fetching data and almost always end up mapping realm data objects to Pojo immutable entities generated with Autovalue, thus giving up the lazy loading speedup (since I have to copy all the data). also sometimes the asObservable() method returns empty collections a few times before returning the whole collection EVEN IF FILTERED WITH .isLoaded() and is wayyy too much spammy (it redelivers to subscribers a whole lot more times than when the underlying model is written to), therefore I have to debounce the stream with 1 second to save the adapter from rebinding the whole dataset again.

Been using Realm in all of my Android projects for the last 8+ months. It has now become my defacto mobile database for all my mobile projects. It's that easy to use. Gets out of the way and lets me focus on business problems, not bit shifting problems.

Just curious, why is there no Realm support for web & mobile web using localstorage? It seems like that could be useful, to re-use your model/data tier between react native and a react web app.

That would be really neat, but the core technology of Realm is written in native code. Somebody in the community could certainly create a Realm-like API for web local storage. If you (or anyone else reading this) is interested in building it, I would suggest building it on top of localForage (https://github.com/mozilla/localforage), and definitely hit me up if you'd like to further discuss implementation details!

As I understand it Realm is an Object database. It's similar to the GemStone Smalltalk database in that regard but only local to the device.

How does Realm tackle data syncing between devices ?

Probably left to the developer and their server to re-populate the data, as needed, in the other devices

Does Realm support NSFetchedResultsController-like functionality ?

I'm currently starting work on an iOS application which will display large lists of data which will be constantly updated on a background thread...

I wonder how Realm implements lazy-loading / faulting of off-screen items .. ?

Yes, see https://realm.io/docs/swift/latest/#collection-notifications for details on how you can watch query results for changes to update your TableViews.

All elements of Realm (queries, lists and even individual properties on objects) are lazy loaded. That is an intrinsic property of the zero-copy architecture where objects directly reflect what is in the underlying database.

Thank you, I'm going to try and use it in my project then.

So, not a whole lot is known durability. Any pointers?

https://www.google.com/search?q=site%3Arealm.io%20durability...

If you're into React Native, check out Realm. Great experience for me so far.

I'm using Realm for my React Native project it awesome! Thanks Realm team.

Likewise. Really making me happy so far.

Is react native going to be 1.0 shortly? Or is that not included in this release?

Chris from Realm here - we're definitely working toward 1.0 for React Native as well, but it won't be part of this release :).

Well I read the front page of the website which says that Realm is a replacement for sqlite and coredata.

I didn't gain any understanding though of why they needed to be replaced and how Realm is different.

Is having to use the Realm objects in the same thread they were created it much of a burden on how you use it? Having not tried it yet I wondered about that limitation.

(JP from Realm here)

It certainly forces you to think about threading (which is a good thing IMO). I just responded to a similar comment[0], and there are always our docs on threading[1] which I'd encourage you to read.

[0]: https://news.ycombinator.com/item?id=11771558 [1]: https://realm.io/docs/swift/latest#threading

Really wish there was a C interface to tinker with.

Realm's team, congrats for the good work.

I use Realm for React Native and it's awesome! Congrats to the Realm Team. great work!

Awesome! Congrats to Realm and their entire team!

Will we see some NativeScript support?

We're considering it depending on the level of interest. You can vote on it in this GitHub issue: https://github.com/realm/realm-js/issues/428

for react native, how does realm compare to relay?

Realm is for local storage on the device, whereas Relay is really for getting data from cloud storage.

The "What is Realm?" paragraph begins with "Realm is not an ORM".

Jesus Christ, that was the very first line.

Well thats the reason I asked to elaborate on how they are different.

One is an ORM, the other is not. If you want I can explain to you the difference between airplanes and horses, u interested?

Your comments in this thread break the HN guidelines by being uncivil. We ban accounts that do this, so please (re-)read the site guidelines and only post civilly and substantively from now on: https://news.ycombinator.com/newsguidelines.html

We detached this subthread from https://news.ycombinator.com/item?id=11769795 and marked it off-topic.

In order to use this, you have to have know SQLite, given that, why would I spend time to learn something that wraps around it?

Most of the time you do a insert/update/view, something that a simple helper class would do well.

> Realm is not an ORM, and is not built on top of SQLite. Instead we’ve built a full database for mobile app developers, one that uses native objects that are dynamically mapped to a full, custom database engine (not just a key-value store).

> In order to use this, you have to have know SQLite, given that, why would I spend time to learn something that wraps around it?

The linked article explicitly says it's not a wrapper around SQLite - what gave you that impression?

> Realm is not an ORM, and is not built on top of SQLite. Instead we’ve built a full database for mobile app developers, one that uses native objects that are dynamically mapped to a full, custom database engine (not just a key-value store)

Reply on news.ycombinator.com