emileifrem

Off the top of my head, I know of two big and complex pricing engine projects in production. One is the Marriott one I referenced in a previous post. Scott Grimes spoke about it at GraphConnect 2016:

   https://www.youtube.com/watch?v=wdFYEuWWpzo

The other one is confidential but is in the airline industry.

Indeed, and Neo4j has scaled out horizontally for reads since 2011.

However, the really hard problem with scaling graphs is scaling writes, i.e. partitioning. In my mind, no one has solved that well today because you can't just slap it on top of a partitioning algorithm that's designed for data without explicit relationships (think documents, key value pairs). In order to gain something other than a checkbox feature and "sharding claims" you have to partition based on the shape of the graph at the time of insertion, but also revise it continuously as the graph evolves over time. That's a non-trivial problem that no one has solved today. (Yes, we're obviously working on it here at Neo4j.)

The good news is that you can get really really far with the replicated horizontal scale out model. "Big RAM is growing faster than big data" as you've probably heard and today there are massive Neo4j deployments in production using our third generation scale out architecture (Raft based, multi clustering, causal consistency).

Feel free to have a look again and let me know your thoughts. emil@neo4j.com or @emileifrem.

20 of the 25 biggest banks in the world use Neo4j today, and fraud detection (specifically fraud rings) is a really popular use case. (So much so that if you search for "fraud detection" in an incog chrome window, https://neo4j.com/use-cases/fraud-detection/ is on the first page of google).

Btw, we've scaled out horizontally for reads since 2011 but our initial Paxos based implementation was shaky. Since then we've revised it twice and our third generation Raft based architecture is significantly better. If you add multi clustering support + causal consistency and a bunch of other improvements I think our horizontal scalability story is very different today from a few years ago. Lemme know if you try it out and disagree!

Oh, there's a bunch of stuff in addition to that. I could give it a shot, but would probably mess it up. :) Way better if you hear it from the horse's mouth. Here's Scott Grimes from Marriott at GraphConnect 2016:

https://www.youtube.com/watch?v=wdFYEuWWpzo

Pretty amazing stats in there!

Hey Mark. Yours was the first ever book on AI that I read back in the 90s so thank you for that. :)

I think we're in the early parts of an exciting journey connecting (no pun intended) graphs and AI. I'm personally really excited about connected feature extraction. I wrote a little bit about it here: https://neo4j.com/emil/80-million-series-e/ There's more in depth info here in this graphs & AI overview video from GraphConnect last month: https://neo4j.com/graphconnect-2018/session-topics/?topic=AI...

We just did our annual conference GraphConnect in NYC last month. There's a BUNCH of talks online of people who are very successfully and very happily using it in production deployments. Check it out here: https://neo4j.com/graphconnect-2018

Btw, if you've ever stayed at a Marriott (or anyone they now own: Starwood, Hilton, Ritz Carlton) then you've used Neo4j in production (to book that room). If you've ever purchased a flight ticket, then you've used Neo4j in production (over 99% of all fare calculations are done with Neo4j). Etc.

I hear you. We're working on it. I will note that we have a startup program where you can get Neo4j Enterprise for free if you're below 50 employees and also have several other programs for educational institutions, journalists, etc. And obviously the long term solution here is a DBaaS with grow-as-you-grow pricing. Watch this space! :)

That sucks. :( Sorry about that. Neo4j isn't perfect today and it certainly wasn't perfect 4-5 years ago. We're working hard on it tho!

And thanks for being specific (amazed that you remember specific issues from five years ago!). I don't remember the 127.0.0.1 security problems, but I don't hear anything about them so my guess is they've been addressed. We have a lot of finance and government customers that have high requirements on security. As for your Lucene issues, we did a complete overhaul of our search and indexing story in Neo4j 2.0 (released late 2013). We've continuously improved import performance (which has traditionally been a weak spot) and Neo4j 2.2 includes a batch importer which injects >1M records / sec sustained pace at scale (10s of billions of records) on commodity hardware. As for the memory management issues, we like many other data products written in Java struggled with GC for a long time, and like many others we ultimately concluded that we had to move a lot of the critical parts off heap / manage the memory ourselves, which significantly improved memory utilization.

I understand that you got stung historically and therefore hesitate to check us out again. And if SQLite is working well for you, there's no need to! But Neo4j and the graph space has matured a LOT since 2010 and fortunately I don't think your "bleeding edge" experience from 4-5 years ago will be replicated anymore for someone coming new into the space.

Thanks for the feedback.

I'm curious, what is lacking from Neo4j to qualify it as a mature graph database?

We've worked on it for over a decade, it's used in production by thousands of community users, hundreds of customers and 75+ Global 2000 companies (see http://neo4j.com/customers). For many of those Neo4j is used in business critical use cases, i.e. they require Neo4j to be up and running every minute of every day or it'll show up in their next earnings call. If you've shopped online or in a US retail store this week for example, it's very likely that you've used Neo4j. There's rich support for pretty much any programming language and framework out there, an ecosystem of consulting partners whose sole business it is to do Neo4j implementations, 10+ books written specifically about Neo4j, rich online training, formal enterprise support backed by a global commercial organization, an active community. What's missing?

I'm not trying to be facetious -- I'm genuinely curious as to what you feel is missing to consider it mature.

I can't speak about Titan, but Neo4j HQ is in San Mateo and I'm more than happy to buy anyone lunch or caffeine and discuss graph databases. Neo4j is not the right choice for every situation. But according to most metrics (e.g. http://db-engines.com/en/ranking/graph+dbms) it's around 8x more widely deployed than any other graph db, so we do have a lot of accumulated practical know-how on what works and doesn't when it comes to running graph databases in prod. I'm emil at neotechnology, just drop me a note if you're interested.

For more background info on what's holding back NOSQL in the enterprise and why we raised $10M+, feel free to check out this post:

http://blogs.neotechnology.com/emil/2011/09/nosql-the-web-an...

Re: permissive, I concur! Poor wording on our part. Permissive-r than AGPL perhaps was our thinking. :) But you're absolutely correct that the GPL is not a permissive license. Updated the post!

As for GPLv2 vs v3, in a high-level post like this they're reasonably identical. v3 brings in some nice subtleties like ASL compatibility (a biggie for us!) as well as better patent language. But in terms of use cases where the "virality" kicks in, they're the same.

The main feature (software-wise) in Neo4j Enterprise is High Availability clustering support, which gives you fault tolerance and read scalability. On top of that Neo4j Enterprise also comes with 24/7 enterprise-grade commercial support. You can run it on one core, or 768 cores all for the same price.

Please note that this post talks about Neo4j Community. Neo4j is now split into three editions:

* Neo4j Community, a high performance, fully ACID transactional graph database – licensed under the GPL

* Neo4j Advanced, adding advanced monitoring – licensed under the AGPL or a commercial license

* Neo4j Enterprise, adding online backup and High Availability clustering – licensed under AGPL or commercial

See more in the 1.3 announcement (http://blog.neo4j.org/2011/04/neo4j-13-abisko-lampa-released...) and our licensing guide (http://neo4j.org/licensing-guide/).

How can we improve? We certainly don't want to hide our choice of licensing. Currently it's at the top of the http://neo4j.org page, immediately after the introductory description of a what a graph database is:

"Neo4j is released under a dual free software/commercial license model (which basically means that it’s open source under the AGPLv3 but if you’re interested in using it with closed source software, then you must buy a commercial license)." (http://neo4j.org)

Also, we have an AGPLv3 logo on our web page, there's a LICENSE.txt in our release tarballs, we mention it at the end of every presentation we give, and Neo4j is tagged with AGPL on sites like Freshmeat (http://freshmeat.net/projects/neo4j).

Please let me know how we can make it easier to find!

-EE

Well, the data model seems very similar to Redis' from first glances [1], but FlockDB certainly seems to have completely different durability characteristics. So even if they started anew today they may end up building their own.

1] Which would make FlockDB less a graph db and more a key-value store with social network semantics for the values.

-EE [http://neo4j.org]

I agree that it's not apples/apples. From the first few minutes, I think the main strength of Neo4j is the rich ecosystem and functionality on top of it, and the fact that it stores an infinite-levels deep graph. In comparison FlockDB stores one level (e.g. user -> followers). The main strength of FlockDB seems to be that it has built-in distribution, which is something we're working on for Neo4j but it's not yet generally available.

All this of course based on just a quick glance, so I may come back all the wiser and revise my opinion later. :)

-EE [http://neo4j.org]

I haven't looked deeply into this particular project but I agree with you: the hypergraph data model seems to add too much complexity for too little bang compared to a generic graph model. The Neo4j graph database:

http://neo4j.org

is based on generic property graphs. We sometimes get requests for hypergraph features, but we've found that the real-world problems are always solvable through a normal graph. Either way, it's very nice to see more open source graph databases out there!

-EE

For the Neo4j graph database, check out:

* the 10 minute code-level getting started guide: http://wiki.neo4j.org/content/Getting_Started_Guide

* a full-blown reference example, including diagrams, graph layout screen shots, code walk through etc: http://wiki.neo4j.org/content/IMDB_Example

* the domain modeling gallery, which shows how to model a bunch of domains in a graph db: http://wiki.neo4j.org/content/Domain_Modeling_Gallery

* popular blog post about how to model a social network in Neo4j: http://blog.neo4j.org/2009/09/social-networks-in-database-us...

* how to work Neo4j in python, ruby, clojure, scala & grails: http://wiki.neo4j.org/content/Main_Page#Language_bindings

If you have more questions, feel free go all mailing list on us: https://lists.neo4j.org.

Hope that helps!

In Scandinavia, the typical length of a VC fund is actually 10 years with a standard option to extend for 2 years. First half (5 years) for new investments, second half for followups and then 2 years for padding (option to be exercised at the management company's discretion, not the LPs). I would assume a similar setup is common overseas.

So if you're part of an investment out of a young fund then you typically have at least a 7 years time horizon. Obviously things have to go well in the meantime or they won't sustain you for that long. But the difference between a Marc Andreesen VC and any other VC is def not the length of the fund.

I'm part of the Neo4j team and I'm puzzled about the import problem. I don't know about the size requirements you have but you mention 2.5M nodes and 60M edges and we run systems in production with a LOT more data (billions range). So it definitely shouldn't blow up. Maybe you ran into some bug in an older release or something else was wrong.

It's also important to note that Neo4j through the normal API is optimized for the most common use cases: reading data and transactional updates. Those operations are executed all the time during normal operation, whereas an import is typically done once at system bootstrap and then never again.

To ease migration, as part of our 1.0 release (June time frame) we will expose a new "batch injection" API that is faster for one-time imports of data sets. This is currently being developed. If you have feedback on how an API like that should behave, feel free to join the discussions on the list:

   http://neo4j.org

Cheers,

-EE