Comment by e12e | Hacker News Reader

Comment by e12e | Hacker News Reader

Parent Full thread

This is actually the first real project I've come across that uses neo4j - thank you for pointing that out.

Given the requirements, I actually think this project looks rather clean between the docker-compose-file and the npm dependencies.

I very much doubt it can compete with Mattermost or Rocket.chat for self-host for small organizations - but I'm really happy to see this thing open sourced!

I much prefer it to be opened up as soon as possible, and then flicked on later.

If there's one thing I'd like to see (beyond a reduction in what appears to be a default of three? mongodb instances) - it would probably be a set of juju charms for bringing the thing up under lxc/lxd on Ubuntu.

I just revisited the combination of juju+ubuntu+lxd - and while I was skeptical when juju was introduced, it appears to be a rather sane toolkit for automating deployment IMNHO.

For those interested, if you start here:

https://insights.ubuntu.com/2016/06/08/lxd-2-0-lxd-and-juju/

you might think that this thing is insane overkill - but if you start at the beginning of the series[s], you will (or I did) get a warm fuzzy feeling about how much fun it is to play with lxd on top of zfs. I suppose some people think docker is just as fun (and warm and fuzzy) - but I don't know. LXD does feel like it combines the good parts of linux with the good parts of bsd jails.

[s] https://insights.ubuntu.com/2016/03/14/the-lxd-2-0-story-pro...

(Gitter Co-founder here)

Unfortunately our neo4j setup as it stands really doesn't scale. At an application-level we're built error-handling to gracefully handle the frequent outages we experience running it.

These are probably our fault, rather than a failing in the product, but I've wanted to replace the current neo4j-based suggestion algo with a new one that uses batching/clustering for a while. As soon as I get a chance, I would like to remove the dependency we have on neo4j.

You might want to reach out to Al Baker or Kendall Clark at Stardog. Their product is more RDF centric than for generic graphs like Neo4j, but is in my opinion a much better product. They have a community version, and have open sourced several pieces, but it is commercial. https://www.stardog.com/docs/

If you want pure Open Source then Titan might be a good fit for Gitter. http://titan.thinkaurelius.com/

The big problem with the way we use neo4j is around huge rooms with ~100K users. The number of possible rooms that neo4j has to traverse (even a shallow traversal) for a user in a big room is huge.

Thanks for your recommendations. I'll definitely check them out.

Those huge rooms really don't add anything to recommendations (like everyone buys milk), if almost everyone is in the room you can also just leave it off (the same is true for rooms with very few users).

That's why in recommendations, I usually filter them out b degree (which is a constant value read).

MATCH (r:Room) WHERE size( (r)<-[:MEMBER]-() ) < 5000) ...

You're also not using directions in your queries, which will also help.

I looked at your other queries, they can also be improved easily, if you want to I can help you do that. Just need some sample data or a read-only access to the db. Just ping me at michael@neo4j.com.

It looks like you're still using Neo4j 2.3. Have you tried using newer versions of Neo4j? There have been some pretty substantial performance improvements.

Additionally, Neo4j does support clustering with their Enterprise edition. It also has much better tooling (better metrics/logging, backups, etc.). It is AGPL licensed, so there's no reason not to use it really.

you should really look at Janus (with Cassandra as the backing datastore). http://janusgraph.org/

Since it uses Gremlin, you should not have that big a problem in porting.

Why not get in touch with us and let us help you instead of dropping us?

I'm totally open to that.

I'm not going to be changing anything around the suggestions any time soon, but who should I get in touch with when I do?

Email me: max.demarzi@neotechnology.com

Reply on news.ycombinator.com