Is this embedded in the same process, or just within the same unit?
Aside: would an embeddable redis be a useful thing for apps and other isolated devices?
Aside: would an embeddable redis be a useful thing for apps and other isolated devices?
1. Embedded systems are often used in environments where you need very resilient software. To crash the DB because there is a bug in your app is usually a bad idea.
2. As a variation of "1", it's good to have different modules as different processes, and Redis works as a glue (message bus) in that case. So again, all should talk to Redis via a unix socket or alike.
3. Latency is usually very acceptable even for the most demanding applications: when it is not, a common pattern to solve such problem is to write to a buffer from within the embedded process, that a different thread moves to Redis. Anyway if you have Redis latencies of any kind, you don't want to block your embedded app main thread.
4. Redis persistence is not compatible with that approach.
5. Many tried such projects (embedded Redis forks or reimplementations) and nobody cared. There must be a reason.
I use sqlite for this purpose, essentially as an in-memory cache of data populated from disk and incoming server packets. Having redis as an option to replace mysql (or at least to compare memory use and speed) would be great.
I looked for an embedded Redis fork and came up blank, do you have links? I found Vedis, but I would rather have something built off of the Redis code than a re-implementation.
I would be surprised if "actual Redis" was ever the right answer to "sqlite is too much".
But I do wonder if there are some lessons to take from Redis api and wrap something like lmdb/bdb etc.
I'm not familiar enough with Redis to know when/if this would make sense over just using sqlite, though.
Which is topical, because watching for updates is a core feature of Redis streams (and Redis already had pub/sub channels before that). For that use case, SQLite is too little, even if your needs are otherwise quite basic.
Unfortunately, this difference in capabilities seems to be partly a result of limitations in the underlying OS APIs. SQLite uses POSIX advisory locks to lock ranges of the database file, but I don’t think there’s any similar API that provides an event or semaphore associated with a given file, instead of a lock. There are plenty of messaging APIs that aren’t associated with an arbitrary file – there are semaphores, message queues, and shared memory objects, in fact two sets of APIs for each of those (SysV and POSIX), plus signals, etc. But those all have their own namespaces, and if the two processes trying to synchronize with each other are in different containers, they might not share those namespaces. There are Unix sockets – those are a decent option, but they require one process to set itself up as the server, which is a bit weird in the SQLite model where all the processes are on an equal footing, and any may quit at any time. They also don’t work over NFS (whereas locking does, at least sometimes). You can try to mmap a regular file and then treat it as shared memory, but that’s not guaranteed to work in all cases, and again doesn’t work over NFS. I suppose you could try to abuse a lock as a semaphore, but that has its limitations…
But it’s not like many people use SQLite over NFS anyway. Whatever the approach, I’d love to see a “SQLite for notifications”. It would probably be a pretty simple library, but with the needed bells and whistles like bindings to higher level languages. If a library like this exists, I’d be very interested to hear, because a while back I searched for one in vain.
The current use of sqlite is to allow our scripted code (lua and actionscript) to make queries of the exposed data without having to write C++ code for every possible query and data object type (and implement new ones on demand).
Redis might not be the correct thing for this exact use case (some of the queries are more complex than a simple key or range look-up) but I may be prepared to take those limitations in exchange for a substantial speed and/or memory use improvement.
I've used Redis very successfully on the backend, so maybe I'm just trying to find some reason, any reason, to play with it in player facing code!
It may be useful to you as well: RediSLQ.com
Or on GitHub: https://github.com/RedBeardLab/rediSQL
Full disclaimer: I am the author
It is pretty cool to be able to share live interconnected objects between processes with full transactional safety.
For some of those using something as Redis as the existing service might be interesting, for others it will be a no-go.
I worked on automative infotainment system in the past, and throwing Redis on an embedded Linux system there would have been fine if it would have fulfilled a particular task in a good fashion. I think I even proposed it once for something.
My payload are lists of int64's. I need to do set operations on those lists before sending the result over the wire. If you advise against embedding redis, can I instead embed my logic in redis? As a filter of sorts?
Redis feels like that. It’s a simple data structure server. Now if we could have those datastructurs sync with flatfiles with the same redis API, a lot of applications would become much simpler.
I’m not sure how big of an undertaking it is though.
I’m willing to bet, a fast general datasrtuctures database syncable to flat files would open up many possibilities.