Redcon – Fast Redis-compatible server framework for Go | Hacker News Reader

Redcon – Fast Redis-compatible server framework for Go

github.com

105 pointstidwall49 comments

Note that it's a redis serialization protocol and tcp server implementation. It's up to you to define redis commands such as get, set etc.

This can be used coupled with a backend storage to provide custom redis servers.

It's not a redis implementation by itself.

I don't see how it's useful then. Maybe someone can give me a good use case?

redis is already very bare-bones and dead simple to set up and maintain, what more does one want? Wouldn't adding more logic into a redis-like environment be a little cumbersome from an architecture perspective?

Could be used for proxying redis connections, be it for clustering or I don't know, add sophisticated access control or something.

You could expose a memcached cluster as a redis server using this. (Not sure why would you want to do that.. Maybe if you have an existing memcached cluster and have redis client code?)

Or you could go crazy and implement the entire set of redis commands on top of sqlite and call it a "strongly persistent redis" or something like that.

It's just a nice building block, I'm sure people would find ways to abuse it :)

I'm not sure, either. By the time you've implemented all the actual Redis functions, there's no telling whether the server will be more or less performant than Redis itself. (Most likely less, since Redis is written in optimized C.)

Of course the existing benchmark shows that the example Go implementation is faster. For one, the Go implementation provided avoids a lot of functionality that the Redis server implements, such as TTL checking.

This is not for recreating Redis, but for custom server implementations such as the Tile38 project, proxying services, and microservices that might need to use the RESP protocol.

Then please don't call it a "Redis server implementation." That is misleading, at the very least. A "framework" would be an accurate description.

Thanks a bunch for your feedback. I totally agree and changed the title to "Fast Redis compatible server framework for Go".

Doesn't seem to have taken...

He meant he changed the project description on GitHub. You can't change HN titles once they're posted afaik.

Because it's written in Go of course.

You have a more complicated data storage, or an established one that you don't wanna change, or a mandate from on high that all data must be put in X due to auditing or whatever - and a program that wants to talk to Redis.

The use case would be implementing the redis protocol for client server communication in your own software. There is a specification for using the redis protocol [1]. The fact that redis is widely used and there are a number of client libraries in many languages may make it an attractive option compared to creating your own protocol from scratch.

I believe disque, the distributed message broker [2] uses the redis protocol for client server communication.

[1] http://redis.io/topics/protocol [2] https://github.com/antirez/disque

If you need an easy to write, debug, and operate RPC tech for building networked components with minimal dependencies/effort, writing Go servers with RESP (iirc, REdis Serialization Protocol) is surely the way to go. I've built several of these things (although, with a custom RESP parser), and, from my experience, the ROI is pretty high -- it's up there with bash scripts for getting stuff done.

This is a not a good way to do this kind of thing. Simple stateless redis protocol parser and an example to use it would be a much better interface.

Note that this is not a full reimplementation of Redis in Go, but rather a framework for implementing a Redis-like server in Go.

If what you are looking for is a Redis implementation, I did create a Redis v1 clone in Go a while back. It's is only a proof-of-concept and lead to the development of Redcon.

https://github.com/tidwall/sider

I reimplemented it in Java, including all the commands, with similar results. Mostly useful as a mocking tool during unit tests.

But why: https://github.com/kstyrc/embedded-redis

Because reinventing the wheel is an immense amount of fun, if you get to do it yourself. ;) A good learning experience too.

Everyone should at one point write their own compiler. And maybe their own OS.

(I'm only half kidding here)

That's a good reason.

Someone should tell the JS crowd though, that you shouldn't neccessarily reinvent everything every 6 months and use it in production.

We did tell them, and they went and started to reinvent everything every 3 months.

Sometimes it may worth it. FFI was never fun regardless the host language except perhaps C++ -> C.

Everyone should at least try writing their own OS. Not necessarily their own kernel, or even the lowest-level utilities - things like Fiasco.OC with Genode are decent enough if you're not interested in register fiddling - but their own model for how things could work if they got to design a system for programmers and users to use from scratch.

Capability systems? Network transparent message passing? Dynamic load-balancing of running processes across machines? What if you tried to let users script their workflows in a less ad-hoc way than they can now? How about trying to replace the concept of a filesystem, at the same time as trying to encourage applications to share data with each other?

OS development research died a couple of decades ago, and now we have an entire generation of developers who believe the Linux/POSIX model to be the one true system. But IMHO, that's just a quirk of fate, and a local maxima, and it keeps people in the dark about alternative solutions to their problems. For example, capability system advocates will point out that their system almost implements Docker/containerization by default. There have been tons of experiments with efficient message-passing under microkernels which would likely benefit anyone attempting to design Yet Another IPC Framework in the Linux kernel. And so on and so forth.

I could ask the same of them I suppose, since I wrote mine 2 years before their initial commit.

> Mostly useful as a mocking tool during unit tests.

AFAIK redis has tests, you don't need to rewrite tests for redis in Java. Mocking implies test assertions in the Mock. I'm sure redis driver for Java has tests too. That's what you should mock.

The parent probably means an in-process substitute for actual Redis.

Indeed.

You're not alone, I wrote wire-compatible version of Redis in perl:

https://github.com/skx/predis

Of course I simplified things by only implementing keys and sets, rather than all the operations such as hashes, etc. But allowing pluggable storage back-ends made it useful for my use-case(s).

It's not a server. You can't implement a redis-compatible server in 100 lines of Go code. How did this make it to the front page of HN?

Do we really need one post a day on the frontpage where somebody reimplements something in Go?

Just because the language is really hyped right now?

i would say rust is the new kid on the block, go is old news.

The main problem of rust is that it's not implemented in go. The reverse is also true.

Didn't see a lot of HN posts about "How one rewrote Y from Z to Rust" yet.

Plenty of Rust related posts, though.

My point is, Go posts easily appears on HN homepage, more than anything else. I didn't see a lot of posts about Java or Python there lately.

BTW, where's the guy who replies "What an amazing time to be alive!" to every go post? I'm worried.

Maybe the drugs wore off? :D

Redis reimplemented with Rust

https://news.ycombinator.com/item?id=9741625

Go 1.0 is 4 years old now, after 5 years working on developing what is essentially and was always "C with garbage collection and threading". Rust is a year old, after 6 years of research building something novel outside of academic languages, with the final shape of the language heavily changing in the last year. Rust also doesn't ship with an extensive standard library including nearly all the things you'd want to make a basic web app.

Basically - Go had a very strong focus on web servers from the start, making it easy to pick up and develop web servers with. Web developers make good early adopters - they'll pick up new tools faster than almost any other group of developers. Rust was released without any of those tools, and they're still changing as the ecosystem develops (especially around async frameworks). And even with that, there's a fair few companies using it for all sorts of cool things: https://www.rust-lang.org/en-US/friends.html

And Go is rather pointless to begin with. The only thing it get's right is the excellent microthreads and CSP implementation.

The only reason it gained any traction is the big name of the mothership...

It's ok to dislike the hype but it's disingenuous to say it only gets one thing right.

> The only thing it get's write is the excellent microthreads and CSP implementation.

No at all. It's very far from even remotely decent implementation, the interface is too inconsistent and full of implicit behaviors and the performance is an order of magnitude slower than anything acceptable. But anyone actually using it already knows..

I admit we dropped Go before getting to production, with any kind of performance requirements.

I liked the goroutine related apis though.

What I remember being PAINFULLY slow is reflection. Which is very inconvenient for certain tasks (building objects from db queries, (json) serializing,...). Macros could help here, but alas, code generation is the only alternative available. Which I'm not a fan of.

---

What's are your gripes with Gos CSP?

The reflection is slow indeed but most of the time you can use caching to improve it(at least that was my case). I like Go a lot but I must admit that some features would make my life easier and perhaps the language more expressive too(i.e. variant types). If that is not good enough you can use code generation(last resort)

What it gets right is that it's just a tool not a play field for academics.

I admit to having a personal gripe with Go.

I initially really liked it. But the languages designers just skipped over countless years of achievements in language design and implemented a C with garbage collection, with interfaces and a baked in CSP runtime.

If you read a bit in the groups / release notes, they were just too lazy to implement generics. Or it was too hard.

Go's ecosystem is really weak (for example, barely any library does proper versioning or any release procedure at all).

I think that is at least part due to the difficulty of writing clean, generic code. And the lack of proper packaging.

The lack of features is not a selling point, even if they try.

Easy distribution/static linking is what got me.

Actually I'm a little more annoyed by the Rust and Elixir people. At least the Go stuff is "Hey I did something in Go". The Rust and Elixir stuff is almost always as a response to something someone did in Go, C, C++, Java or some other language, and almost always in the form of: "You might like Rust", "You should check out Elixir".

At least Go can generate it's own hype, and doesn't need to piggyback on others.

Editors: can you please change the title? It's not a "Redis server implementation" at all.

Reply on news.ycombinator.com