Build Your Own Redis with C/C++
build-your-own.org
build-your-own.org
- Be asynchronous and based on Boost.Asio.
- Perform automatic command pipelining [1]. All C++ libraries I looked at the time would open new connections for that, which results in unacceptable performance losses.
- Parse Redis responses directly in their final data structures avoiding extra copies. This is useful for example to store json strings in Redis and read them back efficiently.
With time I built more performance features
- Full duplex communication.
- Support for RESP3 and server pushes on the same connection that is being used for request/response.
- Event demultiplexing: It can server thousands of requests (e.g. websocket sessions) on a single connection to Redis with back pressure. This is important to keep the number of connections to Redis low and avoid latency introduced by countermeasures like [3].
This client was proposed and accepted in Boost (but not yet integrated), interested readers can find links to the review here [2].
[1] https://redis.io/docs/manual/pipelining/
In fact, I find that fitting my code into patterns that work with redis actively improve these aspects of my own code.
If I had a monkey wrench into redis (beyond lua), I imagine one or more of these aspects would rapidly deteriorate.
https://github.com/gabrieledarrigo/ducky
I did choose UDP for the sake of simplicity, and I implemented a very naive protocol, using select to handle I/O. I also remember that In tried various benchmarks, but the hash implementation that I used failed after 500k inserts or something. Now, I started to program quite late (after 25 years old), and I mostly do web development, and even if I'm a complete C noob, it was a super funny project! And if someone would like to take a look at the code and share with me an opinion, it would be great!
It is a DB in the first place so "Learn network programming [by writing Redis]" is kind of questionable.
https://gist.github.com/antirez/6ca04dd191bdb82aad9fb241013e...
> There are several Linux system calls we need to learn before we can start socket programming.
Have we reached the state where Linux by default is New Unix/New POSIX ? Isn't it alarming for say FreeBSD guys?
there is NO SUCH THING
Looks like they meant C.
So is it C or C++?
However programming a Redis clone in C/C++ is definitly not something that happens, it is either C or C++, and if it is in the common subset compiled by compilers of both languages, there are definitly better ways to express it.
I do. Who's to say if you use C++, thou shalt always use: smart pointers, move semantics, OOP, templates, and more. Also, you're discounting a very large and very real industry of people writing code for embedded hardware that doesn't necessarily have the luxurious memory resources modern devices have.
Why could I ever possibly want to use C++ if I don't want to use smart pointers and OOP and all that other stuff? Well, generic type safety is very nice. Templates don't exist in C. Constexpr and references are also quite nice. They allow me to annotate my code to make my intentions clearer. Operator overloading is also insanely helpful at times. Matrix/vector math is so much nicer to write/read using mathematical operators instead of function calls everywhere. This also isn't possible in C.
Anyways, don't go making claims that if somebody doesn't use a language exactly the same way you do, then they're using it wrong. Especially when it's a language as massively overengineered as C++. There are plenty of reasons to prefer a subset of C++ to C, and plenty of valid reasons to prefer to use C instead of dealing with the complexities of C++.
In the C++ world I live/work Boost is avoided like the plague. I always wondered how Boost continues to exist, but it seems in certain communities it's held up as a standard to strive for.
Boost continues to exist because there is large overlap with the standards committee and they use it as a staging ground to try out new ideas before they are standardized.
What’s the joke? I don’t get it.
Someone on reddit joked that Rust is becoming the new Crossfit meme;
"How can you tell when someone programs in Rust?" "Don't worry, they'll tell you."
I’m really not big fan of Rust, but I love that’s it’s always looking over my shoulder for stupid mistakes.
What are you building with Rust?
I’m Rust I’m found various side projects. An all in one PiHole like dns server, a distributed key value store based on raft and an openwrt module.
But nothing interesting or revolutionary enough to share.
How about you?
Except DJB, of course.... ;-)
> "This claim is denied. Nobody gives gigabytes of memory to each qmail-smtpd process, so there is no problem with qmail's assumption that allocated array lengths fit comfortably into 32 bits."
Years later it turns out that well, "nobody" was lots of people. Not DJB of course, DJB can claim that DJB's software works exactly the way DJB intended, but for everybody else not so much.
I maintain a very large C++ project used to run airlines around the world that hasn’t had a bug in production for 5+ years. No crashes. No memory leaks. No production shutdown. Not even once. So no it most definitely is possible to write solid code in C++. However I do agree that it is hard :) I managed to do it by having 9000+ tests testing all aspects of the code.
Another (more public) example is SQLite. I am sure there are more examples out there. Airbus avionics software is written in C for example.
List of SQLLite exploits,
https://www.cvedetails.com/vulnerability-list/vendor_id-9237...
But if I wanted to let loose these days and write some code without worrying about the safety that rust has to offer, I’d reach for Zig. It seems to solve a lot of C’s problems while being a much cleaner, well thought through language for low level code. Im a fan!
I would recommend doing C at some point in your career, nothing comes close in terms of forcing you to understand the hardware you are using.
Edit: I just saw your edited comment. Yeah the point is you really don't need to know much about the hardware to write C. It doesn't force you to understand what's actually going on behind the scenes.
The point the OP was making is that it still forces you to learn more about the hardware than other languages. Arguably, it also forces you to learn more about the software itself with things being much more explicit than in a modern complex language such as Rust. I'm also just not aware of _any_ language that actually maps well to what the CPU is doing nowadays since they are such complex pieces of silicon.
C has memory pointers, but those are pointers in a flat memory space abstracted over the physical memory hierarchy.
So yes, you do need to understand that the hardware has memory addresses and that pointers can reference memory addresses. Aside from that I don't see much difference to Java or Python in terms of requiring a deeper understanding. Even Python has bitwise operators.
The big difference would be that it requires you to understand the difference between stack vs heap allocation.
So thinking in terms of the abstract machine are valid for now. The exceptions mostly have to do with caching and understanding that processor can and does execute short sequences of instructions in parallel.
1. Modern CPUs use instruction level parallelism (ILP)
2. Memory isn't linear (you have separate caches, L1, L2, L3 and main memory)
If you've ever debugged a C or C++ program in release, you've quickly found out about ILP. The code still maps relatively close to the hardware, it just won't run in the sequential order you've provided it in. Many C and C++ programmers know this and try to make the implicit assumptions in their code explicit to allow the compiler to reorder instructions more easily/reduce memory dependency chains[0].
And I'm sure you've heard several proponents over the past few years (Mike Acton comes to mind) espousing data oriented design. This is an entire code methodology intended to help the CPU caches, and it shows an implicit understanding that memory is not linear. Heck, most C/C++ programmers realize that they're using VRAM all the time, and the memory locations they get aren't necessarily backed by physical memory until the OS sorts it out. This is especially transparent if you've ever done any sort of memmapping with files or played with virtual memory.
Anyways, C doesn't necessarily map directly to the hardware, but it's a heck of a lot easier to intuit what a C program will end up doing on the CPU vs what a Python program will do. And most C/C++ programmers realize this fact, and actively write code to utilize the fact that C does not map directly to hardware.
[0]: https://johnnysswlab.com/instruction-level-parallelism-in-pr...
When I was teaching programming, it was always a rite of passage for my students to implement a linked list in C. Once it clicked for them, the world of programming opened up.
C is also still the universal glue language for FFI. Wanna call Swift from Rust? You can always reach for C.
If the author actually can't understand the difference between C and C++, there's a pretty low chance of any good code being in the book. If they do, they should keep an eye on the editor next time they go into empty marketing mode for the title.
Not a typo for C#. He really wanted someone with experience in C+
Sounds fair to me
https://news.ycombinator.com/newsguidelines.html
"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead."