Comment by osivertsson | Hacker News Reader

Comment by osivertsson | Hacker News Reader

Parent Full thread

From the top of my head, these are some areas I often would like other system programmers to know better:

- Understanding how to write purely event-based programs using e.g. epoll since that will avoid lots of complexity and potential problems compared to throwing threads at everything.

- Floating point representation and gotchas ("What Every Computer Scientist Should Know About Floating-Point").

- Unit-testing as part of designing interfaces to get low coupling, sane ergonomics, and forcing some thought to error handling instead of just working in the happy-path case.

- Interrupt- and thread-safety (races, mutexes, deadlocks, orderings etc.)

- Storing state in maintainable ways (i.e. not just dumping C-structs to disk).

- Understanding basic computer architecture concepts like virtual memory, caches, and pipelines.

- Know what can be achieved with OpenMP and GPU programming using OpenCL/CUDA etc.

- Write great commit messages (What is the problem being solved, why this way, etc.)

- Basics of database and distributed systems theory like ACID properties and Lamport timestamps.

- Using regular expressions instead of lots of a complicated set of if-statements.

- Understanding networks regarding error sources and latencies.

Maybe it is just me, but I've seen this issue multiple times where a programmer used a regular expression thinking they were clever, only to have it backfire later when there was some corner case not covered by their regex (which likely would have obviously been caught if they had just taken the time to write out each case as an if statement). You're usually just gaining complexity in exchange for fewer lines of code, and I'm not sure that is always the best tradeoff. Something to keep in mind when deciding whether or not to use regular expressions.

With that being said, your list of things to know are all things I have to know for my job (these are things almost all programmers should know, in fact) and I would not consider myself a low-level or systems programmer, just an application programmer.

A good in-between option is a parser generator like Ragel ,http://www.colm.net/open-source/ragel/

This takes in the definition of a regular language as a set of regular expressions, and generates C code for finite state machine to parse the language. You can visualise the state machine in Graphviz to manually verify all paths, making it much easier to spot hidden corner cases, while being a lot quicker to code than a big pile of if statements.

Been using ragel for a while now, it’s awesome. Ragel is a bit like the vim of parsing: the learning curve can be pretty steep, but once you get it, you’ll be the parser magician. Parsing in general just becomes pretty easy with ragel.

Regardless of implementation (regex vs. conditionals), there should be sufficient unittesting to make sure that all the corner cases are tested. For embedded systems, the complexity tradeoff is relevant though, since a regex library will (probably) take up more code space than some conditionals.

In principle, I agree. In practice, it's easier to miss an edge or corner case in a regular expression than it is in a series of conditionals. That's just another consequence of the complexity tradeoff.

Just use re2c. There's no library.

There’s a fine line between being clever and being a smartass.

This looks more like what an Application programmer needs to know. A system programmer writes these api like epoll.

The only thing I would add to this list is security. Understanding how low level code can be exploit and how to code defensively to insure you don't cause bad things. Also the basics about cryptography. It isn't enough to just use a library that implements encryption you must know what and why. You be surprise how much software use encryption that is fundamentally broken causing it being useless to even use.

I would expand on your last point. Latencies all programmers should know: https://gist.github.com/jboner/2841832

One event driven programming framework to check out is Quantum Leaps QP framework, it models state machines easily.

The other points are solid, although I have to admit I have never used regular expressions on a microcontroller

I agree with you on everything except regular expressions. I would hope a system programmer would move beyond regular expressions towards safe, linear complexity binary parsers.

And I would also add fuzz-testing to your list.

Regular expressions isn’t the issue, it’s the libraries. Regular expressions are, mathematically, the quickest and safest way to parse text, and they are linear in complexity. But most libraries aren’t...

Are you sure about your epoll statement concerning complexity? 1 thread per client connection that blocks is as simple as it gets to reason about in my experience, and if you don't care about the stack space the thread occupies (mostly virtual anyways) your contemporary Linux kernel handles lots of threads very well.

Sure, there is no complexity to speak of when your threads do not need to cooperate nor share any data.

But then someone has a bright idea that e.g. a global cache to share some state between all threads would be a good optimization, or that threads need to be able to store some global settings, etc. and complexity related to threads start to creep in to your application.

Reply on news.ycombinator.com