Hacker News Reader

Comment by deanCommie | Hacker News Reader

It shouldn't feel wrong to you. In today's connected big-data world, the vast majority of your performance latency is from querying your data (if you structure your db incorrectly), followed closely by client to server communication.

That's the p90 use case for development people are doing.

View on HN

So many simple things are only easy if you know bits. Interpreting a packet capture, poking at memory, designing cache friendly data structures, ... It is like second nature to me. If it isn't required (obviously it isn't, it must at least be a strong competitive advantage). I am not that old. I don't have get off my lawn moments. I grew up (really learned at least) on a processor (P133) where bits started mattering less, but they still mattered.

What are some good resources to learn about bits?

There are some quite good pages, I don't remember the exact titles, but the first results are quite ok: https://www.google.de/search?q=bit%20hacks

You can write mostly C code. Each processor will have different quirks.

https://en.wikipedia.org/wiki/CPU_cache

https://www.akkadia.org/drepper/cpumemory.pdf

These are great places to start.

Especially Ulrich Drepper's PDF on memory.

edit: even simple things like looping can require knowledge of "bits" and "memory layout" and "CPU cache" to write the fastest code. Most people don't need to write the fastest code though.

Never needed to interpret a packet, I intercept communications at the 7th layer to troubleshoot things - namely http connections.

I don't poke at memory, I use a profiler that tells me what every piece of code and variables I use take in memory.

I don't design cache friendly data structures, I use redis.

I would wager that my experience is closer to the development situation and needs of the majority of programmers.

I don't see why knowledge of bit operations are necessary to do a good programming job.

These are good points. At some point architecture and systems like Redis will become the only tools that can reliably implement these behaviors if no one ever learns about bits. Similar to how hard it can be to outperform compilers in many areas when writing assembly by hand.

Then again there are programming disciplines where you have to know all of this and more. It ultimately comes back to those capable of implementing the tools/frameworks/compilers and those that just use them. Safe crypto libraries are impossible without bit wrangling.

I will still argue that having these skills has at times saved order of magnitudes of time for me. Maybe it wasn't worth all the effort I spent if it only saves time once in a while. Then again I haven't been a full time developer in like 11 years, now I break other peoples software for a living and bang bits together on a weekly basis :)

So you're looking at a http request/response pair, from a service that you did not design, and somewhere in one of the headers you spot a string that looks like this:

YmFzZTY0IGFsd2F5cyB0ZW5kcyB0byBsb29rIGtpbmQgb2YgbGlrZSB0aGlz

Some people will immediately know what to do with that string in the blink of an eye, and the reason why they can spot that with a single glance has very much to do with basic knowledge about bit manipulations.

> I don't see why knowledge of bit operations are necessary to do a good programming job.

The programmer that can't do the above will be stuck. Probably they can still do a good programming job regardless, but the programmer that can is a better programmer.

Knowledge that is related to your practical work but not technically part of it, yet it is part of the field of your work. That is very useful because it makes you better and more well-rounded, meaning you have more mental tools to tackle unforeseen problems. It makes you more well-rounded and better at your job.

If you only know the things you strictly need to know for your job, you're going to get stuck as soon as you encounter a problem that requires novel thinking. And I dunno, I also consider that ability to be part of a "real job".

These design|use patterns depend entirely on a level of abstraction facilitated by code you don't understand and I have seen this bite people badly.

Using redis or memcache or whatever semi-persistent , network available data store for key value pairs only has middling value (to me) and relies on adopting a data model that is severely limited in scope. If you can't understand the structure of a 'packet' on the network then you can't really troubleshoot network dependent services that provide your L7 function.

What's a "p90 use case"?

90th percentile use case

Ha I thought it was related to P90X somehow

https://en.wikipedia.org/wiki/P90X

Reply on news.ycombinator.com