Hacker News Reader

Comment by fake-name | Hacker News Reader

Ooooor, maybe you like to think about what's actually going on when you write code?

I certainly basically mentally execute code to some extent as I write it. Is there a way to program that doesn't do something like that?

View on HN

Nobody has any idea what a modern x86 cpu does under the hood because we have no access to the microcode.

The only thing you can do is run your code and get an empirical answer to the question 'Is it fast enough'.

Compilers almost never emit instructions that are microcoded on any modern processor

Please explain what you mean. The microcode is the “firmware” of sorts of the CPU. What instructions we tell the CPU to execute and what the CPU decides to actually do to achieve the same result you are asking for can be different. What do you mean by the instructions not being microcoded?

There is no rule that every instruction has to be implemented with microcode. Simple instructions like adding two numbers don't need it. Microcode is generally used for more complicated instructions, often those with a variable number of cycles. They are also usually not the fastest way to achieve a result.

> What do you mean by the instructions not being microcoded?

Most common instructions in modern CPU cores are decoded directly into the corresponding micro-operation(s), without involving the microcode ROM. Only the most complex instructions, involving many micro-operations, will use the microcode decoder. Also, there are often several copies of the simpler instruction decoder, so the CPU can decode several instructions in a single cycle, while there's normally a single copy of the complex (microcode-using) instruction decoder.

A good resource if you want to read more is part 3 of https://www.agner.org/optimize/ which describes the decoder (and other parts) of several families of x86 processors.

Thanks

A lot of the relevant low level optimizations are documented - like micro and macro op fusion. The published optimization guide is thousands of pages long.

Technically true, but also useless point of view, because it's essentially saying "it's all arbitrary chaos underneath". But it isn't. You can get 90% there with basic understanding of x86 assembly and some CPU-level abstractions.

(I too execute the code in my head, but these days usually a couple layers above x86 ASM, in my own mental "bytecode" that tracks how expensive are some of programming language's operations and stdlib functions.)

Whether relevant things fit into L1 cache will make an order of magnitude more difference than whether you're using a nominally 2-cycle or 4-cycle assembly instruction. I don't believe any human programmer on a modern CPU is able to keep track of what cache level their code operates at entirely in their head; certainly such a human would not be thinking in terms of x86 assembly or normal language stdlib functions. So attempting to execute the code in one's head is overwhelmingly likely to be a waste of time.

Sometimes, you have to.

An example - some code I was working with gave subtly different results on two processors. It turned out this was because one of them implemented FMA - Fused Multiply Add.

https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_op...

It wasn't particularly painful to find this, but if you were attempting to debug this kind of problem without considering lower level issues, you'd spend a whole lot of time banging your head against a wall.

If the explicit fused instruction (so VFMADD..., we're not talking micro-code fusing, are we?) doesn't have the same semantics as a separate multiply and add then it's a compiler bug to select it, isn't it?

I think if you're talking micro-code fusing then Intel does guarantee it has exactly the same semantics as a separate multiply and add.

Interesting fact - I believe modern Intel architectures actually only have fused multiply add. If you do just a multiply it'll do a fused multiply and add zero.

> Is there a way to program that doesn't do something like that?

Logic and relational programming (Prolog, SQL) comes pretty close. "Here's some rules about the data. Execute."

And yet for any but toy-level (and logic/relational math research) programming, you have to pay attention to how it works under the hood.

I remember university classes in Prolog. Solving logical puzzles tended to be declarative, but trying to write any actual program involved shifting to the mindset that you're dealing with a regular programming language with a built-in DFS engine underneath (the same way programming in JS involves learning you have a hidden event loop running in the background). For practical use of SQL, you need to be able to choose between equivalent queries based on how the database engine actually turns them into lookups.

> And yet for any but toy-level (and logic/relational math research) programming, you have to pay attention to how it works under the hood.

Maybe 30 years ago. It's really not so important nowadays. Computers are fast.

> For practical use of SQL, you need to be able to choose between equivalent queries based on how the database engine actually turns them into lookups.

Maybe there are some cases where you need to, but there are also million-dollar businesses that have succeeded while only ever writing their SQL queries as naively as possible. The notion that all non-toy practical uses need to understand such low-level details is just utterly false.

> there are also million-dollar businesses that have succeeded while only ever writing their SQL queries as naively as possible.

Citation needed. Given engineering staff sizes and the amount of performance problems RDBMSes have, that seems highly doubtful.

Personal working experience; afraid I'm not willing to name specific past employers here.

Even then you need to understand the underlying execution model. Prolog has cuts that guide the backtracking engine. SQL performance is hard to optimize without looking at query plans.

If you are working with large data sets and want it fast then yes.

But in regards to "Is there a way to program that doesn't do something like that?", I would say relational is that way.

When writing SQL, I think of "what is the DB engine going to do when I ask this". I don't see that as being substantially different.

You must be psychic if you can figure out what the DB engine is going to do without running it.

Bitmap scan, parallel index scan, merge join, nested loop join, etc. I always have to try and see.

It's not a superpower at all--instead, I'd argue that it's the most important skillset (after understanding the relational model itself) for interacting with RDBMSes at scale.

You don't necessarily have to know whether an index scan is going to be parallelized or whether the CBO is going to switch to a loop because its heuristics think your result set is likely tiny to think about things like "is this data being accessed via the right indexes? Is the data likely in memory or not? Will my joins, given my consistency level, impose locks or concurrency considerations on the tables I expect?"

Oh, I'm not generally correct, but my process of "writing" generally goes:

  - Guess what I think is going to happen  
  - EXPLAIN ANALYZE  
  - Ohhh, it did [thing]. Lemme tweak if I can convince it to do [other thing] that should work better  
  - <Repeat>

I'd argue testing against the DB is a core part of the process for writing a query.

The Python bytecode is not where the work is occurring. The bytecode is just a sentinel for a polymorphic dispatch table. Even if the bytecode were infinitely fast, it might speed up your program by 20%.

Reply on news.ycombinator.com