Hacker News Reader

Comment by chrisseaton | Hacker News Reader

The author hints at the problem

> Don’t count on your operator overloads to work here, though.

> by keeping these variables around, debugging is more straightforward

The transformations the author is suggesting are not in general legal. Missing operator overloads and inconsistent debugging states aren’t something to gloss over - they’re showing you your optimisations are just wrong!

You would need a sophisticated deoptimisation system with frame states, like JVMs or V8 has, to make them legal.

The Python compiler isn’t dumb - it’s correct.

View on HN

From half way in the 2nd paragraph, it was obvious that the author had been looking at Python all wrong

> I wonder if the code I’m writing is putting undue constraints on the bytecode compiler and limiting its options

If that's what you're wondering while writing Python, then you probably shouldn't be writing Python.

Ooooor, maybe you like to think about what's actually going on when you write code?

I certainly basically mentally execute code to some extent as I write it. Is there a way to program that doesn't do something like that?

Nobody has any idea what a modern x86 cpu does under the hood because we have no access to the microcode.

The only thing you can do is run your code and get an empirical answer to the question 'Is it fast enough'.

Compilers almost never emit instructions that are microcoded on any modern processor

Please explain what you mean. The microcode is the “firmware” of sorts of the CPU. What instructions we tell the CPU to execute and what the CPU decides to actually do to achieve the same result you are asking for can be different. What do you mean by the instructions not being microcoded?

There is no rule that every instruction has to be implemented with microcode. Simple instructions like adding two numbers don't need it. Microcode is generally used for more complicated instructions, often those with a variable number of cycles. They are also usually not the fastest way to achieve a result.

> What do you mean by the instructions not being microcoded?

Most common instructions in modern CPU cores are decoded directly into the corresponding micro-operation(s), without involving the microcode ROM. Only the most complex instructions, involving many micro-operations, will use the microcode decoder. Also, there are often several copies of the simpler instruction decoder, so the CPU can decode several instructions in a single cycle, while there's normally a single copy of the complex (microcode-using) instruction decoder.

A good resource if you want to read more is part 3 of https://www.agner.org/optimize/ which describes the decoder (and other parts) of several families of x86 processors.

Thanks

A lot of the relevant low level optimizations are documented - like micro and macro op fusion. The published optimization guide is thousands of pages long.

Technically true, but also useless point of view, because it's essentially saying "it's all arbitrary chaos underneath". But it isn't. You can get 90% there with basic understanding of x86 assembly and some CPU-level abstractions.

(I too execute the code in my head, but these days usually a couple layers above x86 ASM, in my own mental "bytecode" that tracks how expensive are some of programming language's operations and stdlib functions.)

Whether relevant things fit into L1 cache will make an order of magnitude more difference than whether you're using a nominally 2-cycle or 4-cycle assembly instruction. I don't believe any human programmer on a modern CPU is able to keep track of what cache level their code operates at entirely in their head; certainly such a human would not be thinking in terms of x86 assembly or normal language stdlib functions. So attempting to execute the code in one's head is overwhelmingly likely to be a waste of time.

Sometimes, you have to.

An example - some code I was working with gave subtly different results on two processors. It turned out this was because one of them implemented FMA - Fused Multiply Add.

https://en.wikipedia.org/wiki/Multiply%E2%80%93accumulate_op...

It wasn't particularly painful to find this, but if you were attempting to debug this kind of problem without considering lower level issues, you'd spend a whole lot of time banging your head against a wall.

If the explicit fused instruction (so VFMADD..., we're not talking micro-code fusing, are we?) doesn't have the same semantics as a separate multiply and add then it's a compiler bug to select it, isn't it?

I think if you're talking micro-code fusing then Intel does guarantee it has exactly the same semantics as a separate multiply and add.

Interesting fact - I believe modern Intel architectures actually only have fused multiply add. If you do just a multiply it'll do a fused multiply and add zero.

> Is there a way to program that doesn't do something like that?

Logic and relational programming (Prolog, SQL) comes pretty close. "Here's some rules about the data. Execute."

And yet for any but toy-level (and logic/relational math research) programming, you have to pay attention to how it works under the hood.

I remember university classes in Prolog. Solving logical puzzles tended to be declarative, but trying to write any actual program involved shifting to the mindset that you're dealing with a regular programming language with a built-in DFS engine underneath (the same way programming in JS involves learning you have a hidden event loop running in the background). For practical use of SQL, you need to be able to choose between equivalent queries based on how the database engine actually turns them into lookups.

> And yet for any but toy-level (and logic/relational math research) programming, you have to pay attention to how it works under the hood.

Maybe 30 years ago. It's really not so important nowadays. Computers are fast.

> For practical use of SQL, you need to be able to choose between equivalent queries based on how the database engine actually turns them into lookups.

Maybe there are some cases where you need to, but there are also million-dollar businesses that have succeeded while only ever writing their SQL queries as naively as possible. The notion that all non-toy practical uses need to understand such low-level details is just utterly false.

> there are also million-dollar businesses that have succeeded while only ever writing their SQL queries as naively as possible.

Citation needed. Given engineering staff sizes and the amount of performance problems RDBMSes have, that seems highly doubtful.

Personal working experience; afraid I'm not willing to name specific past employers here.

Even then you need to understand the underlying execution model. Prolog has cuts that guide the backtracking engine. SQL performance is hard to optimize without looking at query plans.

If you are working with large data sets and want it fast then yes.

But in regards to "Is there a way to program that doesn't do something like that?", I would say relational is that way.

When writing SQL, I think of "what is the DB engine going to do when I ask this". I don't see that as being substantially different.

You must be psychic if you can figure out what the DB engine is going to do without running it.

Bitmap scan, parallel index scan, merge join, nested loop join, etc. I always have to try and see.

It's not a superpower at all--instead, I'd argue that it's the most important skillset (after understanding the relational model itself) for interacting with RDBMSes at scale.

You don't necessarily have to know whether an index scan is going to be parallelized or whether the CBO is going to switch to a loop because its heuristics think your result set is likely tiny to think about things like "is this data being accessed via the right indexes? Is the data likely in memory or not? Will my joins, given my consistency level, impose locks or concurrency considerations on the tables I expect?"

Oh, I'm not generally correct, but my process of "writing" generally goes:

  - Guess what I think is going to happen  
  - EXPLAIN ANALYZE  
  - Ohhh, it did [thing]. Lemme tweak if I can convince it to do [other thing] that should work better  
  - <Repeat>

I'd argue testing against the DB is a core part of the process for writing a query.

The Python bytecode is not where the work is occurring. The bytecode is just a sentinel for a polymorphic dispatch table. Even if the bytecode were infinitely fast, it might speed up your program by 20%.

Human curiosity and meeting their tasks' requirements can coexist in this instance (or any).

Also, the author points out in the first sentence that they are subject to external constraints that require Python.

Dumb was not used in a derrogative way in this article.

As a long time Pythonista, I find the article quite balanced actually.

+1. I'm reminded of a tweet of yours from a few years back!

https://twitter.com/chrisgseaton/status/619885182104043520

Try doing something like that at compile time.

... I am not sure I am impressed. The rules for partially evaluating eval are the same for partially evaluating any code. I am just not sure anybody else bothered something like that with eval.

Edit: I have seen Guile's optimizer/peval turn 30 lines of messy (to human eyes) code to a single atom result. This is actually rather simple as long as you van reason enough about the code and it has no side effects. The hard part is developing heuristics to not have it impact compile time too much

> I am not sure I am impressed

My PhD defence committee was impressed, so that was good enough for me.

In Ruby almost no code is refentially transparent so just simple PE is not enough. You need to speculate in some complex ways and writing that to be practical is not easy.

Then I am impressed :) I don't know enough ruby to know exactly why, though. I am just used to seeing this kind of behaviour in guile, and for guile I don't really see why that would be hard, at least if you have enough visibility.

I might have gotten it wrong for guile as well :) sorry to sound so dismissive. I would love to read your thesis!

Common Lisp does that for breakfast :).

I think it might also be a "lazy" solution to a more complex problem, which this post does not address: Python allows nested functions to, IIRC, an unspecified depth (probably limited by something like stack size). I imagine that that given how Python implements variable lookups, this could be problematic with optimizations in that in the face of nested functions and control paths, it would be very difficult (probably not impossible) to prove they are not used.

Reply on news.ycombinator.com