Hacker News Reader

Comment by kylejrp | Hacker News Reader

As part of the Stack Overflow April Fools' prank, we did some data analysis on copy behavior on the site [0]. The most copied answer during the collection period (~1 month) was "How to iterate over rows in a DataFrame in Pandas" [1], receiving 11k copies!

[0] https://stackoverflow.blog/2021/04/19/how-often-do-people-ac...

[1] https://stackoverflow.com/a/16476974/16476924

View on HN

That’s sad, as when you find yourself iterating over rows in pandas you’re almost invariably doing some wrong or very very sub optimally.

To me it's an means to an end. I don't care if my solution takes 100ms instead of 1ms, it's the superior choice for me if it takes me 1 minute to do it instead of 10 minutes to learn something new.

The flaw of human nature on display. And I don't mean that personal, not to you anyway, but to the human species.

Why? It seems very rational. Especially if you're just going to run it once to get a value and not as a part of some system.

Exactly, that's not a flaw, that's rational behaviour. Why design an intricate solution for a one-off. Using 15 times the amount of time it would have taken you manually to automate a pretty standard task is just stupid, though we all do it.

Do the first thing that works, don't overthink it.

Why learn anything at all then? Why bother learning OOP paradigms if procedural just works? Why bother ...? Do you see the flaw in your argument?

Because when it's not a one off?

One doesn’t learn rock climbing to step over a brick, man.

I think the flaw is you misinterpreting the argument.

In my opinion, the bottom line with obvious caveats is this - Human-time is more valuable than CPU-time.

If you are shipping at scale then the calculus is different - Don't waste end-users' human-time and their cpu-time and/or server's cpu-time.

If you're writing code with a team the calculus is different - Use/Learn techniques and tools to reduce the teams' human-time wastage plus all the above.

If you're writing code just for yourself the calculus is different - Save your own human-time.

If that was a fair comment then we’d be writing all our code in assembly still.

True, but sometimes these 10 minutes help you to discover something new that will improve your code.

I had a few of these cases in my life:

- discovering optimized patterns in Perl, which led to code I could not understand the next day

- discovering decorators in Python, which led to better code

- discovering comprehensions in Python (a magical thing) that led to better code, except when I wanted to be too clever and ended up with Perl-like code

I iterate over rows in pandas fairly often for plotting purposes. Anytime I want to draw something more complicated than a single point for each row, I find it's simple and straight-forward to just iterrows() and call the appropriate matplotlib functions for each. It does mean some plots that are conceptually pretty simple end up taking ~5 seconds to draw, but I don't mind. Is there really a better alternative that isn't super complicated? Keep in mind that I frequently change my mind about what I'm plotting, so simple code is really good (it's usually easier to modify) even if it's a little slower.

I'm glad that DataFrames don't iterate by default. It's good design to make suboptimal features hard to access.

>That’s sad, as when you find yourself iterating over rows in pandas you’re almost invariably doing some wrong or very very sub optimally.

Humans writing code is suboptimal. I can't wait for the day when robots/AI do it for us. I just hope it leads to a utopia and not a dystopia.

I got bitten by that prank when copying code from a question, to see what it did (it was something obviously harmless). I was rather annoyed for about two seconds before I realized what date it was. :)

Reply on news.ycombinator.com