Hacker News Reader

Comment by Avicebron | Hacker News Reader

statistics is pretty useful in computer science...and as someone has has worked in biotech on the software side...chemistry can be as well. Heck even anthropology and philosophy were pretty relevant when I took my courses on NLP years ago.

View on HN

As someone with 10 years as a software engineer I haven't needed it.

I’m sure there are 10y SWEs (and 1 yr) that use it every day.

Obviously it’s going to be irrelevant to compilers or something, but not to heuristic approaches or anything with some quantifiable (and actively quantified) nondeterminism.

It will be very relevant in any field that does performance testing. And that includes compilers.

I don't think you need a statistics class to do performance testing.

You change an optimizer pass. Looks good on a microbenchmark. But then you try it out on some samples of real code. Turns out, it probably makes some real code a little bit faster. No difference on other real code, but the data are noisy, so it's hard to tell. And on one piece of code, it seems to actually cause a regression - but again, the data from multiple runs are noisy.

Should you enable the optimizer change by default, or not? Or do you still need to collect more data? How much more data? What data - more runs or more different code samples? How confident do you want to be, and how confident can you be?

These are questions you will face in your real day to day work, and a few statistics courses will be incredibly helpful to you in answering them.

I knew someone would say something like that, but I've never seen that sort of thing done personally and I really doubt lack of statistical knowledge is even close to the biggest obstacle to writing faster software. For micro-optimizations so subtle you need fancy techniques to even tell they work, non-quantitative factors (code impact, will it enable other optimizations, etc.) are more likely to be decisive. Techniques to reduce noise are either non-statistical (warming up caches) or unsophisticated (average many trials, best of three).

The amount of programming jobs (out of all possible programming positions available) that require proper statistics or chemistry for that matter are so low that it doesn't make sense to learn those subjects (in the context of computer science) preemptively. Even if people who take and succeed in subjects like Chemistry who do end up in a job that may use it down the line probably won't remember what they learned anyway.

The jobs don't require it, but the people who are not aware of statistics or chemistry are unable to recognize when it would help them. I.e. it becomes a self-fulfilling prophecy.

(It has nothing to do with "in the context of computer science").

I imagine they'd either have coworkers who are more involved in those domains (if actually needed for the job) who can guide them on the right path or can just google their problems and end up on a path of self-discovery that leads them to those subjects. Prematurely surveying other subjects not pertinent to the job one is going to university in the first place for seems silly in that regard. I'm sure many would rather focus their time on learning more of the things that are shared in the majority of the job market (or at least more things that actually interest them).

You'd imagine wrong. (In my experience.)

For example, in mechanical engineering, I know how to make reliable systems out of unreliable parts. Software engineering is still focused on a hopeless quest to make software perfect. How to do it assuming unreliability has been slowly seeping into software engineering the hard way, bit by bit, over my entire career.

One example: running the brake software on the same computer that is connected wirelessly to the internet.

I have to disagree with that. One obvious example is Google, who made a fabulously reliable and fast machine out of cobbled-together ultra cheap commodity hardware, and conquered the world with it. That was in the 90s, and their way of doing things took over completely. It's why companies like Sun went out of business -- they had the super-reliable (and overpriced) systems, but nobody wanted them anymore.

What you're describing (the brake system) is just simply incompetence.

You're right about how companies managed to scale their internet server farms. But that was late (1990s) and still has not seeped into the rest of the industry.

> What you're describing (the brake system) is just simply incompetence.

I regularly discuss this with people who are convinced they can make such a system secure.

For another example, I frequently advocate on HN for embedded systems to have physical write-enable switches for reprogramming the system memory. This makes it a physical impossibility for malware to infect that memory. Nobody agrees with me. They all think they can write bulletproof code.

I can't buy disk drives with physical write-enable switches, either, not since the 1990s. This is necessary so if you try to restore from a backup drive, you can't make a mistake and overwrite the backup, and the ransomware on your system cannot write to it. This is a regression in the industry.

And yet nobody on HN thinks this is a good idea, because it's inconvenient. Or they'll suggest a software switch, which of course is inherently corruptible.

BTW, the aviation industry gets this right. The stabilizer trim has a physical cutoff switch on all their airplanes, including the 737MAX. Unfortunately, in the 3 incidents of MCAS runaway, only one use the switch properly (and you never hear about that incident). The second never used the switch at all, and the third crew decided to disable the trim system when the airplane was in a non-recoverable dive without using trim. (The electric trim switches also are physical and override the software.)

The number of programming jobs that require statistics is huge though?

The MLE job market is booming.

For that matter the number of programming jobs that require computer science is pretty low.

For a programming job that requires knowledge of chemistry, it's easier to find a chemist who can program. Likewise physics, math, statistics, electronics, etc.

I’d say psychology is a good a thing to know as well.

Reply on news.ycombinator.com