Hacker News Reader

Comment by GauntletWizard | Hacker News Reader

Almost nobody can. Some people think they can, and charge forward with it, but rarely do they produce something that actually works. Usually, they bang on it and test it and cargo-cult it until it works, then convince themselves that it's from 'first principles'.

Even if it were a common ability to produce engineering work from written design principles, I'm not certain that's what we want. Experimentation is the key to science. You have to observe, change, guess, run, observe, change, observe until you can successfully predict what will happen for each change you make. The scientific method is taught to everyone, even if most ignore it and skip steps. 'Guess/check' is the correct way to do real-world engineering and design, not pure-logic simulation, because we still don't know all the first principles.

View on HN

> Experimentation is the key to science.

Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness.

Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However, both experimenting and reasoning can be flawed, and part of the trick is to learn how to do that properly. Computer science is difficult.

Programming is a branch of engineering. Very little programming is done by proof; It's done by experimentation, debugger, and tests are human-reasoned and hopefully cover all codepaths and critical junctures (for every x > 0, you test x=1, x=0, x=-1).

Proof is of critical importance - It's how you find those junctures, for example, how you optimize, how you design and how you reason about runtime, etc. But programming as practiced in the field is engineering first, mathematics second.

Reminds me of the famous Knuth quote:

> Beware of bugs in the above code; I have only proved it correct, not tried it.

Deep thinking and mathematical proofs are tremendously useful when thinking about a well-defined and scoped problem at a more or less fixed level of abstraction.

The problem with software engineering is that in practice all abstractions are leaky, and so you inevitably find yourself dealing with issues that are too far-flung and random to be mathematically tractable. The best you can do is be rigorous about the core problem you are solving, but there is always some amount of hammering and duct tape to build it into a non-trivial real-world system. It's possible to attack this problem asymptotically with a rigorous engineering process such as NASA employs, but we don't because it's simply not cost effective for the majority of software.

because it's simply not cost effective for the majority of software.

That's a great point. We utilize best practices, like automated tests, to narrow the gap between pure experimentation and mathematical proof. Proofs in most code would be difficult to impossible because most of use libraries that use other libraries, etc., so we just do the best we can given the time and monetary constraints that we have.

This is a great comment. The category error you've identified is manifest in more than the comment you're replying to as well: it's at the heart of one of the biggest problems with software interviews in a certain sector of this industry (the heavy focus on academic CS in startups and many Bay Area tech companies).

Physics of also a branch of mathematics. But experimentation is key to proving correctness.

What? Physics is not a branch of mathematics, it just uses mathematics and mathematical properties. That's like saying psychology is a branch of mathematics because it involves statistics. Using math isn't the same as being math. Admittedly physics is one of the "mathiest" fields, but it still involves doing experiments and coming up with mathematical models that fit the data rather than coming up new kinds of mathematics. The line was much more blurred in Newton's time, but these days the distinction is pretty obvious.

Certainly physics is not a branch of mathematics, but it isn't uncommon for papers in theoretical physics to essentially be math papers with an eye toward applications to some physical notions, and physicists do come up with new math.

I would restate this so that software engineering and physics are fields concerned about building practical models. They are similar in a way that they both employ and invent models communicated through a mathematical formalism. "Model" is the key here, not mathematics.

Even in math, you don't build models, you discover them. Pythagoras wasn't worried about the axiom of choice, he was discovering how geometry works.

I don't mean this to be a "mathematics from nature argument" (even infinite sets are hard to justify as coming from nature), but the course of history has usually been "solve similar problems in an ad-hoc manner many times, then generalize"

I think "write code and see if it works" may be tad more succinct and closer to reality than "build models, reason about them, find properties, then prove them, then rely on them. Tests ... provide guarantees of correctness." wrt to OP's comment.

Then most programming is not computer science.

It's not really worthwhile to try to code from first principles, because the context can always multiply in complexity. You want to make a tic-tac-toe game. What language are you going to program it in? Is it going to be cross-platform? Are you going to code an AI engine?

Trying to do anything in computing from first principles is like trying to code without mistakes. You're losing the value of iterative design.

I disagree. It's worth working things out from first principals as a method of practice, because when you have to do something completely novel, you won't be totally lost (If you never have to do this in your work, then I guess it doesn't apply).

Also, if you never work things out yourself, we'll only ever have one way to do things. What if a better approach exists but nobody's ever tried it?

It's good to double/sanity check what you came up with afterwards, and if the only way you can think of to do something is obviously seriously flawed, then don't go through with it and look it up instead.

Any time I have to develop something from first principles, I do it 2-3 times in different ways, and only then make a serious attempt at actually writing it. It's still terrible, of course, but at least has a chance of working since I have a bit of experience in what can go wrong.

That's exactly my method. What I've noticed in the last couple of years that there are more and more programmers that could not start from a 'blank page' if their life depended on it. That's a sad thing in my opinion, programming is a creative job and creation sometimes needs to start from nothing or extremely little.

Engineering is not a science, there is very little space for discovery there. In engineering you're applying science, not creating a new scientific knowledge.

Engineering lives on a continuum from almost pure maths to pure cookbook canned solutions, with a much smaller arrow going in the opposite direction from unexplained observations to theorising and model building.

Things like Shannon's description of information-as-entropy were certainly a mathematical discovery about engineered systems, and led directly to a lot of coding and data compression theory.

There wasn't a whole lot of interest in quantum channels at the time, so the fact that the theory fed something back to mainstream physics was a bonus.

> Experimentation is the key to science.

A sort of humanitarian science, since almost everything we touch have been made by other humans. But somehow STEM people are famous for looking down their nose on the non-hard sciences.

Reply on news.ycombinator.com