Hacker News Reader

Comment by cesarbs | Hacker News Reader

> Fast programmers build hacky tools to get around the hacky tools that they built to get around the hacky tools that they built to help them code.

This resonates so much with me. Sometimes I worry that I'm falling behind on keeping up with technology, but whenever I try to learn something new it seems like I have to install a package manager, then another package manager inside the first one, then pull a million other tools and libraries and frameworks before I can even begin doing something with this stuff I'm trying to learn. No, thanks.

I love computers, I love computing, I love thinking about solving problems using computers, but I hate the direction things seem to be taking. I'm seriously considering a career change in the immediate future because all this crazy tooling truly burns me out.

View on HN

One treatment of this exact problem: Just let me code [1]

[1] http://www.drdobbs.com/tools/just-let-me-code/240168735

That was a nice read, thanks. I'm glad to know there are more people that share the same feeling as I do towards the state of our craft.

> Fast programmers build hacky tools to get around the hacky tools that they built to get around the hacky tools that they built to help them code.

Yes, this is extremely problematic. The hacky junk built by hacky developers is frustrating and a direct contributor to my burn-out quotient.

Try Go, seriously. There's just the one tool. No package manager, no dependencies... And the code it produces is the same. Just a binary. "Here, have this tool, just run it." It's small, simple, and the community actively searches out simple solutions.

I found go to be quite cumbersome in comparison to npm. Magic folders in the filesystem and such. With node you never have to do more than npm install and everything is ready.

<rant>

npm (and the CommonJS module system) is the #1 reason I use Node.js. I don't care if Javascript is ugly, npm totally makes up for it. I don't get why most module systems implicitly import symbols in the scope (Python, Ruby) or worse, those that pollute the global scope (PHP).

Brew (Ruby) is not that bad but Python's package ecosystem is a complete mess (distutils, setuptools, pip, etc.). Cabal (Haskell) is pretty good too in comparison to C/C++ (installing dependencies usually involves following instructions in a README file, that is, if there are available instructions for your OS). I haven't tried Go.

I believe the language of the future will be built around, and win mainly because of, its package system/ecosystem.

</rant>

<rant>

npm is easily the worst. Python is a lot better. I hold the C library model as the golden standard, so we're not going to see eye-to-eye.

The worst part of npm is the sub-dependencies. Having enabled them, they proliferate endlessly. Each time I improve the efficiency of deploying a strictly controlled tree of modules, the front-end devs manage to double the number in use. 200, 350, 650, over 1000... oh and these improvements come from realizing that npm-shrinkwrap isn't 100% reliable, and ending up with scripts comparing npm-ls output and doing full wipe and replace with a tarball for even the slightest change to the dependency tree.

If some serious bug is discovered in one of these things, there's no way the 20 copies of it at various levels of this tree of 1000 modules are going to get patched. No one really has a handle on what's in these hundreds of megs of various versions of modules. This is the true fast code movement - serious problems can't be fixed in there, they'll just be ignored and replaced with other bugs in the twice-yearly full rewrite. Don't get me wrong, our frontend devs are among the best I've seen, but the pressures and environment they're in make them focus on churning out the latest fad in web design and skimping on engineering quality everywhere possible.

Python doesn't do implicit import of symbols into a scope if you don't use "from blah import (star)". Don't use the (star).

Managing, and using, a list of python modules, each at a particular version, is way simpler, more reliable, and more efficient (than npm style). Pip can reliably list installed module state (freeze) and install from source tarballs or git tag checkout.

You need to fully control and understand the versions of all libraries installed and used on a system in order to have a fully reproducible deployment state, and be able to reliably roll back to a previous state. Given that, the Python or C model is much easier to work with.

</rant>

We definitely have a philosophical disagreement. The fact that an app (especially a server side one where storage is usually cheap) has hundreds of dependency signals to me that the package system is successful. It tells me that there is a lot of code re-use going on and that it's presumably easy to import new dependencies. By now, I believe the number of NPM packages outnumbers any other package manager by far despite being relatively new.

So basically, I think your point is that a project should have as few dependencies as possible whereas I think the opposite is good. Both philosophies have their pros and cons. If you are writing mission critical software that handles financial data, it's probably a good idea to know about every line of code that gets executed. However, if your goal is to release something as fast as possible, have a more manageable code base and security is not as critical, using lots of dependencies makes sense.

Go's model is pretty good. This is why there's just a single $GOPATH and every package has exactly one location on disk, so even if 100 dependencies use the same subdependency... you only have to update one spot if it needs a patch. And again, this only needs to be done at development time. It's baked in when you compile, and from there on, deployment is just a file copy of your binary. No dependencies during deployment.

Maybe I'm missing something, but this seems like a bad idea to me. What if projects are relying on differing versions of the same package? What if two developers collaborating on the same project have different versions of that dependency installed?

Sure, it's space-efficient, but disk is cheap, and developer hours are expensive. I certainly wouldn't want two employees burning man-hours trying to figure out why some app is behaving differently on their respective machines, only to fix the problem and unwittingly break another project in the most expensively subtle of ways.

I'd really like to think I'm missing something though, I certainly don't presume to be more insightful than the collective Go team.

So, in theory, any particular import path should be stable. There are no "different versions" of the same import path. A different version would be a different path.

So, for example, github.com/natefinch/lumberjack is an import path. When I wanted to change the API, I have to put my project at a different import path so I don't break projects that use this version. I could just make a new github repo called lumberjack-v2 and that would work fine.

Of course, reality and theory don't always see eye to eye, and the Go community realizes this. That's why there are several community-made tools to help out.

The most well-respected one is godep, which has the ability to pin revisions on your dependencies. That is, it'll look at the git commit hash (or equiv for hg/bzr/svn) for every repo that you depend on, and record it in a file in your repo. It can then reset those dependencies to those specific hashes. Then all members of your team just need to use godep to make sure they're all using the exact same git commits for all dependencies. Bam - now you're insulated from someone updating a dependency with breaking changes, and your team can test new revisions and decide when they want to start using them.

Godep has a second function that lets you copy all your dependencies into your repo and renames all import statements to reference the copied paths. This is an even more extreme version of the above, which not only insulates you from changes in dependencies, but also insulates you from those dependencies disappearing entirely.

Godep is definitely a good tool. One thing that gives me pause, though, is that this functionality is not built into `go get`. Failing to put hard versioning into the canonical package manager seems short-sighted to me. I'd much prefer that over something like the race detector.

There's a philosophical disagreement here. It's true, developers are expensive. But I think it's a fallacy that what's less efficient for a computer is better for a developer. The npm-style dependency tree enables and encourages much more complexity, which developers then have to deal with when debugging or deploying. They need new tools to help them get a handle on the huge number of modules. That's more expensive than a system kept more "under control".

> I think it's a fallacy that what's less efficient for a computer is better for a developer.

I don't think that was ever said; I claimed that trading disk space for developer hours is a good trade-off in this case.

> The npm-style dependency tree enables and encourages much more complexity, which developers then have to deal with when debugging or deploying.

I don't believe this is the case. When developing an app (as opposed to a library) it's considered a best practice to check your node modules into source control, ensuring that there's never a mismatch between installed dependencies on developer machines. If you don't want to do that, you take what comes - but even before I picked up on that, I never ran into issues even when collaborating with 5+ developers. You just need to make sure your package.json locks your versions appropriately.

FWIW, Go namespaces everything by default, so if you import a package called foo, all types, functions, etc from that package are namespaced by foo, i.e. "foo.whatever"

The thing I love about Go's packaging is that your VCS and package manager are the same thing. There's no need to wonder where the code from package foo comes from... because you know you imported it from github.com/jimbob/foo It also means there's no fighting over namespaces, since it's just done by domain name (i.e. I can have the package npf.io/foo because I control that domain, and I don't have to to fight over who gets to use that name).

Hear, hear!

npm is without a doubt the best module system I've come across in my years of development. RubyGems is kind of a pain, .NET dependency management is a joke, Python is super fragmented, and Go doesn't have one. Godeps is an okay tool, and `go get` is cute, but I have yet to use a package manager that couldn't stand to borrow a thing or two from npm.

In Go, the dependencies only matter during development not during deployment. Let the developer hash out what specific version of each library works, and then during deployment, it's all baked into a single statically linked executable.

That's the nice thing about using tools written in Go. There is no npm install. There's a file copy. Bam, done. You don't even need to know it was written in Go unless you're using the code for your own development.

The magic folders in the filesystem exist in every package manager, it's just more obvious in Go. So, for example, npm puts code it downloads into /usr/local/lib/node. I'm very far from a Node expert, but this doesn't sound terribly different from GOPATH.

Go has a couple wonky conventions, but it's a very very simple system. Once you accept and learn to deal with the GOPATH quirks, that's pretty much the only tooling weirdness in the whole ecosystem.

No package manager or dependencies seems strange. What would you call all the projects on this page? https://code.google.com/p/go-wiki/wiki/PackageManagementTool...

There's no required package manager. By default, your VCS is your package manager. You can write code for a long long time with just the default go tool. Those are optional tools that you can use once you become comfortable with the language and understand their tradeoffs. You really don't need any of those tools until you want to write a professional project with multiple people on a team.

Yeah, I was also about to post a comment recommending Go. It's not completely immune to tooling bloat, but it's actively reductionist in that respect. It's relatively pleasant.

That sounds awesome, I'll definitely give it a try.

I am afraid a language change is not going to help your burn feeling

I don't know that I agree with that. Tooling bloat was specifically mentioned, and Go is quite good with respect to that.

There's just the one go tool to install, which can be installed just by unzipping a zip file. From there on in, you don't need any other tools. It just uses VCS for package management. You don't need an IDE... the single go tool has support for pretty much every part of the development cycle - compiling, testing, code formatting, profiling... there are no other tools you need to create real deliverable software. I've been writing Go for over two years and still haven't added any other tools to my standard workflow - git and go are the only two commands I type while writing go code. (At work we use a revision pinning tool, but I haven't needed that on my side projects.)

It is not the fault of these tools. You can think of code as data, and programmers producing this data, just like many other professions are producing data, too. Programmers often have a variety of far better tools to handle this data. I think the reason for this are of course the programmers dogfooding, but also the culture of life-long learning, which minimizes UI design to just API design. An intuitive GUI for a crowd less open to learning new things is so much harder to make.

I nevertheless see this as a gift. It is your attitude with these tools, that makes you unhappy. I I often have this strange feeling of missing out on an even better technology, the cool kids might be using. I think the key here is to be confident with your choices of tools, improve them, when their is pain and identify when something is good enough for some time (measured in years) before you should reconsider again. In the end it is also about handling the tools. This sounds very reasonable but in practice it is often hard to overcome the urge to use the new shiny tool, which seems to make you a better programmer, when you define yourself as one.

I don't think the extremist view of skipping them altogether and condemning the whole thing is the solution. I believe people, who feel they can't keep up in this imaginary race either search for ways to define themselves in another way e.g. in alternative careers (like you) or in their opposing way of programming (like the article author) throwing the others in a bottle of people, who do it wrong. There is a middle-ground between the Node developer and his weekly changing recursive package managers and the Coldfusion shop not doing scm.

Curious which languages you use that don't have this problem?

Usually the point of package managers is to allow you to leverage the work of others, so you don't have continually reinvent the wheel.

I've yet to run into a programming environment that did not have it's shares of headaches and nuances.

> Curious which languages you use that don't have this problem?

None. It's why I'm burnt out and want to so something different. Not something different from my current job. Something different from developing software.

I've lost count of how many times I've spent entire work days battling against the environment which is supposed to help me write the code I need to, to make a customer happy. Entire days of trying to figure out why IIS is acting crazy, or why Visual Studio is crashing, or why is Windows so damn slow all of a sudden. Entire days of trying to align the planets so that service A can talk to service B, because for some reason there's a cryptic SSL error that's not giving me any helpful information as to what's happening. Or entire days just pulling things from a dozen different sources and figuring out where to place each other so they can work together.

I've recently been to the Living Computer Museum in Seattle. You can play around with old computers there. I was fascinated by the old machines where you'd boot into a REPL. The shell was a REPL to the language you programmed the system in. I think despite all the advances we've made in the past decades and all the great performing technology we have nowadays, we've lost that essence of simplicity along the way.

> Windows so damn slow all of a sudden

VS is an awesome development environment, but I was having your problems with Windows and programs running under Windows for most of my adult life. I switched to Linux full-time at about 5-6 years ago, and I've been happy ever since. I can leave the OS running for weeks with no issues, and it would probably go for months but I'll turn the power off.

Oh, and with Linux, if you don't run a graphical desktop environment like Gnome or KDE (which most distros have the option of doing), then you boot into a REPL you can program the system in (Bash shell). Windows used to have the same with DOS, but no more...

Sounds like you have a problem with the shitty ms stack. Have to do some c# at work and I feel your pain.

I feel the same way you do about everything you said, including considering a career change for this same reason.

I'm now fiddling with Tcl/Tk as a way to quickly throw some visual ideas around. I'm using it to build an idea for a tabular programming language that is more visual and allows for letting go of incidental details like syntax, argument order, etc.

This is why something like golang is appealing. Simple simple and more simple.

Simplicity for simplicity's sake isn't a virtue. I often feel that Go chooses that path ideologically, whereas real world use cases should have more innings (yes, it's the generics and shitty type system thing again, I'm not expecting us to agree, I'm just pointing it out).

To that end, Go doesn't work for me. I use a fairly straightforward stack atop the JVM because I can hold the whole thing in my head (nothing in either Dropwizard or Play is deep magic) and have the expressiveness in Scala to be clear with my code.

And the issue isn't so much whether or not a programmer can hold the "entire system" in their head so much as, if they don't need to, they can be doing a whole lot more. This is, after all, what computers are good for.

I can do a lot more when I can trust my system to evaluate my code and throw compile-time errors. I can't trust Go's you're system in the same way I can Scala.

I noted the simplicity of the stack mostly to forestall the usual tired complaints about complexity, nothing more. What you call "academic", I call "building at scale."

Parsimony my friend. Parsimony. All else is academic tail wagging.

Reply on news.ycombinator.com