Comment by pron | Hacker News Reader

Comment by pron | Hacker News Reader

Programming languages matter some, but algorithms and data structures matter a hell of a lot more.

While it is true that learning different programming paradigms can help you evolve as a programmer, I strongly believe (I have been programming professionally for about 20 years) that that is far from the best way to evolve as a developer, and learning algorithms and data structures will help you "level up" faster and better.

The reason is simple. Programming languages are a medium of transforming programmer intent into something the computer can execute (perhaps indirectly). This is not an easy task by any means, but all an ideal (nonexistent) general purpose programming language can hope to achieve is the reduction of what is known as accidental complexity. Programming languages vary only in how much burden, and -- just as important -- what kind of burden they place on you when you commit your chosen algorithm from your mind to code on the screen.

Knowledge of algorithms, on the other hand, will help you tackle the far more important essential complexity of the problem at hand.

So, in a way, focusing on programming languages is indeed a distraction, at least as long as you haven't yet familiarized yourself with the more fundamental -- and more useful -- tools of computer science.

When people talk about learning algorithms, what do they mean?

I recently spent a few hours reading about and implementing A* for the NPCs in a video game I'm working on.

Are you talking about being able to write A* and other algorithms from scratch with no reference material?

Some aspects of this in the culture are disturbingly binary and the world has WAY too many people who don't know the engineering application of A* so they just make something up. Knowing that it exists and has certain (detailed) characteristics is probably 95% of the use. Why didn't you use B* (rhetorical question, whats important is you have an answer not what it is) Folks who don't know path algos probably think I'm making up B* but it really exists and was invented just before I started programming and I have a memory for "new" stuff vs old.

Its useful to have a steel beam and know how to apply it to construction. Knowing how its made and the metallurgy and how that impacts the application levels you up. Then there's a whole sea of lower level construction vocational laborers who don't know steel beams exist and they'll just bodge something together with stuff from the junk yard and when it collapses, well, there's no guarantees in life and it was cheap and the academics never know anything anyway so stop complaining, and last but not least the social butterflies who heard the Jones's next door, who are billionaires, built their wall out of recycled plastic so we'll use that to hold up the roof instead of proletarian wood or even this steel beam stuff.

I can't speak for anyone else but I think the important thing is understanding the tradeoffs of different ways of accessing data.

At it's simplest you have things like checking for existence using linear scans of arrays. Anyone with experience sees that as a bad thing to do because they understand that it's inefficient for any sort of sizeable array. But those who don't understand algorithms or data structures won't even know there's an issue. And that's the simplest case really.

When you have a list and you need to efficiently add and remove items from both ends - then you want some sort of double-ended queue. I know where to find the one in my std lib and I know when I need to use it - which it turns out is almost never - I thought I did the other day, then I found a simpler way, which was actually a bit disappointing :)

Then, as you say, you have things like A* or R-trees. The more of these you have an handle on - even just to know they exist, the better position you'll be in when you need them.

But I don't even think that's the most important part of knowing data structures and algorithms (and really, I'm sure none of that's news to you if you're implementing A* searching). For me it's more about understanding efficient patterns for data access. So much of the work I do is about efficiently pushing data around in different forms. Often having a core chunk of data and then a couple of different indexing structures to allow me to interrogate it depending on the info I need.

> Are you talking about being able to write A* and other algorithms from scratch with no reference material?

No, I mean knowing what algorithms/data-structures are available, what their characteristics are, and how to tweak them. This way you can say stuff like: with those data access patterns, we'll need a B+-tree with links for concurrency, modified to not delete depleted nodes until such-and-such. Or, something like: doing that efficiently requires an optimization that's NP-hard because the problem reduces to graph partitioning, but we can use an approximate solution that should work well-enough in this case.

First of all, you should know that there exists an A* , and what it can be used for.

Second, you should know when to use A* or something else, the trade-offs etc.

Third, you should know how to implement A* , when needed, with reference material (you'd be surprised how many programmers can't read/understand the reference material even when its given to them, or don't know enough programming to put it in code).

Of course you could get by with never using A* or tries or whatever advanced algorithm, not even in some ready-made API form. E.g. if you just do some CRUD or some simple web programming. But for the kind of programmers we're talking here, those three are paramount.

The thing you mention, implementing it from scratch with no reference material might be impressive, but it's more of a circus act.

Aside from as a learning experience or display of prowess, what's the point of implementing A* yourself?

In my experience, knowing which prebuilt algorithms and data structures to reach for matters more than being able to implement them myself. Not saying there isn't value in learning or that it doesn't help flex the problem solving muscles, but in most software domains, I think that practically speaking there are more important skills to focus on after you've mastered the basics of time and space complexity.

>Aside from as a learning experience or display of prowess, what's the point of implementing A yourself?*

Because for people working in those kinds of problems we're talking about (we started from Valve and games IIRC, but it could also be Google, Facebook engineers etc) you don't just download some off the shelf API.

You'll need to create something of your own to:

- have the whole IP

- mold it specifically to the domain logic you need

- take control of it's memory and performance characteristics based on your constraints

- implement it in the language your company uses, for which no ready made A* (or some other algorithm) is available

- you'll need to make it talk to different infrastructure, libs etc

- you might be the one writing the library API yourself

As I said, this is not about what a CRUD programmer will need. But as you can see everyday in HN, people write these and other algorithms all the time in their jobs doing more serious engineering.

Time an Space complexity are moving targets too. The rapid change of the importance of memory locality will soon make a lot of the hard and fast data structure advice useless. At least if you program in something like C. Those n's are a lot bigger then they used to be.

When programming in high level OO languages, which have ready algorithms and data structures are pretty-much standard, what's your use-case of algorithms?

I've seen entire projects which do not require specific knowledge or application of any algorithm. Even when they do, one can google and find "best algorithm for XYZ", then find a library which has the algorithm implemented and off you go.

I understand the use of algorithms in low level languages and specific domains, but when you need to sort a ruby array you just "array.sort" and a default (IIRC the default in ruby is a version of quick-sort) sorting algorithm is applied, which 9 out of 10 is the faster solution you'll get given the language constrains.

Can you offer an example of specific situation, in any OO-language with reasonable amount of available libraries, where deep knowledge of design and analysis of algorithms is actually required?

> Can you offer an example of specific situation, in any OO-language with reasonable amount of available libraries, where deep knowledge of design and analysis of algorithms is actually required?

Interviews :)

"Can you offer an example of specific situation, in any OO-language with reasonable amount of available libraries, where deep knowledge of design and analysis of algorithms is actually required?"

You won't get very many good replies because you are asking the wrong questions. The truth is you don't even need a decent understanding of OO for most day to day dev. You will just produce suboptimal work.

A better question is: "...where reasonable knowledge of design and analysis of algorithms allows you to do a better job?"

And the answer to this more reasonable question is: Every single project I've worked on as an enterprise developer over the last 10+ years. Being able to say "hey actually this problem we are working on can be formulated as a graph problem" can turn a 3 month problem into a 3 day problem.

For example we recently had a rules engine rule dependency issue that turned out to be expressable as graph problem and was then easily solved. If nobody in the team had a good knowledge of algorithms we still could have solved the issue but it just would have taken a lot longer.

And unfortunately until you can phrase the problem in the right way you can't just google "find best algorithm for XYZ".

That is an enlightening answer, thanks.

There are cases where you must adapt a known algorithm to fit your domain. In my job, for example, we work with automatic train dispatch planning [1]. The application is built in Java and one of the main components of the route planning is a custom depth-first search. The search tries to move a train through a series of connected tracks while considering the passage of time and checking for time-dependent constraints such as track blocks, other trains, speed restrictions, etc. This couldn't be done using a standard library implementation.

Also, I believe such knowledge will make you a better programmer, even if you don't have to build everything from scratch. I have seen experienced developers using a list to store unique elements, when the only read operation they would do was to check if an element was in the list. Their solution to speed up the checks was to sort the list every time it was modified and then run a binary search. In this case, better knowledge of algorithm complexity and data structures could lead to a much faster solution, using a set for example.

[1] http://www.academia.edu/6128585/A_Heuristic_Approach_to_Trai...

Also programming languages and frameworks quickly go out of date, while the fundmentals like algorithms help you out no matter which language you're working in.

Reply on news.ycombinator.com