Rob Pike’s Rules of Programming (1989)
users.ece.utexas.edu
users.ece.utexas.edu
It's very close (in my opinion) to a restatement of Brook's quote "Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won’t usually need your flowcharts; they’ll be obvious."
Things like whether you have an array of structs or a struct of arrays can have a massive effect on how the same algorithm will perform, because of factors like data locality, alignment and cache coherence (the latter mainly if it's a concurrent algorithm).
The trouble with studying algorithms without actually working on optimisation in a low-level language like C or Go is that it completely abstracts away what is really happening in the hardware, which is absolutely critical to real-world performance. In a LISP-like language you generally have no idea what the compiler is actually doing.
Just making up an example, often I need to return multiple values from a function in languages that don't directly support that. The pure & efficient way might be to make a new data structure and return that. The "lazy" way is to just stuff them in a map / dictionary / hashtable and return it instead. The cost of those key-value lookups is enormous compared to a direct field lookup, but I can rationalize it as avoiding "premature optimziation". But if you end up with a whole code base that is doing this, eventually the whole thing is operating an order of magnitude slower than it should be. (it's also going to be a nightmare to maintain and refactor, but that's another story ...).
It's a lot easier to optimize well-architected code than to re-architect optimized code.
Should I spend more time here making this variable readable? Should I structure this script to be maintainable, or is it going to be of no use in 2+ weeks?
Sometimes the answer is yes; sometimes the answer is no. The key is not to pre-maturely optimize. You have to use your own good judgment and an iterative problem solving process to figure out what the metaphorical bottle neck is and fix it (I say metaphorical because, again, it's not just about speed).
But it is not about devaluing having a careful discipline and intuitive sense of the performance of code and a rigour about how you think about performance in your work. All those things are still incredibly important.
The article is talking about optimising before you can prove where the problems are... Apple had excellent testing which showed where a lot of issues were. Some issues may well have not been discovered until a wider audience had access though.
Testing can happen before you release a product you know? You new fangled startup MVP types only think good testing happens on paying customers. Fuck you guys.
Also, the more you worry about writing performant code, the easier it will be for you to write it that way in the first place without making sacrifices in code readability or maintainability. It's a myth that high performance code is always more complex and error prone. If performance is always something put off as something to address later, you can end up with as you describe 'a giant mass of moderately slow code'.
What I do agree with is that picking the correct data structures has more of an impact that algorithm noodling, though that has it's place.
What you want to do is try to get most of your FINITE and EXPENSIVE development time on optimising things that actually matter to user experience.
Worth reading for some context.
http://c2.com/cgi/wiki?PrematureOptimization
seems to concur.
There is no doubt that the grail of efficiency leads to abuse.
Programmers waste enormous amounts of time thinking about, or worrying
about, the speed of noncritical parts of their programs, and these
attempts at efficiency actually have a strong negative impact when
debugging and maintenance are considered. We should forget about small
efficiencies, say about 97% of the time: premature optimization is the
root of all evil.
Yet we should not pass up our opportunities in that critical 3 %. A
good programmer will not be lulled into complacency by such reasoning,
he will be wise to look carefully at the critical code; but only after
that code has been identified. It is often a mistake to make a priori
judgments about what parts of a program are really critical, since the
universal experience of programmers who have been using measurement
tools has been that their intuitive guesses fail. After working with
such tools for seven years, I've become convinced that all compilers
written from now on should be designed to provide all programmers with
feedback indicating what parts of their programs are costing the
most; indeed, this feedback should be supplied automatically unless it
has been specifically turned off.
of course it doesn't tell you anything about the speed of the C code, but I find the line coloring to be a nice UI for showing computational cost.
Most code we write will either not be used or eventually rewritten anyways.
But it's like the advertsing paradox: we know most of our code will be discarded or rewritten, but we don't know which parts won't. And even worse, it is often the bad parts that survive because people become afraid to touch them.
Company has Departments with many Employees. One Employee is always a DepartmentHead.
POOR DATA STRUCTURE:
class Department {
property Integer DepartmentId;
property String DepartmentName;
class Employee {
property Integer EmployeeId;
property String EmployeeName;
property Integer DepartmentId;
property Boolean IsDepartmentHead;This will require programs to make sure that no more than 1 employee can be a DepartmentHead. For every update to the IsDepartmentHead property of an instance of Employee the program will have to iterate through the list of Employees to flush the switch before assigning the new one.
BETTER DATA STRUCTURE:
class Department {
property Integer DepartmentId;
property String DepartmentName;
property Employee HeadEmployee;
class Employee {
property EmployeeId Integer;
property EmployeeName String;
property DepartmentId Integer;
This is a much better data structure since you can only ever have one Employee assigned to a Department as a head. The program will of course need to prevent Employees from Departments they don't belong to from becoming Heads of the Departments they are NOT in.
The really persistent students would get things to almost work -- often with hundreds or thousands of lines for something that, with a tree, doesn't take more than 10-50 lines.
This is also an interesting case study in human behavior; the students were typically too impatient to spend 20 minutes thinking about the problem before coding, but could spend 20-30 hours on a single assignment. Odd combination of work ethic and lack of patience...
This tends to get rewarded in industry as well, hard work, lots of code, must be good.
I wouldn't call it a lack of patience, per se. In most modern models of the brain, exercising executive control to think abstractly ("using System 2") expends energy for as long as you're doing it, and the explanation for many of the brain's inherent shortcuts and biases is to to prevent you from switching into System 2 for longer than absolutely necessary.
I have a feeling that a large part of what is characterized as "intelligence" is simply the brain being willing, for whatever reason, to switch into System 2 "mode" more often, and stay in it for longer.
From Thinking, Fast and Slow by Daniel Kahneman
[0] https://schacon.github.io/gitbook/1_the_git_object_model.htm...
I was the developer of one of the most used app for managing schools in my country (think: scores, grades, students, etc)
In my country, this look like this: (Ugly as hell, but that is not the one I design for the app)
http://www.slideshare.net/wildercondori/planilla-de-califica...
At first, the database was normalized. You have the obvious relations where of students, periods, class/grade, etc... so was a tree.
At the time of editing and printing (this was circa 2000 with FoxPro) things get complicated... fast (was important that editing the scores for each students be very fast).
It was my first serious job, and I'm a self-taught developer, so I don't know back them that my tables were a tree, neither any data-structure apart from the used in Fox.
But one day it hit me: Why I'm doing the tables this way? I see the pice of paper that show how manually a teacher do the scores, and I think: I will do the tables EXACTLY LIKE THIS (ie: As you see in the image above).
Suddenly, everything else get easy. Printing was easy. Editing was fast. Less errors. The app was a hit in part for that "small" change. Look like the competitors never get that idea back them.
Thing is, I don't know; n is definitely small in our unit tests, but if it isn't guaranteed¹ to be small, I'd rather start the code off as something with O(decent) as opposed to O(whatever an array gets me). It's not clear to me if this is what Rob Pike means though: does simply using a hashtable when appropriate to avoid iterating through a list count as "fancy"?
When the ultimate bounds of n aren't certain, I'd rather start with a good O(...) algorithm, and then apply the "measure, and optimize."
¹and even then, if my "guarantee" is coming from something other than physics, I'm wary.
Edit: and it's not in the middle of a tight loop
Make the format simple and obvious enough to be applicable for a wide range of purposes. Because humility.
http://keithp.com/blogs/Repository_Formats_Matter/
On-the-wire formats matter, on-disk formats matter.
Too many developers plow forward in a brute force manner creating unmanageable inconsistent messes and use rules like these to justify their lousy (lack of) design choices.
My rules:
1. Understand the problem domain.
2. Design flexible solution using common patterns.
3. Stub out skeleton of system encompassing majority of specs.
4. Revisit design and consider all conceivable exceptions and outlier and tweak design as necessary.
5. Review above work with others and tweak design further as necessary.
6. Coding.
Maybe this is part of the problem. I'm relatively young, so I've only seen a few design patterns, but it'd be interesting to have a high-level look at some of the most common patterns and what they're good at and what they aren't. For instance, I've never really used a messaging queue, but it seems like a genius design for a particular type of problem. I wonder how many other designs are out there that I'm not using, instead opting to shoe-horn in my existing ways of thinking.
No one can look into the future.
It is the numerous scores of small hacks that bogs down programmers nowadays.
Analyze traffic/usage of parts of the application - mark areas which are used most - analyze performance - optimize if not up to mark.
If stress is always on elegant code - it'll always end up taking more time. In real-life scenario, no module is ever "bug-free". The cases change from day to day. When you're absolutely sure that the functionality is exactly what is required - then you should spend time optimizing it.
Rule 6. There is no Rule 6.
Quite ironic coming from the creator of Go. Algebraic data types, anyone?
Edit: since when having ADTs are so taxing for the machines? I did a quick search and I found no source saying they are expensive to implement.
There are many problems that require some efficiency to solve effectively, especially in pike's field of systems.
PS: Coding go is my day job.
Go was designed as a system's language, I think it just eventually went in a different direction when people realized it would never match C++ or even Java in performance.
Now scale this to a situation where solving the problem requires ten thousand machines for each developer working on code, and where each minute of time spent writing code translates into two days of machine time running that code, and the numbers start to look different.
Meanwhile I just want to deliver the features for my product owner as quickly as possible! So we can both go home to our families in time and still deliver tons of business value.
Developer time is often more expensive, but it's always a one-off cost. Machine costs are ongoing. Machines can also impose hard limits on scalability.
In the real world, software optimisation is often necessary. Ever play a computer game? Those have pretty hard limits on much time they can spend processing. You can't just throw hardware at the problem to make it go away when you don't control the client.
"Just add more hardware" is also an ecologically unsound and unsustainable approach. Ever wondered about the carbon footprint of the average data centre?
Maybe one day we'll have optimising compilers so good that thinking about machine-level data structures won't be necessary. They've come a long way in the last 20 years, but aren't quite there yet. In the meantime, if you ever find yourself actually needing to make something run within some hard limits on CPU time, RAM, etc., listen to people like Rob Pike: he know's what he's talking about, even if you don't like what he's saying.
In the meantime if you're working on an MVP, by all means optimise for developer time. In that context it's almost always the right decision.
I hope I don't prove to be too detrimental to your health.
"Developer time is often more expensive, but it's always a one-off cost. Machine costs are ongoing. Machines can also impose hard limits on scalability."
A one off cost? I have never seen a codebase which gained consciousness, became self operating, fixed the bugs in itself and implemented new features, I hope I will, that's gonna be a truly glorious moment for humanity.
"Ever play a computer game?"
I did, but Go is rarely used for creating games. Typical use case: backend server services.
"In the meantime if you're working on an MVP, by all means optimise for developer time. In that context it's almost always the right decision."
Yes! On this site most people are working on some startup which will fail in 2 years. Performance is barely an issue.
No, but I've seen lots of projects who were completed, shrink wrapped, and shipped, with the team disbanded or moving on to other projects (sometimes with a few people left behind for bug fixing).
Especially most large scale enterprise /government / organisational projects are mostly one off, fire and forget affairs. A large team is assembled to create them, and then the support is offloaded to smaller team for fixes (and some tacked-on new features), and they run for decades on end.
>I did, but Go is rarely used for creating games. Typical use case: backend server services.
Which is bedide the point. The discussion was about those "programming principles" Rob Pike put forward, and the costs of developer vs machine etc -- not about Go in the least.
Machine time eventually translates to user time. Slow code leads to poor user experience and if your product is successful, wasting the time of millions of people.
Bear in mind that opportunity cost is an unrecoverable loss. Game programming where you press a DVD is an exception, but in the majority of areas the greater cost is actually in maintenance.
BTW, if you want to gain an understanding of this stuff, the difference it can make and why, I'd recommend reading basically anything by Michael Abrash.
No compiler optimization is going to get you a 100 to 1 improvement with a conventional language, but choosing the right data structure and algorithm for a task certainly can.
This probably isn't worth the debate, though.
You can argue that advanced language constructs will make it easier to write correct code to implement data structures, but it's not a magic bullet.
The Go authors argue that a simple programming language with loops, pointers and arrays is all you need to implement any data structure you need.
I find this a good example of a data structure for which I see little or no benefit in being modeled as an ADTs:
http://research.swtch.com/sparse
Even simple things like doubly linked lists are complicated with ADTs. On the other hand I don't know about good examples of ADTs used outside of the context of functional programming, do you have some pointers?
There is nothing advanced about ADTs.
"The Go authors argue that a simple programming language with loops, pointers and arrays is all you need to implement any data structure you need."
It's all about increasing the level of abstraction. There is a reason we don't code in assembly anymore. Of course machine code is all you need to implement any data structure, but how convenient is it?
"I find this a good example of a data structure for which I see little or no benefit in being modeled as an ADTs:"
So you found a single counterexample which (according to you) would not benefit from ADTs.
" On the other hand I don't know about good examples of ADTs used outside of the context of functional programming, do you have some pointers?"
I dislike pointers (a necessary evil if you have mutation I guess...), but here is one example:
data Gender = Male | Female vs type Gender int; const ( Male Gender = iota; Female )
Combine the first with pattern matching and you find yourself in awesomenessland (ie. code becomes readable, type safe).
Rules of abstraction: https://pbs.twimg.com/media/BqkVxmrCUAEZSTZ.jpg
Perhaps there was a misunderstanding. I asked you if you had some links to some interesting examples of ADT outside the context of functional programming. Because I was assuming that this wasn't a discussion about Go vs purely functional data structures.
Your example clearly shows the benefits of strong typing with fully described types. My point that this is orthogonal with Rob Pike's position about the importance of data structures.
Data structures exist independently on how we represent them. I don't see any irony or conflict in highlighting the importance of data structure vs. implementing language features that make it easier to write correct implementation of them.
Back in my university data structures class, an Abstract Data Type was just the collection of "a data structure and its operations". For example, a tree ADT could be a struct of two pointers and some value, along with operations like addNode, removeNode, findNode, copy, create, destroy, etc... We wrote lists, stacks, trees, hashes, and graphs in C as self-contained ADT libraries. This wikipedia page [1] explains the differences in approach.
The concept of data+operations was a lead in to objects for us, and seems very simple. Is the ADT idea in functional programming inherently more complex? I see from that page that it is intended to be immutable, which is certainly a difference.
1. http://en.wikipedia.org/wiki/Abstract_data_type#Imperative_v...
I think it is a question of how far you're willing to take generalization over how easy it is to come up with a reasonable concrete solution to a concrete problem.
See eg: https://groups.google.com/d/msg/golang-nuts/DLxFvfdRKBY/NDkW...
The c mindset of unions/structs coupled with functions doesn't contradict the idea of "chose the right data structures".
[edit: It appears others have made this point better while I was typing. The gist seems to be (to paraphrase another comment): (abstract) data types and types are not the same thing. types can help with the implementation of ADTs, but that's not really relevent to the 5th rule.]
"data types and types are not the same thing. types can help with the implementation of ADTs"
Can you please show me how can you implement a language (type system) feature without touching the compiler source?
Type safety (checking that all integers pushed to the "record-of-type-library-card stack" are in fact valid pointers to valid library-card-records) can be helpful, but not needed to implement ADTs.
Am I close to clearing up what I was talking about above?
[edit: typo, also: you mis-quoted me, dropping the "abstract" in "abstract data type" -- which might be the source of the confusion?]
As to the meat of the discussion, algebraic data types imply that you can apply algebraic operators to types. Since this is traditionally a purely compile-time feature you can imagine why the parent comment is incredulous that you could implement this without language support.
Your specific statement, "data types and types are not the same thing," is particularly confusing. You are making assumptions that aren't warranted, for example that "data types" encompass both a compile-time type check and an interface allowing abstract operations, while "types" are simply static type guarantees on values. In fact, both terms can refer to either.
Anyway, I think we've cleared up what I apparently didn't put very clearly in the first place: Algebraic Data Types can help with implementing Abstract Data Types, but one doesn't need the former for the latter -- and I suspect the data structures of "Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming." are indeed closer to using the right simple data structures along with the correct abstract data types -- so there's no direct contradiction between the 5th rule and Go not having (proper) Algebraic Types.
The middle brow dismissal is apt/damaging in these situations because it saves everybody a lot of time by quickly demonstrating that the author ignored those pesky real world facts and complexities that spoilt the ideological narrative.