Life at a startup
abstractnonsense.com
abstractnonsense.com
I think that's the hardest part. Generally you say know even when it's technically possible. Hell you probably know exactly how to do it, just not in the timeframe you have. Learning to keep your ego (I can do this) from getting in the way of what's best for the product and the business is the hardest lesson a good developer has to learn IMHO. It's the programmers version of crossing the chasm.
Build fast and push it live.
I found that quite irritating from somebody who seems so experienced.
Okay, but is he wrong?
Let's be specific. He says "[Unit testing] just doesn't cover a large enough class of potential problems to be generally useful."
He said how he evaluates bug-finding tools and discussed his experience. What is yours?
If a particular unit test costs you more to write than it saves you in bugs, don't write it. And if it starts costing more in maintenance than it saves in confidence, get rid of it. There's a fair bit of cargo culting going on about how much to test - in my experience, it's not worth testing getters/setters, it's not worth testing simple "if this then set these variables" logic, and it's not worth unit testing database or external calls (you should have some integration tests for them though).
It is very much worth testing tricky cases like a parser, or a bit-level decoder, or complicated algorithms. I just recently got up to 100% code coverage on a UTF-8 decoder I'd written. I found about a dozen bugs in the process. They were silly things, like leaving a digit off a hex constant, but they rippled throughout the decoder so that the results were completely wrong. If I'd just included that in another library without testing, it'd make the dependent library subtly wrong too, and I'd never hear about it until users started complaining, and they'd waste hours trying to find bugs in their code that are really bugs in mine.
I'll also mention one other thing: the point of unit testing isn't to find bugs, it's to give you enough confidence in your code that you can freely change it. The real cost of bugs isn't that one erroneous event that customers will probably never notice, it's that the fear of that one unpredictable event makes management (which is sometimes you, if it's your startup) say "It works. Don't touch it, because you'll break it" and never improve the software. That's how you get shitty software that you can't fix.
I think it's fair to say that integration tests (make sure that the functionality works) are useful; period. Not knowing whether your app is going to work when you push it is not fun.
From a maintenance perspective and assuming that a problem will show up in integration testing or via a bug report, further testing is only really useful to the extent that it lets you debug a failure faster. However that acceleration is a tradeoff against the time taken to write the tests in the first place - ergo: is it really worth it?
I personally don't think the real win from tests is in saving debugging time. I have found it's in making it quicker to write code. I know that sounds crazy as most people view tests as overhead but I view them as development tools.
The key to them being useful though is writing them before you write the code. If you write them afterwards you will have already verified the code in an ad-hoc fashion and the tests are pure overhead. If you write them before they do the verification for you.
I'm coding in Rails and I've found the outside-in approach to development: write the six line Cucumber integration test and then write the RSpec expectations for the object; act as a verified to-do list and make my development far more efficient and robust.
I'm sure that other developers with better focus than I might not get the same benefit but I find it tremendously useful to immediately list down all of the edge cases and general behaviour that a piece of code should exhibit and then keep coding till growl glows green. I trade the time that I'd otherwise spend faffing around in a browser or a debugger for writing tests instead.
It's a bonus that all of those tests then ensure ongoing integrity, the real win is that tests stop me from wandering around the trail and keep me knocking things off in the order that they need to be done. [Caveat: it has taken me months and months to come up to speed on how to write tests well] (great article btw Bill)
You must be critical in your thinking for each one of these. Just because it works for someone else doesn't mean it will work for you.
Personally, I think that most design patterns are used as an excuse to make a simple system complex. I think that unit tests are more painful to maintain than they are worth (in many cases). I think that new technology should be approached carefully and only proven tech with minimum lockin should be used.
Don't go round saying that "experienced developers always do X" because a real experienced developer knows that X is great for some things and a huge waste of time for others.
In his defense, it sounds like he has mostly worked by himself. Some practices don't have a very good ROI when you are just hacking away by yourself. I would argue that unit testing is one of them, if you are not very experienced with it anyways.
I.e Are you an author of a library, a browser, a startup, experimental code, rapidly evolving code, space shuttle code ?
Each one has different measurements of "success" and these can effect your choices re Unit Tests.
Does Sweeney, Carmack, Cutler, Joy, Thompson, Knuth, Page, Steele, Zuckerburg, Anders, etc... use unit tests in a big way?
I'm actually a fan of unit tests, although I generally dislike TDD.
I'd stand by pretty much everything I wrote in my original post, for what it's worth, but I guess I don't really understand why it seemed to ruffle so many feathers to be honest.
Unit testing doesn't help me enough for the work I have to put in (compared to things like good logging and monitoring, for example, unless I'm writing something like a parser, with well-specified behavior). It doesn't seem to have helped justin.tv too much either, when we've tried it. But if it helps someone else, then good for them!
I think the main reason why people want to burn your article to ground was because there are way more companies out there that would benefit from their people writing unit-tests.
Now when some lazy developers who think that they're smart starting to quote "the VP of Software Development from JTV sez unit-test is useless", then we have a problem there. These passionate software developers don't want to go back to the time of Wild Wild West where coders slinging code with no reasons and start throwing incomplete shitty crap to QA and call it "DONE", get the gold medals from their team lead for being able to solve things very fast.
You know that movie... Spiderman? "With Power comes great responsibility"... since you're advertising how cool your company is and you're their "VP", then there's that responsibility attached with it.
The thing is, it requires a smart person with high level of discipline, strong principal, and the ability to say "No" to crazy requirements in order to have the luxury of not writing unit-tests. Especially when there's no safety nets (I mean QA) around that person.
I hold nothing against you though.
But he's not the only bright dev who tried it an ran into issues or lack of benefit in doing it. The thing that I seem to read from the more zealous of the TDD crowd is "you don't know what you're doing -- you don't understand code/software/testing/development if TDD doesn't obviously work for you".
My concern then is if a large percentage of strong devs don't seem to "get it", does it make sense to push into an organization w/o strong devs?
And the reason I asked the other question about the "hacker gods" is that outside of Josh Bloch (now), I don't really know of outstanding devs who do TDD in a big way. And this scares, because I remember when I used to have certain devs tell me that UML was the way to do everything, and if I didn't use UML throughout the product lifecycle I was no real software engineer -- and UML was the only way that in 20 years people would still be able to understand the design and code. I pushed back on UML and don't regret it for a second. I don't feel like I'm going to regret pushing back on TDD/BDD so far.
TDD is a major change in coding style and doesn't seem to work for a lot of people, including myself. Unit testing, on the other hand, is absolutely essential and fundamental to the task of building large, complex systems.
I haven't read Coders at Work, but if the guys in there are doing their thing without some kind of automated testing, I would be very surprised, and I'd wonder why they're making it so hard on themselves.
Random guesses are a poor substitute for logical certainty.
Take a look at Dr. Hipps projects. I believe almost all of them have a suite of test automation.
Keep in mind that there was no unit-test library for C/C++ back then. Even now, the unit-test libraries for C/C++ aren't as solid/easy-to-use as those in Java, C#, Python, Ruby.
The software back in the age of Ken Thompson, Dennis Ritchie are different than what we have these days as well.
I agree with nostrademons, some of these so-called "great developers" that you follow, they don't do maintenance programming. They seed a project and let someone else maintain it. Similar to the much-debated "software architect" role; when you don't feel the pain of maintenance programmer, you don't know what's wrong with the process.
Update: UML still exist today. For certain projects, there are values for using UML especially for communicating between people across different functions. Just like anything else: when you abused it, it starts to crumble down.
Your article was a very nice read, btw.
Zuckerberg and Carmarck are typical geeks that just love coding and don't care about the rest (like documentation, etc). Besides, it's their own company, they can do whatever they want and don't have to deal with the consequences; they have hordes of developers to clean up.
Knuth writes something for himself. He's probably the only coders there.
Anders has thousands of qualified MSc, PhD, and a bunch of other smart people that would write automated test for him (they call them SDET at Microsoft). (Besides, he seems to be an "architecture" kind of guy, does he still write 1k LoC everyday?)
Half of the people you mentioned came from the time before unit-tests were relatively unknown and they believe that to ship means to spend countless night and sleep below their desks. Coding, Coding, Coding is definitely the mantra. They don't know the other type of software development.
If it breaks, just fix it. No punishment.
I don't mean to paint it in such a bad way. If you look at Facebook during the earlier years, if it breaks, who cares, you're not going to lose million of dollars. The situation, culture, environment, products, and market segment are totally different.
And got Josh Bloch as the answer.
Not surprisingly, I find myself like Knuth. I like unit tests where I feel very uncomfortable. And I usually known when designing.
At the TDD level it does seem like religion. There doesn't seem to be strong evidence that it hurts nor strong evidence that it helps. A lot of people are successful w/o it and some are successful with it.
All of our early press had evaporated, it turned out nobody actually wanted to broadcast their lives, and we really had no idea what to do.
How did you solve this? I look into Justin.tv every few months and there's nothing compelling there for me, but it must be aimed at a different demographic.
Fluid layouts are great, but there is a reason books aren't generally 12" wide. (hard for humans to keep their place on really wide texts)
Also, increasing the line-height of the paragraphs would help a ton.
"I didn't bother to read the irc spec, I just watched the real-world traffic and used that as a spec"
"no code review, not even any conversation about how I'd implemented things"
Is this really the work environment we want to idolize?
From the guy's page, it looks like he's a super hacker - learning to program at nine, embedding a Scheme interpreter in the Linux kernel. Between this and jacquesm's recent "it's easy" post, I wonder, do you have to be this kind of crazy passionate coder to be able to accomplish this kind of thing? I feel like I could be passionate about coding certain things, but nothing I'm doing now comes close.
Be able to get things done. Be good at taking into account what you don't know. Know how to cooperate and delegate. This is the stuff that's most useful.
Be the worst guy in every band you’re in.
Don't worry about people being better at hacking than you though. There's always someone better, and in the end the only thing that really matters is your ability to get stuff done.