You Release Late and Infrequently | Hacker News Reader

You Release Late and Infrequently

measuringmeasures.blogspot.com

65 pointsliebke22 comments

Just this weekend I watched Machinima.com's documentary series on the history of Valve Software:

http://www.machinima.com/film/view&id=42271

One of the common themes in the series is Valve's frequent choice to delay shipping in order to get things right. Gabe Newell says something like, "Your software is late for a while, but it's bad forever." As far as I can tell, this has worked extremely well for them.

Which makes me wonder what the difference is between Valve's games and Gmail. I think it has something to do with their games being about experiential storytelling, and Gmail is a tool to accomplish tasks. I imagine most software falls somewhere on a spectrum between the "tool" and "experience" endpoints and I wonder if other software besides games might be hurt more than helped by "release early and often."

The difference is that valve's software has to have a build installed on the client's computer or worse, a disk you shove into a console.

On the web, your releases update 100% of your client base instantly.

Though I do agree that depending on your market, "release early and often" can hurt you. Getting a bad reputation early can be hard to shrug off later.

Yes, one part is the ease of productionizing for the web.

That said, I think there are always techniques for using the early, often, listen approach with private betas, multivariate testing, etc.

Release doesn't necessarily mean releasing to the public, and does not even necessarily mean releasing something user facing.

It's not just the medium. A game with plot and direction won't be the same when replayed. Imagine someone trying to repeat a joke to make it sound funnier, when you've already heard the punch line. So, danger of over-generalisation, but certainly games like TF2 can be improved with updates, extensions rather than revisions.

Perhaps that's his point about bugs vs new features. Releasing often works when you are adding more good bits, not so much when fixing broken bits that should never have been released in the first place.

Another story on valve that I read indicated that they do lots of testing and iterative design in house in the early stages, so they seem to have an internal 'release early, release often' culture, at least as compared to other large game company cultures.

This is also the philosophy @ Pixar - you can't release a movie twice, so it had better be right the first time. You achieve this by having 1000's of internal reviews, checkpoints, and pivots along the way where you trust your employees to challenge problems they see.

Valve can't afford to release bad stuff and patch it later - a large number of their games are single-player, and lack significant replayability. The first run-through for most players is the only run-through... you can't correct the bad experience later. It's a little bit like how you can't release a bad movie, correct it with a "Director's Cut" later, and expect significant numbers of people to come back and re-watch it.

As for Valve's multiplayer games, they do release often. Team Fortress 2 gets sometimes weekly updates (balancing and bug fixes mostly), with large content releases every few months.

Interesting. I was going to come in here and ask if this advice was good for me to follow as a long-format fiction writer--but you seem to have answered that.

I know you mentioned long-format, but if you are able to serialize your work you can go for the "release often" format.

Shows like 24 actually do not map out the entire upcoming season from head to tail before shooting - and in fact the back half of the season is entirely un-shot when the season begins airing. This allows show runners to tweak the show based on fan response - serializing fiction may have a similar advantage.

On frequent releases: I absolutely love that there's a new interface tweak almost every time I login to my GitHub account.

yea, although a couple of the tweaks have ended up tweaking me out when i can't find something. :-)

Wikipedia has links that go nowhere.

But it does still sound a bit dodgey, I think because to get accurate feedback, you necessarily need to fool people. For example, if you had a link that stated it was a proposed feature, it would affect the clickrate. Or maybe it wouldn't? Maybe interest would be enough. Perhaps this could be experimentally tested.

Personally, I think I would feel ok about putting the link up and having it go simply to a landing page that said "This feature is still in production. Please comment below on how we could best suit it to your needs."

I also feel it's a breach of trust when you mislead the users you have implemented something in order to test demand. Never mind the ethics, it seems it doesn't always work:

http://news.ycombinator.com/item?id=960721

I was about to raise the point that "release early, release often" is a phrase from free software, and wonder at its appropriation by commerce, but it turns out it's from ESR. Perhaps it is appropriate, then.

(The majority of my hack is not for customers.)

The Wufoo example is awful. Shouldn't have taken more than 5 min for them to realize, "HEY the Flash experience on GNU/Linux sucks! Maybe we shouldn't use it so that we can have more customers use the thing!"

Those 3 versions of GMail could have been one version that was most useful. There was no time constraint for it since it was a side-project.

The links to nowhere sounds like a good idea except I don't think it indicates interest in the feature being built. A link's text is usually very short and you don't really get any demographics from a click. If you knew 15 to 20 yr olds were clicking a link but the rest of the site was mainly used by 30 to 40 yr olds, you might want to spin out the link as a separate site instead of just a new feature or something. But how do you know how interested people really are? :/

Interesting criticisms of PB's approach to building gmail. Can you share a link to a product you've shipped in the past 12 months and some of your experiences about building it?

Is that an appeal to authority I hear? I haven't worked on a product that has shipped in the last 12 months but I have worked with several programming languages and have read some Dijkstra and have done some literate programming, and have gone to university/college for a few years now.

My criticism was based on the stuff PB said. All of that functionality could have been done in a single version but PB didn't want to waste company resources unless he could offer up proof that they were being used wisely.

There's nothing wrong with that, but to say that it's a good development model is awful. How long did GMail stay in beta for? It never really shipped.

GMail has 100+ million users[1]. Imagine how many they could have if they actually did ship something. Crappy development model or not, does it really matter? The thing is widely successful and any insight into how it came about is valuable as a learning experience.

[1]: http://www.ft.com/cms/s/0/fc3d8a86-02a0-11de-b58b-000077b076...

Actually, it turned out that the complaints coming from Linux users about Flash being used in Wufoo in the early prototype was disproportionate to the future demographics of our actual customers and so it honestly didn't make any difference in terms of our revenue today based on our current application usage. I think Linux users account for less than 2% of our user base.

In retrospect, a good number of those complaints came from Reddit users when we were just getting out the door. Our startup was only a few weeks old then and I have to say that the feedback felt overwhelming.

In the end, the decision to remove Flash in the builder was a good one because it resulted in a much simpler interface that was easier to use, but to say that it should have been obvious or that it really mattered, would be overstating it.

Then the Wufoo example is mistated in the blog. The feedback was useful for deciding on interface changes but not useful for deciding on platform/technology changes which is what it seems to imply.

So I'll change what I said...It shouldn't have taken you more than 5 min to realize that GNU/Linux users wouldn't have been a huge percentage of your users and you could have gotten away with ignoring their advice on Flash/Javascript.

It's a good case against releasing early and asking for feedback...you have no idea if the market segment using your product is the one you will be targetting in a few months.

So the conclusion you think is obvious changes after you get more data, but you still think the data are not important for reaching that conclusion?

Reply on news.ycombinator.com