Hacker News Reader

Comment by wamatt | Hacker News Reader

Encouraging to know this model scales to 100 employees at least.

Purely out of intellectual interest, I wonder if a company the size of Google or Facebook could also ship in this way, or if the whole release manager/team is essential.

View on HN

Sarbanes-Oxley puts a big damper on production deployments at big companies. I don't fully understand it so I won't try to explain it.

(I will complain though: the law says developers shouldn't have control over production systems. If that's a requirement, who's going to write the software?)

Wait, what? Could someone elaborate on S-O restricting developer control over production systems?

A lot of this stuff is open to interpretation by auditors. SOX doesn't literally specify any of this sort of stuff.

In my experience, SOX usually ends up meaning that developers don't have access to production systems, or significantly limited access. However, a continuous deployment system should generally be very much in the spirit of SOX, in that it's pretty hard to do without well-defined, highly-repeatable, automated and auditable processes.

Everyone just 'knows' what is in Sarbanes Oxley but when you ask them to point it out to you in the legislation they cannot find what they were so certain about 2 minutes prior. We have compliance people and auditors are always coming in, but when someone claims something is required for Sox compliance, challenge them on it as 99% of the time it is a convention because someone told them, or they did it like that somewhere else once, rather then what is required by law. At the least it will make them justify the compliance/overhead they are causing you to do as an engineer.

Here is the legislation if you want to read through it or use it to challenge someone's assumptions about the Sarbanes-Oxley; http://www.sec.gov/about/laws/soa2002.pdf

I don't know if that's a SOX law. However, I do know that it is a PCI requirement. A single person shouldn't be able to introduce new code and then be able to push their own change out to production.

Can you point to the bit in the PCI spec that says that? My understanding is that people should only have access to the systems they require. But that doesn't stop a developer having access to a continuous deployment server that can push code that meets requirements to production. But that's based on my memories, and may not reflect reality.

It hasn't slowed us down at Netflix. :) In fact what it has done is caused us to be really good about separating what needs auditing from what doesn't, so that only a very minimal set of services has to have separations and release processes that are in line with SOX controls.

That's good to hear.

I believe you're somewhat mistaken. SOX generally applies to finance systems and financial reporting at public companies. So if you're publicly traded you couldn't use this process for your accounting system. But if Facebook wants to let a junior engineer push out new code without independent review, SOX isn't stopping them.

Amazon generally doesn't have release teams (except for the retail website).

I can't find it right now, but I know someone gave a talk on Apollo, which is how Amazon does deployments.

I think I know the thing you mean, but I can't find it either.

I did find 'flippers' from Flickr[1] which sounds like a very similar "keep it all in master and turn stuff on and off" methodology.

[1] http://code.flickr.com/blog/2009/12/02/flipping-out/

Here we go:

http://news.ycombinator.com/item?id=2971521

I think you might be looking for this: http://www.youtube.com/watch?v=dxk8b9rSKOo

Ars has an interesting article about Facebook's release engineering process [1] with a discussion on HN.

My impression is that the different scale and technology mix at FB would make it difficult for them to deploy as frequently as github.

[1] http://arstechnica.com/business/2012/04/exclusive-a-behind-t... [2] http://news.ycombinator.com/item?id=3803026

Reply on news.ycombinator.com