If you do a commit and find out in the middle of the day the latest deploy is having problems and people are still committing in new code wouldn't this make things much harder to narrow down?
If you do a commit and find out in the middle of the day the latest deploy is having problems and people are still committing in new code wouldn't this make things much harder to narrow down?
Most problems will be discovered by someone and reported in an hour, and most of those will also be discoverable in a dataset on a system like Scuba - https://www.facebook.com/notes/facebook-engineering/under-th... - and you can identify the first time that particular issue happened.
If you're lucky, it lines up exactly to a commit landing, and you only need to look at that. Otherwise, due to sampling, maybe you need to look at two or three commits before your first report/dataset hit. You can also use some intuition to look at which of the last n commits are the likely cause. A URL generation issue? Probably the commit in the URL generation code. You'd do the same thing with a larger bundled rollout, but over a larger number of commits (50, in the case of a daily push).