Benefits of Continuous Delivery
henrikwarne.com
henrikwarne.com
[1]: http://www.androidpolice.com/2016/07/23/new-play-store-tools...
[2]: https://developer.apple.com/library/content/qa/qa1779/_index...
That way you get some of the benefits of CD, but you can still do relatively infrequent releases to the "real" app store.
It's important that your deployment process is repeatable and simple (i.e. typing one or two commands into a terminal), but if a human is still kicking it off that's still a net positive over big timed releases.
You start covering the most essential needs just so they become useful and usable. Then you go ahead and iterate from there, building something which suits your team, company & product. Learning and adapting in the process.
[1] https://www.amazon.de/Continuous-Delivery-Deployment-Automat...
This is a very nice escape hatch for when you want to do manual steps (like manual testing) before a deployment, but you can still rely on the fact that deployment to prod will use the same playbook as deploying to staging did.
If you want to learn more quickly - I did a talk on the topic last week, and did my best to provide a concise overview of the most essential terms. Check out the slides for a high-level view on ci/cd [1] and deployment pipelines in general [2] if you want to learn more.
[1] https://www.slideshare.net/VladislavSupalov/automated-testin...
[2] https://www.slideshare.net/VladislavSupalov/automated-testin...
After many years in big enterprise, I've learned something important - the appearance of risk is more important than the existence of risk. Continuous delivery looks "risky". Slow, deliberate release cycles on a quarterly or even yearly basis look "safe", because "testing".
In practice, those quarterly deployments have far too many changes embedded in them all at once. Worse, teams race to get their features in under the deadline, knowing it can be months before they'll get another chance, leading to careless coding and inadequate testing. So, based on both my experience and a little beyond-common-sense logic, slow release cycles are more risky than fast ones.
However, consider the enterprise context where I have thousands or even tens of thousands of people that have to use the software in some coordinated fashion. A new application feature may be useful under the right circumstances, but trying to be sure what the best and most appropriate uses of a new feature in a large organization often times just cannot be an ad hoc affair. Part of the job of enterprise business applications is to centralize information and make processing that information more efficient. When the usage of that system in the field is not consistent however, then you lose a lot of that benefit.
I'm sure the author wasn't thinking about the boring old enterprise, but without context in the article it's difficult to say whether the advice is meant to pertain to the consumer focused startup or all comers.
Which repo should own the integration tests? How do I synchronize the releases of matching front and back ends?
For REST apis, use semantic versioning, and you can embed specific versions in the request headers. The server should be able to respond correctly to a range of different versions (an api call to say what range of versions the server speaks isn't a bad idea). Coarse-grained api revisions can be embedded in the url (the ever popular /v1 and /v2, etc).
If it were in a true client/server environment, where you had a variety of different client versions rolled out, then I expect you would have already found the obvious answer: the server has to support multiple versions of clients and clients use a mechanism like feature switches or capability detection to enable functionality as it becomes available server side. Both repos have their own integration tests: the server's make sure the server supports multiple client versions, and the client's make sure that the client degrades gracefully in the face of server variation.
From the way you talk, though, it sounds to me like you expect client and server to always be released at the same time (e.g., where it's a web front end and web back end). If that's really the case, then I'd just have everybody work as one team, working off a common set of features switches.
Is that helpful?
It is helpful, but certainly doesn't sound simple. Feature switches would require the messages to already be designed, which is most of the work already done. It also sounds like an edge-case nightmare.
I usually put integration tests in the client repo but it doesn't matter. The key is that you put in a trigger so they get run by changes to any of the projects.
You don't want a lot of half-baked stuff hanging around, so you should definitely limit WIP and be aggressive about removing switches once you're sure you won't be switching something off. Without that, it could indeed be nightmareish. But branching can also get nightmareish. Either way, the trick is to minimize complexity.
- app-backend
- app-frontend
- app-bundle (or simply "app")
Your backend and frontend can be independently developed and use semantic versioning, they could run their independent checks like linting and unit tests.
The bundle repo could consist of a docker compose file that uses image versions of backend and frontend that are compatible with one another. You can choose to anchor the version to the minor, or revision etc. This way you can bump your versions when they are ready.
Also, pushes to your backend or frontend could trigger a build of the bundle repo. I would run there the integration tests.
That being said, if the app is relatively simple, I would probably pack everything up in one repo to avoid the overhead.
> If the change deployed is small, there is less code to look through in case of a problem. If you only deploy new software every three weeks, there is a lot more code that could be causing a problem.
That's relative. Pushing out an accumulated amount of small changes once a week will most likely have the same end the result. The difference is, if you commit more than one breaking change you are dynamically expanding the window of service degradation. One release with three breaking changes is better than three broken pushes.
> If a problem can’t be found or fixed quickly, it is also a lot easier to revert a small deploy than a large deploy.
It is also harder to revert two non-consecutive deploys out of three.
> If I deploy a new feature as soon as it is ready, everything about it is fresh in my mind. So if there is a problem, trouble shooting is easier than if I have worked on other features in between.
Personally, I favor stability vs. easier troubleshooting. This works for some products and not others.
> It is also frees up mental energy to be completely done with a feature (including deployed to production).
Anecdotal evidence, but my team would usually catch and correct bugs when they have to come back to green light a production push. Engineers that ship clean and fast are rare.
> All things being equal, the faster a feature reaches the customer, the better. Having a feature ready for production, but not deploying it, is wasteful.
Something like this would usually be pushed out manually to align with other non-engineering parties within your company. Pushing broken features to the customer faster is not a good thing. Unless you can assume 100% success rate; which is not possible.
> The sooner the customer starts using the new feature, the sooner you hear what works, what doesn’t work, and what improvements they would like.
This depends on the stage of the company, the product, and your customers.
> Furthermore, as valuable as testing is, it is never as good as running new code in production. The configuration and data in the production environment will reveal problems that you would never find in testing.
All of the environments I govern match production 1:1 (sans data sanitation) in every way possible. I feel pretty strongly about this, if you can't test your code without pushing it into production, you should not be automating anything.
> Continuous delivery works best when the developers creating the new features are the ones deploying them. There are no hand-offs – the same person writes the code, tests, deploys and debugs if necessary. This quote (from Werner Vogels, CTO of Amazon) sums it up perfectly: “You built it, you run it.”
Don't compare a start-up to Amazon. Amazon has dedicated teams to govern the process and you most likely not replicate that. Also, hiring people that 'just send it' without doing damage takes money, time and a lot of training. It's expensive.
Why? Each of those pushes you have one thing to check, and if it is messed up only one thing to revert. With a batched release you have multiple things to check, and are reverting other people's working stuff when you have to revert. Even worse, you have to choose between reverting slowly (but checking every feature) and possibly having to revert a second time because there was another bug you missed!
> Personally, I favor stability vs. easier troubleshooting. This works for some products and not others.
I don't understand. If you make the same number of changes with the same number of breakages, is packing them into a smaller window really more stable? Even worse the more time it takes you to fix those breakages, the less uptime you have... The opposite of stability.
> All of the environments I govern match production 1:1 (sans data sanitation) in every way possible. I feel pretty strongly about this, if you can't test your code without pushing it into production, you should not be automating anything
I agree with this! But... Then why are you advocating for staging to digress further from production waiting for a big release?