Perhaps this helps: https://quine.sh/
I have no experience with the platform myself, just bumped into it by coincidence when my OSS project (written it Typescript) was featured
48 karma · joined January 5, 2019
Co-founder of Garden (https://docs.garden.io)
Feel free to contact me on these topics at eythor@garden.io
I have no experience with the platform myself, just bumped into it by coincidence when my OSS project (written it Typescript) was featured
Admittedly, documentation and stability weren’t quite what we’d like and we’ve done a massive overhaul of the foundational pieces in the past 12 months.
If you want to share feedback I’m all ears, my email is in my profile.
Similarly if another dev (or a CI runner) triggers a build of one of our services, I won’t have to next time I start my dev environment. And because it’s built in the cloud there’s no “works on my machine”.
Same applies to tests actually. They run in the cloud in an independent and trusted environment and the results are cached and stored centrally.
Garden knows all the files and config that belong to a given test suite. So the very first CI run may run tests for service A, service B, and service C. I then write code that only changes service B, open a PR and only the relevant tests get run in CI.
And because it’s all in prod-like environments, I can run integ and e2e tests from my laptop as I code, instead of only having that set up for CI.
Code sits on our laptops but live syncs to the remote services without requiring a Docker build or K8s deploy. It really does feel like local.
In particular it lets us do away with the commit-push-pray cycle because we can run integ tests and beyond as we code as opposed to waiting for CI.
We use Garden, (https://docs.garden.io) for this. (And yes I am afilliated :)).
But whether you use Garden or not, leveraging the power of the cloud for “inner loop” dev can be pretty amazing with right tooling.
I wrote a bit more about our experience here: https://thenewstack.io/one-year-of-remote-kubernetes-develop...
Your pipelines should mostly be CI provider agnostic and runnable from anywhere, including your laptop during development.
If you ever need to change a CI provider, you just move your pipelines.
I'm definitely biased, I'm working at Garden[0] which allows you to do just that but it's mostly applicable for teams using Kubernetes. I paused at your comment because Garden has sometimes been called "a makefile for the cloud".
Whatever tooling you use, having portable pipelines that you can iterate on and debug from your laptop is the only sane approach imo.
[0] https://docs.garden.io/overview/use-cases#faster-simpler-and...
We did indeed have a reputation for being a bit more complicated then some of the other tools out there (and in turn more flexible). But we've put enormous effort into simplifying Garden itself and also improving documentation.
If you already have your stack containerised you should be good to go but experience with Kubernetes is definitely a plus. Most of our users use K8s in production and use Garden for developing and testing in production-like environments in "inner loop" dev and CI.
You can check out our quickstart guide, it can get you up and running in a few minutes: https://docs.garden.io/getting-started/quickstart
We're also always happy to help on Discord if you have questions: https://discord.gg/FrmhuUjFs6
Which is pretty fantastic except for how walled off they are. You can't really re-use these descriptions for e.g. development, they're not vendor agnostic, and they only way to run them is by pushing your code.
Maybe it's a silly analogy but it's almost like being a web dev that doesn't have a browser and needs to send their code to a friend who can tell them if that font size looks good.
I think we're way over due for freeing these "blueprints" of our system from the confines of CI and making them portable and flexible. And containers are the technology that's enabling that.
Full disclaimer (as always): I work at Garden[0] where we're also solving that problem but taking a slightly different approach to Dagger (it's still a DAG). Garden config is declarative and the jobs (we call them actions) have a semantic meaning. You can e.g. have a Build action of type container or a Deploy action of type Helm and Garden will figure out what to do with it.
We've also put a lot of emphasis on the inner loop development story with hot reloading functionality, log streaming and more.
Just to add to what Tao was saying, our pipelines are not only portable but also "smart".
Instead of having to specify every step of a job—either in code or config—you instead tell Garden that it's e.g. a build of type container, or a deploy of type Helm, and our plugin system figures out the rest (of course with escape hatches when needed).
We also track the files that go "into" each job and cache the results, so the same job never has to run twice if the code doesn't change. So if you have a large distributed system and a change only affects a small part of it, you don't need to re-run everything. It can save _a lot_ of time.
Unrelated, but love that the Cicada team created a Treesitter grammar for Neovim!
And if I'm not mistaken, two of us worked briefly with @halldorel (commented below) at an earlier Icelandic start-up. It's a small world (if you're Icelandic).
Edit: Scratch that, three Gardeners worked with @halldorel.
It really is a fantastic piece of technology and completely transformed the whole experience (we were using good 'ol rsync before). In particular it works seamlessly across platforms.
If anyone's interested in how we use it, it's here: https://github.com/garden-io/garden/blob/master/core/src/plu...
It's zero effort which is important for a small team like ours. Allows us to focus on the content as opposed to bikeshedding design.
Overall I'm happy with the look and feel of things and the support is typically good.
That being said, they recently shipped changes that essentially made the docs site impossibly slow for a few days. They've been working on fixing that and it's better, but not as snappy as before. I also preferred the previous look (it's very similar but the new one is a bit more clunky imo).
We do have a lot of long code examples (YAML reference docs) which I think may contribute to the "sluggishness".
But overall I'd recommend if you want to minimise effort and maintenance. In any case it's easy to give it a spin and see if it works for you.
Hi all, we're Garden[1].
Our mission is to keep developers productive and happy in the cloud-native era.
Docker, Kubernetes and other cloud-native technologies have made us better than ever at operating our systems in production, but the day-to-day development experience has been left lagging behind.
Developers spend less time in flow, and more time waiting for builds, debugging scripts, or otherwise fighting their tools. We're here to fix that.
Our platform—which includes our open source core product[2]—allows developers to work on distributed systems with remote, production-like dev environments, while enjoying the same fast and frictionless feedback loops we've come to expect when developing a single service locally.
It's a platform that democratizes the kind of advanced developer tooling that only the largest software companies have the resources to build and maintain.
We're still a small team and you'd be joining at a time where you can have a huge impact on our product and our culture. If you've ever thought to yourself that something's not right with our modern development workflows, now's your chance to help fix it!
We recently re-wrote the hot reload functionality to use Mutagen[2] under the hood and it's insanely fast (<200ms anecdotally). It also does two way sync which can be useful. The old implementation used rsync but a lot of our Windows users struggled with that. So I figured I'd share in case that sounds familiar.
What happens after a sync event depends on the stack, but we've had pretty good success with Entr[3]. We often have it watch a single file so that multiple watchers in a shared dev cluster don't eat up all the node's resources.
1: https://github.com/garden-io/garden 2: https://mutagen.io/ 3: http://eradman.com/entrproject/
But to your point, abstractions are hard, and we've tried to design Garden such that it takes care of the _what_ and _when_ but allows the users to decide the _how_.
You might want to check out Garden for that (https://docs.garden.io/basics/how-garden-works). It thinks about your stack as a graph and manages build and deploy dependencies. And it supports the use case you're describing, i.e., you can combine a Docker image and a set of K8s manifests into a single "module" and the manifests can reference the Docker image.
Full disclosure: I'm affiliated with the project.
You define one part of your system at a time and Garden compiles that into a graph of modules that it knows how to build, test, and deploy. This means you can adopt it piecemeal and use it to run an entire CI pipeline from any given context, e.g. your local machine, VM or actual CI for that matter.
Full disclosure, I'm affiliated with project but your comment describes a problem we see from a lot our users so I couldn't help but replying.
But we've had a lot of success with running on GKE. The tool that we're making takes away the complexity of building, testing and deploying the stack, and GKE takes care of running it.
In fact we use GKE for both our development and staging/prod environments.
There are a lot of great tools out there that vastly improve the K8s developer experience[1] and the cloud providers take away of the pain of operating it. And with tools like Terraform and Pulumi you can codify the whole setup.
Here's an example of how you can quite easily get started on GKE: https://medium.com/garden-io/gke-and-cloud-sql-a-complete-wo...
Here's a video of the same workflow: https://www.youtube.com/watch?v=iHyeD97GrE4.
[1]
https://github.com/GoogleContainerTools/skaffold
This issue is that our example projects are bundled with the main project and they can fall out of sync from the latest stable release. We also version the examples and point to those in our docs but of course the `master` version is always what the users see first on GitHub. We should add a note on this in our main GitHub readme.
And I do feel your pain when it comes to (some) software geared towards other developers. I think UX (and especially UI) tend to take second place to just pure functionality. We try and put a lot of effort into the developer experience but of course we slip up every now and then :)
I hope you'll give it a second chance, now that it's been fixed.
In no particular order:
https://github.com/GoogleContainerTools/skaffold
Garden co-founder, so perhaps (likely) a little biased[0].
Depending on your needs, Garden and Tilt are both great solutions, with slightly different philosophies.
It's easy to get started with Tilt if you already have the Kubernetes manifests. If not, Garden might better suit your needs since its "container" module type allows you to provide only a minimal description of your module which Garden then maps into K8s manifests. So you won't have to write the full specs for 20+ services. (Garden also supports Kubernetes manifests and Helm charts if that's your jazz).
Garden will also manage the dependencies between all your services (at a granularity of builds, deploys, tasks, and tests). So if service12 needs to be built before service20, and service18 needs to be deployed before service5, but only after the database migration for service2 is completed, Garden can take care of that. This also improves dev speed since Garden knows exactly what to re-build and re-deploy as you make changes to your source code.
And on the topic of dev speed, Garden supports both hot-reloading and in-cluster building which allows you to share build caches with the rest of you team.
In general, there are a lot of interesting tools in this space and it's evolving rapidly. So even though developing on Kubernetes on your local laptop _can_ suck, developing distributed systems doesn't have to.
You're right, you shouldn't have to design your project around your tools. Rather you describe the individual components of your system, really just document them in a structured manner, and let your tool manage the resulting graph.
Here's a link the Github repo in case you're interested: https://github.com/garden-io/garden. Always happy to get some feedback!
I just wanted to chime in on the points above. Regarding 2), Garden does indeed support updating pods without re-building and re-deploying via our hot-reload feature (https://docs.garden.io/using-garden/hot-reload) which essentially copies source files into the running container on file save (works best for dynamic languages).
Regarding 3), Garden has a terminal UI that shows the status of individual services and updates as changes are made to the codebase. It will for example print error messages for failed container builds and failed deployments. If configured so, Garden can also run tests on code changes and will print the error output if tests fail. Our next release will also contain the first version of a dashboard which displays service statuses and dependency graphs and updates in real time. However, our terminal UI is not interactive like Tilt’s—which looks really nice!