Dagger: a new way to build CI/CD pipelines
dagger.io
dagger.io
You've raised $27m to what? Give away a free tool for engineers to define their builds in? While they continue to use and pay for another service to actually run the builds? I assume you intend to replace the CI service at some point and move up the stack to monetize?
Without more transparency its easy to imagine something like...
Step 1. Drive uptake of your tool by selling people on the pitfalls of "CI lock-in" Step 2. Introduce your own CI solution, which people can now easily switch to Step 3. Lock people in
However, there is a great opportunity to help businesses manage their software supply chain. What is running where, and how did it get there? Who is authorized to run which pipelines on which machines, and which which inputs? What is the diff between this dev environment and staging? Staging and production? Etc. Keep in mind this is not limited to your production pipeline; there are staging environments, development environments, all running pipelines all day long. It's hard to simply know everything that is going on.
Each time Dagger runs your pipeline, it can produce telemetry about every aspect of your supply chain. Git SHAs, docker image checksums and cryptographic signatures, specific versions of kubernetes and cloudformation configurations, before and after templating, test results, monitoring status... It also integrates with your secrets management backends, your developer's local source code. Basically every node in your supply chain can be a node in your DAG if you want it to. The logical next step is to give you a centralized place to send all this telemetry, and tools to extract insights from it. You could also perhaps manage the configuration of your various Dagger engines in one location.
Another product that is often requested is a visual DAG debugger. When a pipeline break, you want to know why, and staring at your CI logs is definitely not the best experience for that. With a web UI, there's a lot we can do there.
The business opportunity boils down to this: if CI/CD pipelines are software, there ought to be a platform for developing that software, and an ecosystem of developers creating value around that platform. Dagger aspires to create that missing developer ecosystem. If we succeed, there will be no shortage of business opportunities. If we fail, none of the other features will matter.
To me, that still doesn’t seem to go to the core of what value it brings to table. I understand that Dagger can do all this, and that businesses would like to know what runs where and how everything interacts, but… it doesn’t explain to me the CI / build pipeline angle?
How does knowing the telemetry around my build pipelines translate to better software, or cost savings, or other improvements?
If there’s a registry to exchange “components” of a build pipeline, what does that bring me what a regular python / java / etc package can’t?
Don’t get me wrong, I think there are plenty of problems with CI (I have a ~200 job CI pipeline on fire in front of me), I just can’t seem to connect the dots here. :)
question coming from a sales guy among this crowd... Who does this impact the most and what outcomes does that help them achieve? At the end of the day, you need to pay the bills, for you and your team, who is going to sign the dotted line and tell you that yes "I need to invest $100k on this because it solves a major pain!"?
Whilst I can see this as a nice to have, I'm having a hard time understanding who your target market is going to be...
At the end of the day, I presume this is going to end up becoming a full fledged company, with its sales team, marketing and what not. What are you thinking about in terms of revenue channels so far ?
I'm curious to understand what the pricing is going to be and whether it's going to be well recieved or not (and right now, there is nowhere on dagger.io's website where pricing is mentioned)
Yes, very much. There are so many layers, components, and their intricate relations that goes totally ignored today at least in most places. Because, doing so is insane amounts of work. Only BigCos can afford to have dedicated teams for 's/w supply chain management', considering the cost-parity-with-returns. However, the solution on this end that works for BigCo doesn't necessarily work for SMEs & startups. That gap isn't small, if am right.
> Another product that is often requested is a visual DAG debugger. When a pipeline break, you want to know why, and staring at your CI logs is definitely not the best experience for that. With a web UI, there's a lot we can do there.
Yes. This definitely helps. But more than a viz DAG element, people look for an early-warning of a failure. Most common build-failure reasons (other than failed tests) -> expired creds used somewhere in the pipeline, provisioning failed/time-out, problem at some other dependent module totally outside org's control (some OSS/dep). People seem to be bothered equally about how to squash'em rather than just where to squash. Locating the part where pipeline broke is just half the part. Actionable insights as to how that pipeline can be healed is the hard part. And considering the diversity of the ecosystem, that's gonna be a wild ride.
BTW, are you folks hiring? "DevOps OS for enterprises" seems very very enthralling, esp for an old toolmaker.
took a while for E/AKS to catch up with G, things like ingress still vary in DX across clouds
'the same everywhere but it only works here'
(no comment on morality of this, quality of kube generally, or whether this team will do the same. docker IMO missed the chance to be a cloud host or a standard interface)
Sure each provider does things differently, but they still work with primitive Kubernetes manifests at the end of the day.
A migration from one to the other is nothing more than changing some annotations or potentially transforming the "shape" of some lists or maps.
same as like, aws extending mysql and postgres to provide RDS features -- they don't open source those pieces, so 'stock oss' DBs don't have fancy RDS features like backup + restore
with kube, platform-integrated components like network, ingress + storage 1) require custom work on each cloud, so new features won't be available on all clouds at the same time.
But also 2) that means some components, ingress specifically, have different interfaces on different clouds when they do finally get implemented.
That was also the initial goals of k8s - make it universal so it's adopted as a standard by everyone else to make yourself competitive against the AWS juggernaut. And it worked.
Assuming that dagger does take over the world, this has the docker story of "this is a widely used tool that a bunch of people are using and we can't monetise without resorting to lock-in", which is a huge bummer.
If you deal with proprietary toolchains that are tens of gigabytes (Windows WDK, Xilinx Vivado/Vitis) it’s just untenable, and that’s before even mentioning licensing. Even Azure doesn’t have a great solution for WDK development. It’s hard to feel like we’re not being left behind.
Bind mounting the tools into the container is an option but at that point you’re using the container just because that’s what your CI expects, not because it’s any better or more reproducible than a raw Jenkins shell script.
I feel like there's a ton of innovation in the cloud/container CI/CD space, but next to nothing elsewhere. In fact, some of the innovations in CI/CD make things _more_ difficult for those of us developing in other environments (such as game development).
There's a lot of low hanging fruit for improving things elsewhere.
To me, its like they don't have branches (tons of new code not accessible in the game is released in a patch) and they don't have unit tests that catch bugs (a certain ability has 2 modes, with 2 different activation times. In the most recent patch, the activation times ended up the same) but it's possible that developing a game is far different than developing a SaaS product from a coding perspective. Or it could be that studio just has weird practices.
Client side encryption is basically impossible; the decryption key has to be in the executable you ship somewhere, or at least sent to the client after a connection with the server. Perhaps some encryption tiering system could work to keep unreleased locked content locked for longer but I don’t think anyone has gone through that trouble yet.
This is likely by choice, feature flags have a fair few advantages over feature branches and are generally far less of a pain point long term.
It's likely because of inertia. I don't work for EA, but the majority of games studios use Perforce for source control which has... awful support for branches. They've got streams which are a huge improvement, but still nowhere near as flexible or easy to use as branches in git.
It’s possible in file explorer via \\wsl$, but that is not always supported by applications so it’s not 100%.
Oh, hey, there's the C drive in WSL! ;)
ln -s /mnt/c/Users/Somejerk/Documents ~/documents
Oh, man, a shared documents folder!
Granted, access to those files is slow as dirt because of the Plan9 filesystem, and there are some weird bugs where a process sometimes loses access to $cwd if it's not a native wsl filesystem (also a Plan9 bug reported to MS over a year ago). But it's tolerable when interoperability is necessary.
It also facilitates using the same filesystem under multiple WSL instances.
iwr -useb get.scoop.sh | iex # install scoop
scoop install coreutils vim nano [...] # yay
But in my case I wanted to leave something small (and busybox.exe is that small) /portable - for others to use (without the requirement to install scoop).
Who on their right mind uses ssh/scp on Windows development other than connecting to UNIX boxes?
Exactly. That's like complaining that I can't RDP into a linux box to install the toolchain!
Experienced Windows devs I talked to said they used Packer from Hashicorp to entirely recreate their server image whenever they wanted to deploy. This process takes hours, but that was the best I found.
It doesn't seem too hard.
https://docs.microsoft.com/en-us/windows-hardware/drivers/do...
Show me how to use this with GitHub actions, if it’s not too hard.
Depending on your requirements/scale the runner(s) can be a VM in your main machine, a cheapo dedicated server (hetzner/ovh) or even autoscaled hotspot instances in your preferred cloud.
It is less pure than using GH's runners and having an end-to-end setup/teardown for your whole toolchain, but it would work just fine. Definitely better than without any CI.
It's made me super grateful to be where I am now, and able to even view tools like this as an option.
If you have any questions, I'll be happy to answer them here!
I'm very confused by this sentiment. This approach loses the best of existing CI tooling does it not?
Jenkins users lose understanding of what's actually being executed, what stage they're in, and how long it took. It might provide convenience, and the a (great) benefit of running locally the same as in your CI environment, but it seems to me this would make it difficult for devs to easily understand where/why their build failed, since it just has one megastep.
Where is dagger UI and how does it relate to your CI? I don't see it in docs or cli help. Sounds like Dagger UI in this context (above) is providing little value beyond logging if it's not doing workflow execution.
I ask not to talk down on the product, but because I'm actually quite interested. Local execution, plus containerized execution sounds awesome. Just trying to understand the vision.
In the “megastep” approach, it boils down to which tool can provide the most useful information. Jenkins is more mature but Dagger has more information about the DAG. So in some cases developers might actually prefer using Jenkins as a “dumb” runner infrastructure. It depends on the situation.
How mature is this? We have a 20 person team and we're prototyping different pipelines for our next CI/CD pipeline (currently Heroku). Is this ready for production workloads?
That said, one nice aspect of Dagger is that you don't need to rip and replace your entire system: you can use it in a localized way to solve a specific problem. Then expand over time. It's similar to writing a quick shell script, except it's easier to reuse, refactor and compose over time.
We will not rely on unusual licences to restrict competitors from running Dagger as a service. We don't need to, since `dagger` is a client tool. We do encourage cloud providers to run buildkit as a service, though :)
Generally, we take inspiration from Red Hat for their balancing of open-source community and business. They are very open with their code, and tightly control how their trademark is used. You can clone their IP and use it to compete with them - but you have to build your own brand, and you can't confuse and fragment the Dagger developer community itself. We think that is a fair model, and we are using it as a general guideline.
Your "Trademark Guidelines" appear to contradict you:
> Third-party products may not use the Marks to suggest compatibility or interoperability with our platform. For example, the claims “xxx is compatible with Dagger”, “xxx can run your Dagger configurations”, are not allowed.
> but you have to build your own brand, and you can't confuse and fragment the Dagger developer community itself
If I do an incognito Google search for "dagger", the first result is the Wikipedia page for the knife, and the second result is for Dagger, the dependency injection tool. By naming this "Dagger" you're confusing not just your own developer community but the pre-existing one as well.
> Your "Trademark Guidelines" appear to contradict you:
They do not. Software licenses and trademark guidelines are two different things. Some commercial open-source vendors have changed their licenses to restrict use of the software in various ways - typically to limit competition from large cloud providers. We don't do that, and have no intention to. Our license is OSI-approved and we intend to keep it that way. That is what I am referring to.
> but you have to build your own brand, and you can't confuse and fragment the Dagger developer community itself
This is the intent behind the language in the trademark guideline which you quoted: you can redistribute and modify our code. But if you distribute a modified copy, call it something else.
> > Third-party products may not use the Marks to suggest compatibility or interoperability with our platform. For example, the claims “xxx is compatible with Dagger”, “xxx can run your Dagger configurations”, are not allowed.
> By naming this "Dagger" you're confusing not just your own developer community but the pre-existing one as well.
I disagree. Dagger has existed in private beta for over a year, thousands of engineers have been given access, and I can't remember a single instance of any of them being confused by the name. We have registered the trademark, and nobody has raised an issue.
> > Your "Trademark Guidelines" appear to contradict you:
> They do not. Software licenses and trademark guidelines are two different things. Some commercial open-source vendors have changed their licenses to restrict use of the software in various ways - typically to limit competition from large cloud providers. We don't do that, and have no intention to. Our license is OSI-approved and we intend to keep it that way. That is what I am referring to.
I'm glad the product is open source, but that provision isn't in the context of source code, it is a top-level item listed on that page. That's why I interpreted "unusual licences" to generally mean a sort of "legal acrobatics".
When you're threatening people with legal action you need to be clear, and right now the text on that page is not, according to what you're saying here. I doubt many people are going to be searching Hacker News comments for the true intent behind these guidelines.
> I disagree. Dagger has existed in private beta for over a year, thousands of engineers have been given access, and I can't remember a single instance of any of them being confused by the name. We have registered the trademark, and nobody has raised an issue.
I don't think that really addresses the point. Dagger (as started under Square) is nearly ten years old, and Google's 2.0 fork is from 2016. It's used by thousands of published Maven artifacts, countless applications, and tens of thousands of developers (at least). This is the first time I've heard of your project, but that's bound to happen in tech. Whether you registered it or not without complaint doesn't much matter either, the issue is being raised here, now that you've publicly launched.
That's good feedback, thank you. We can try and make it clearer as long as it remains legally correct and enforceable. Do you have specific feedback on which parts you found unclear, and why?
https://www.gfrlaw.com/what-we-do/insights/beyond-brand-x-us...
So as long as they don't imply endorsement, "I'm compatible with X seems fine".
Seems to assume that all CI/CD workflows work in a single container at a time pattern. How about testing when I need to spin up an associated database container for my e2e tests. Is it possible, and just omitted from the documentation?
Not familiar with cue, but can I import/define a common action that is used across multiple jobs? For example on GitHub I get to duplicate the dependency installation/caching/build across various jobs. (yes, I'm aware that now you can makeshift on GitHub a composite action to reuse)
Can you do conditional execution of actions based on passed in input value/env variable?
Any public roadmap of upcoming features?
Dagger runs your workflows as a DAG, where each node is an action running in its own container. The dependency graph is detected automatically, and all containers that can be parallelized (based on their dependencies) will be parallelized. If you specify 10 actions to run, and they don't depend on each other, they will all run in parallel.
> How about testing when I need to spin up an associated database container for my e2e tests. Is it possible, and just omitted from the documentation?
It is possible, but not yet convenient (you need to connect to an external docker engine, via a docker CLI wrapped in a container) We are working on a more pleasant API that will support long-running containers (like your test DB) and more advanced synchronization primitives (wait for an action; terminate; etc.)
This is discussed in the following issues:
- https://github.com/dagger/dagger/issues/1337
- https://github.com/dagger/dagger/issues/1249
- https://github.com/dagger/dagger/issues/1248
> Not familiar with cue, but can I import/define a common action that is used across multiple jobs?
Yes! That is one of the most important features. CUE has a complete packaging system, and we support it natively.
For example here is our "standard library" of CUE packages: https://github.com/dagger/dagger/tree/main/pkg
> For example on GitHub I get to duplicate the dependency installation/caching/build across various jobs. (yes, I'm aware that now you can makeshift on GitHub a composite action to reuse)
Yes code reuse across projects is where Dagger really shines, thanks to CUE + the portable nature of the buildkit API.
Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :)
> Can you do conditional execution of actions based on passed in input value/env variable?
Yes, that is supported.
> Any public roadmap of upcoming features?
For now we rely on raw Github issues, with some labels for crude prioritization. But we started using the new Github projects beta (which is a layer over issues), and plan to open that to the community as well.
Generally, we develop Dagger in the open. Even as a team, we use public Discord channels (text and voice) by default, unless there is a specific reason not to (confidential information, etc.)
> Note: you won't need to configure caching though, because Dagger automatically caches all actions out of the box :)
Is this strictly because it's using Docker underneath and layers can be reused? If so, unless those intermediary layers are somehow pushed/pulled by the dagger github action (or any associated CI/CD tool equivalent), experience on hosting server is going to be slow.
Sidenote, around 2013 I've worked on a hacky custom container automation workflow within Jenkins for ~100 projects, and spent considerable effort in setting up policies to prune intermediary images.
Thus on certain types of workflows without any prunning a local development machine can be polluted with hundreds of images, unless the user is specifically made aware of stale images. Does/will dagger keep track of the images it builds? I think a command like git gc could make sense.
> Is this strictly because it's using Docker underneath and layers can be reused?
Not exactly: we use Buildkit under the hood, not Docker. When you run a Dagger action, it is compiled to a DAG, and run by buildkit. Each node in the DAG has content-addressed inputs. If the same node has been executed with the same inputs, buildkit will cache it. This is the same mechanism that powers caching in "docker build", but generalized to any operation.
The buildkit cache does need to be persisted between runs for this to work. It supports a variety of storage backends, including posix filesystem, a docker registry, or even proprietary key-value services like the Github storage API. If buildkit supports it, Dagger supports it.
Don't let the "docker registry" option confuse you: buildkit cache data isn't the same as docker images, so it doesn't carry the same garbage collection and tag pruning problems.
IIRC doesn't buildkit store its cache data as fake layer blobs + manifest?
I don't see how it can avoid the garbage collection and tag pruning problems since those are limitations of the registry implementation itself.
Dagger currently doesn’t help you remove blobs from your cache, but if/when it does, it will work the same way regardless of where the blobs are stored (except for the blob storage driver).
This sounds like memoization caching: https://en.wikipedia.org/wiki/Memoization
> In computing, memoization or memoisation is an optimization technique used primarily to speed up computer programs by storing the results of expensive function calls and returning the cached result when the same inputs occur again.
Re: SBOM: Software Bill of Materials, OSV (CloudFuzz), CycloneDX, LinkedData, ld-proofs, sigstore, and software supply chain security: "Podman can transfer container images without a registry" https://news.ycombinator.com/item?id=30681387
Can Dagger cache the (layer/task-merged) SBOM for all of the {CodeMeta, SEON OWL} schema.org/Thing s?
Docker's explosive growth is partly due to the convenience of the abstraction it provides, abstracting the entire linux userspace, putting even OS-wide package managers and language-specific package managers inside another box. This usually breaks any caching / code sharing that the now containerized packages had, resulting in the bloat. The docker image is portable, yes, but disk and RAM efficiency of the systems people are building are awful. It has been the norm for every little microservice to add a few GB of bloat to the overall software system. A dev writes "RUN pip install pytorch" and you have CICD servers pulling down 2GB of pytorch to build the container, every time the software is built, probably forever. Meanwhile species are going extinct and a lot of people are starting to wonder if it's ethical to work in technology at all.
What can your team do to reverse this tragedy of the commons? Can you come up with some equally ergonomic tool that can migrate the container ecosystem on to something that has a solid foundation with good caching?
It's tragedies all the way down :-)
Making a successful tool in a competitive space is hard enough. Asking a creator to somehow factor in environmental impact isn't going to work. A creator that places additional constraints on themselves will more likely lose out to a competitor that doesn't.
This is the kind of thing a carbon tax is perfect for. Cost optimization is infused into how businesses and individuals operate. Tax the things you care about and watch all that machinery get to work!
Right now, people use Docker (and dynamic languages, bloated JS libraries, etc) because burning electricity is the cheapest route. (Sort of... the market's obviously not perfect.) Make electricity expensive enough and we'll get more efficient systems.
And of course taxes aren't the only solution. For example, Google Flights now shows the carbon cost of each flight option, which I think might actually move the needle. But that category of solution might only work in an industry with similar characteristics, e.g. infrequent but large purchases, mature industry that can define measurement standards.
That attitude doesn't leave a great impression for me. I take your points about how difficult it is, but I think we can all do better than throw our hands up in the air. For instance, you could talk about how easy containerization of CI/CD makes it easier to move your pipeline where impact is lowest. Or that you can control your own impact rather than leave it up to the whim of someone like CircleCI.
There's no silver bullet with environmental impact, which is why we all have to collectively apply whatever wins we can, wherever we can.
I think the ideal way is to let a startup thrive any way possible and when they're not trying to not starve anymore, begin environmental changes.
Oh yeah -- I'm not at all saying Docker is all bad.
* Like you mentioned, increasing interoperability allows the market be more efficient.
* Docker continued the path that VMs started towards making strong isolation even more efficient and accessible.
* Layers and caching are obviously good for resource consumption.
It's just that the original comment seemed to try shaming the Docker creators about what they've built. All they did was try and make something better. And if they didn't, someone else would have.
Can you share some evidence to support this claim? Who thinks we're better off addressing climate change, etc. with less technology?
The solution is, in a nutshell, to electrify everything, and push to make electricity clean and plentiful. Anything else is doomed to fail because we can't beat climate change by reducing our carbon footprint; we have to eliminate it entirely.
From that perspective, cloud energy usage is not a problem, since it's already electrified by nature. Now we just need to stop emitting carbon in order to make electricity (among other things).
Untrue, in every way.
Why did you say this?
> curl https://dl.dagger.io/dagger/install.ps1 -OutFile install.ps1
This uses the `curl` alias, which is really `Invoke-WebRequest`. It also makes the incorrect assumption I haven't fixed this dumb Microsoft mistake to be an alias to actual curl.exe.
> [Windows] We try to move the dagger binary under C:\Windows\System32
Ack, please don't do this! This is similar to installing something under /usr/sbin/. Malware is the only modern thing that would attempt to deploy to that folder.
> but in case we miss the necessary permissions, we'll save everything under <your home folder>/dagger > C:\<your home folder>\dagger.exe
I'm glad you have a non-admin fallback, but also: yuck. I don't want this polluting my home folder (more importantly: I don't want 100's of other things like this also polluting my home folder).
The "Windows way" is to install system-wide to %ProgramFiles%\dagger\ (eg c:\Program files\dagger\dagger.exe), or to install to %LocalAppData%\dagger\ (eg: c:\Users\shykes\AppData\Local\dagger\dagger.exe). The latter is kind of the closest equivalent to $HOME/.dagger on linux. Add whatever folder to the user's PATH environment variable to make it easy to run.
Honestly, providing just the .zip is better: then Windows users can muck up their own system however they like. Alternatively, package it with something like Scoop [2] which is a fairly popular dev tool, and provides a fairly easy way to get a sane install with updates, versioning and path stuff all handled.
scoop install dagger
scoop install https://gist.github.com/vardrop/a25e0c8e2dc055f86a3ff4dd7a7de309/raw/0b7f1e29454d7d2cbe1ae9d6c807ddfcdacb7feb/dagger.json
[1] https://github.com/dagger/dagger/issues/1946#issuecomment-10...
As someone who's trying to get to grips with the Linux filesystem conventions, would you mind elaborating on a) why that's wrong, and b) what you would suggest instead? This reference[0] suggests that `/usr/sbin` is for "general system-wide binaries with superuser (root) privileges required" (and `/usr/bin` for those that don't require root privileges). I've therefore been using them in my homelab for binaries like the Cloudflare Tunnel Client[1]. Where instead should I have installed it?
* If to a "well-known location" that is commonly-used by convention, how should I find out what that is?
* If to a custom location of my choosing, how should I communicate its location to scripts/tools that _use_ that binary? I see later in your comment that you suggest "Add whatever folder to the user's PATH environment variable to make it easy to run.", but that doesn't seem like a scalable solution for a multi-user environment?
For stuff that isn't just a self-contained executable, consider installing it to the folder `/opt/$MY_APP` and either symlinking the main binary into `/usr/local/bin` or putting a wrapper script in there (if the thing doesn't like being symlinked outside of its primary install dir). The wrapper can be as simple as:
#!/bin/sh
exec /opt/my-app/bin/my-app "$@"Chocolatey is a mixed bag. Since it is basically just wrapper scripts around upstream installers, it very much depends on what the upstream installer does. To me, it acts less like a package manager (apt) and more like a rudimentary installer runner.
This causes all kinds of annoying usability issues. Chocolatey doesn't know if you uninstall via Add/Remove programs, or update via standalone or self-update mechanism, and will just show incorrect install and/or version info, and fail to upgrade properly. A lot of packages don't even install the version that Chocolatey says, but instead just install the latest at time of install.
Scoop is way different, and has none of these problems. Funny enough, they describe it as "not a package manager" but it feels way closer to apt to me. Everything is essentially a portable version of the app, and it puts them in ~/scoop/apps/[name]/[version], creates a junction (symlink) to ~/scoop/apps/[name]/current, and adds executable "shims" to ~/scoop/shims (which is in the user's path). There's no Windows "uninstall" entry, no versions to be desync'd, and no hidden garbage that can sneak in.
[1] https://github.com/ScoopInstaller/Scoop/wiki/Chocolatey-Comp...
CUE pushes nondeterminism into "_tool.cue"[0] files that are allowed to do things like IO and run external processes. Tool files scratch a similar itch to Makefiles, but they lack an integrated plugin system like Bazel (hence why I played with the idea of CUE + Bazel).
With Dagger you seem to be restricted to the set of things that the dagger tool can interpret, just like with my Cuezel tool you are limited to what I happened to implement.
In CUE `_tool` files you are also limited to the set of things that the tool builtins provide, but the difference is that you know that the rest of the CUE program is deterministic/pure (everything not in a _tool file).
There's clearly value in tooling that reads CUE definitions, and dagger is the first commercial interest in CUE that I've seen, which is exciting.
But I'm most interested in some CUE-interpreter meta-tool that would allow you to import cue definitions + their interpreters and version them together, but for use in `_tool` files to keep the delineation clear. Maybe this is where dagger is heading? (if so it wasn't clear from the docs)
Now we use CUE in a more straightforward way: as a superior replacement for YAML. There's a (simple) schema that all Dagger plans must follow; beyond that, you can import any CUE package and apply any definition into your plan. But you don't need to go upstream and annotate these packages with additional Dagger metadata.
I'm not sure if my explanation is clear - even I get confused by this embedding business sometimes.
deploy: netlify.#Deploy & {
contents: build.contents.output
}
And, yes, Dagger will gradually fill the missing values in the CUE tree during runtime. Essentially resolving the DAG on the fly. It is pretty cool :)
We have discussed this topic at length with the CUE developers. Our conclusion is that CUE's deterministic core is what matters, and the `_tool.cue` pattern is more peripheral: more of a reference for how other tools might use cue for non-deterministic operations. It's not realistic for CUE to be both a ubiquitous deterministic language, and a successful non-deterministic tool. Its priority is clearly the former, and we're focusing on the latter.
There's a funny paradox: ask people to "name 10 white things", and they do it SLOWER than if you make the task harder by asking to "name 10 white things in your fridge".
Fine. "DevOps" now means "some something development something something servers something something operations". Are you happy now, tech world? You've made Patrick Debois cry.
Before we had developer and operations teams.
Now operations write yaml and we call them devops.
But, I wish there was some code to show me what makes it so radically different. It seems like this is targeting developers (or is it devops team?) and I'm excited about the new language here, but I don't see any examples of code. Code engages both my head and heart.
I am reusing a lot of code in my CI jobs. I have an upload script that I reuse. I have a DEB package script that I reuse across many projects. So, that assertion rings false to me, and seems to indicate there is an unhealthy wall between devops and the developers that prevents shared code. Maybe I misunderstand.
The thing that always bites me is that I have trouble debugging a job that depends on the artifacts from a prior job. My e2e job is flakey, and I'm loathe to fix it, because I have to re-run the entire pipeline, test -> build -> installer, etc to get the final artifact used in the e2e job. I've not figured out a way with "gitlab-runner exec" to run a job later in the pipeline and somehow pass in my own artifacts locally. This would be something (albeit very specific to gitlab) that would make me very excited.
Sorry to be that guy but maybe try the docs page...?
It isn't hard to find, but my point was that if you say "using an intuitive declarative language" then a developer will get excited by a code snippet that shows that intuitive developer language. It wasn't there, and I think their post could be improved by having less fluffy language and more code, if they are targeting me, that is.
This link does show the code, but it took a few clicks to get there from your links:
https://docs.dagger.io/1202/plan
At first glance I'm not in love with the language. When I look at the first example, there is a lot I have questions about.
https://cuelang.org | https://cuetorials.com
Dagger builds on top of CUE and the (DAG) flow engine therein
Here's the scenario: you're the unfortunate soul who received the first M1 as a new employee, and nothing Docker-related works. Cue multi-arch builds; what a rotten mess. I spent more than a week figuring out the careful orchestration that any build involving `docker manifest` needs. If you aren't within the very fine line that buildx assumes, good luck pal. How long has `docker manifest` been "experimental?" It's abandonware.
Then I decided it would be smart to point out that we don't sign our images, and so I had to figure out how to combine the `docker manifest` mess with `docker trust`, another piece of abandonware. Eventually I figured out that the way to do it was with notary[1], another (poorly documented) piece of abandonware. The new shiny thing is notation[2], which does exactly the same thing, but is nowhere near complete.
At least Google clearly signals that they are killing something, Docker just lets projects go quiet. I'm now looking at replacing everything I've done for the past 4 months with Podman, Buildah, and Cosign.
How long before this project lands up like the rest of them? Coincidentally, we were talking about decoupling our CI from proprietary CI, seeing this was a rollercoaster of emotions.
[1]: https://github.com/notaryproject/notary [2]: https://github.com/notaryproject/notation
I just started a new job a few weeks ago, and guess what, I'm the first one (in my small group of ~5 or so) to have an M1. First it was just figuring out what the issue was, then trying UTM and manual port forwarding and getting horrible performance, then Colima which seems passable, but the default 2GB of RAM is useless for anything like kafka, so I end up having to allocate half the RAM of my machine just to docker containers.
Ever since I did that docker on Mac became bearable. That coupled with vscode remote development even allows me to easily mount volumes.
Plus I'm now less concerned about malicious 'npm post install scripts' which could potentially nuke all my data.
I plan to post a blog about the whole lot. Shoot me an email, or a tweet, or something and I'll ping you when I get round to it.
Also: Dagger can integrate with Podman, Buildah and Cosign :)
At the very least I’d want to see a comparison with what we have now, to show me how this is better.
I get that I can try to explore more, but if I don’t get a compelling reason to do so after reading the introductory post, I’m not very motivated to do so
Any motivation I do have completely hinges on the words ‘from the creators of docker’, not on the merits of this particular product itself.
It turns out, the gap between "works locally!" and "works in CI!" is not negligible, especially when you're not sure "about all that server stuff".
Getting this working locally with a fast cycle time, and then being able to easily move that into a CI environment of your choice sounds exciting to me.
Furthermore, the majority of our customer support is "I can't reproduce this locally but it's broken in CI". Everyone blames the CI tool, but it's almost never the CI tool - just drift between environments. A way to debug locally is a killer feature.
Is it worth an entire, funded company? I'm not sure, but I'm excited for them to exist!
Same old problems with configs/secrets, integration with internal/external services, and the details required by your cloud provider.
This is the sort of solution I have regularly been hired to untangle, after a company entrenches itself.
* The blog post is more high-level. It describes the very real problem of devops engineers being overwhelmed with complexity, and the promise of a more modular system, but does not provide lots of details.
* The dagger.io website does provide more technical detail. For example it talks about the 3 most common problems we solve: drift between dev and CI environments; CI lock-in; and local testing and debugging of pipelines. It also features animated code samples.
* The documentation at https://docs.dagger.io go in even more details, and walk you through a concrete example.
We do feel that we can do a better job explaining the "meat" of the tool. But we decided to launch and continue improving it incrementally, in the open. If you have any specific suggestions for improvements, please keep them coming!
That would come handy once or twice per month
- write code
- run tests
- commit code
- update CI
- commit
- CI broken
- update CI
- commit
- CI broken
- update CI
- ...
The workarounds for this are generally awful.
For Jenkins, you stage your own instance locally and configure your webhooks to use that. It's exactly as terrible as it sounds, and I never recommend this approach.
For Travis and Concourse (I think), you can use their CLI to spin up a runner locally and run your CI/CD yaml against it. It works "fine," as long as you're okay with the runner it creates being different from the runners it actually uses in their environment (and especially your self-hosted runners).
In GitHub Actions, you can use Act to create a Dockerized runner with your own image which parses your YAML file and does what you want. This actually works quite well and is something that threatens Dagger IMO.
Other CI systems that I've used don't have an answer for this very grating problem.
Another lower-order problem Dagger appears to solve is using a markup language to express higher-level constructs like loops, conditionals, and relationships. They're using CUE to do this, though I'm not sure if hiring the creator of BCL (Borgmon Configuration Language) was the move. BCL was notoriously difficult to pickup, despite being very powerful and flexible. I say "lower-order" because many CI systems have decent-enough constructs for these, and this isn't something I'd consider a killer feature.
I also _also_ like that it assumes Dockerized runners by default, as every other CI product still relies on VMs for build work. VMs are useful for bigger projects that have a lot of tight-knit dependencies, but for most projects out there, Dockerized runners are fine, and are often a pain to get going with in CI (though this has changed over the years).
I didn't dig deeply into the docs, but Dagger appears to be doing a multi stage pipeline locally. If that is the case, I wouldn't want that either. I use Concourse, which has very good visualizations of the stages, and if I used Dagger there, it would consolidate those stages into one box without much feedback from the UI. Also, with Concourse you can use `fly execute` to run tasks against your code on the actual server, without having to push anything to a repo.
Gitlab CI has dockerized runners? Works great!
We have a lot of respect (and common friends!) with the Earthly developers, I am confident we can help each other build even better tools, and grow the buildkit ecosystem in the process.
That means I'm more likely to submit bugs and patches to Dagger, and I won't touch Earthly
Our monetization model follows the same fundamentals as Red Hat before us: open code, strictly enforced trademark rules. "You can modify, use and redistribute the code at will. If you distribute a modified version, please call it something else."
Earthly uses Dockerfile style syntax so I don't have to learn a new language, I can leverage my existing knowledge.
Another advantage is that in Earthly I can run up a docker compose within my pipeline so that I have selenium, envoy and postgres running for integration testing.
You can see my integration tests here https://github.com/purton-tech/cloak/blob/main/Earthfile#L14...
Is that possible in dagger?
>Dagger has a wider scope: build, test, deployment, any part of a CI/CD pipeline really
I don't see any reason this can't go into an Earthfile. We have all of these parts in our Earthfiles.
The one common pain point that both Dagger and Earthly haven't solved for me is unifying the machine parallelization with the DAG parallelization. According to this comment[1], it seems like Dagger doesn't have that goal.
For example, we only run our +deploy target if +build, +test and +lint pass. We parallelize each of those targets across workers in Github Actions. I don't know what the solution is to this problem but I know this was annoying to have to handle with Github Action's workflow syntax and horrible to debug locally.
Working on developer tooling, a lot of times I would hear from people that they wanted CI and dev to be 100% the same and wanted a simple "run all CI locally" command to pre-check before posting.
Unsure how Dagger is handling this but my concerns with the scenario I described
- CI normally divides things up into multiple jobs for speed which breaks the shared "do everything" command
- Commands need to scale down to fast iteration as people fix things
- Generally people get the best integration by using the underlying wrapped tools directly due to pre-commit or IDE integeration
This is exactly what developers should want. It's the most efficient workflow for a dev, because we then don't have to think at all. This is a huge dev efficiency anti-pattern: thinking your code is good to merge, pushing the change, and then finding out 10-20mins later that CI isn't happy for some reason that wasn't natural to check locally.
The thesis in the following is the way: https://gregoryszorc.com/blog/2021/04/07/modern-ci-is-too-co...
Ha, finally. The timeline for a pipeline that I wrote recently looks like this:
1. Write local test/deploy script
2. Promote scripts to hosted CI system
3. Local scripts rot
3. CI system down, need to use local scripts again
4. Re-write scripts to be current
5. Force CI system to use my local scripts> If you would like us to document GitLab next, vote for it here: dagger#1677
If you don’t have an example for a specific tool, just don’t add it to your documentation.
We will look for a way to make this more clear in the documentation.
There's another aspect which affects performance: cache persistence. Buildkit needs a place to persist its cache across runs. By default, it relies on local storage. If that is wiped between runs (common in CI environments), everything will still run, it will just be slow - possibly very slow. Luckily, buildkit supports a variety of cache backends, including a Docker registry. In practice, persisting the cache is the most labor-intensive part of integrating in your CI, even though it's technically not a hard dependency. This is the part that we want to automate away for users, but it requires additional engineering work - hence our asking the community for input on what to prioritize.
From what I understand, everything needs to be a container, and that doesn't work for xcode.
The main issue see so far is that I'd also want bringing up a CI instance of the service the same was as new prod (backup) cluster. All of that is written in Terraform already (and GitHub Actions). Why rewrite it at this point? If I need stats I could probably push them to Prometheus.
So yeah, ha, sounds like just writing something like Dropbox would be easy on Linux. But it could be great if I wouldn't need to drop down to Bash Run all the time, since a lot of official or community extensions would need to be created: https://docs.dagger.io/1202/plan#plan-structure. Somewhat reminds me a bit of Earthly: https://earthly.dev/.
Infra is still largely a separate concern (as in, it doesn't matter what I use, I have options). I suppose I need to look into Dagger more to understand the value-prop.
Call yours Cinquedea, Jambiya, Anelace, or Rondel if you really want a knifey name.
This is a CI/CD system. It's not made to configure a single machine from coded configuration, but to build, test, publish software for multiple targets and then manage deployment to different clusters.
It’s not just for building machines, that is NixOS and its module system, which is a library you can use with the build tool.
It’s possible to build your own Nix-based monorepo internally at your company, too, and still ship containers in production (that are also built by Nix, again, without Docker).
Dagger is not a CI provider, it is a build tool and task runner. Integrating Nix with CI is the same deal as integrating Dagger with CI.
Enter my biased opinion, as someone leading a team through solving the problems this tool purports to solve, but by using Nix: this tool will suffer from the same fatal flaws that Docker does, by being built on its foundation (Buildkit). It is abstracting dependency management at the application build level, whereas Nix solves it at the fundamental system dependency level.
I would like to be proven wrong, so best of luck!
One very important thing for my use case is being able to run steps in parallel distributed across multiple agents. Is it capable of this?
There is a lot of ongoing work in that area in the buildkit ecosystem. For example Netflix has a distributed buildkit farm, which I believe relies on containerd storage backends to share cache data. There are also integrations to run buildkit nodes on a kubernetes cluster.
Dagger itself is obviously quite new, but buildkit itself is very mature, thanks to its integration with 'docker build'.
1. They invest in building out their catalog of actions to compete with GitHub's. I maintain a few GitHub Actions and despite the GitHub catalog's depth, it's still lacking in many ways and GitHub don't appear to invest in it too much: a "maintainer fund" and creative poaching from Dagger could rapidly bring them up to par. A few million of their raise, well deployed, could crush GitHub's catalog.
2. They invest in tight integrations with platforms. GitHub Actions is great because of composability, yes, but also the deep integration with GitHub itself. Being able to run Dagger on GitHub Actions is one thing, but being able to leverage deployment environments cross-platform would be another.
3. GitHub Actions is great, I am a fan of it, I'll speak highly of it often, but the codebase... it is bad. If Dagger can build out a platform that competes with GitHub Actions on functionality, and it has a pleasant codebase, they'll make huge gains from community participation. Contributing to GitHub Actions is painful.
So, I agree with you today, but a year from now, I could see a very different situation and I am optimistic.
The patch I made to act was bigger than I thought the act project would accept, so I just worked around it with some well placed docker volumes and running GH actions "by hand"
Additionally, as someone else here has already mentioned, my mind first went to Dagger, the dependency injection tool (https://dagger.dev). That tool in particular was named as a play on DAG (directed acyclic graphs), whereas in this case I don't think it would apply since there may be instances where you'd want cycles in a pipeline.
On a whim, I clicked on "Trademark Guidelines" (https://dagger.io/trademark) and from that page alone I would recommend avoiding this based on the aggressive language used to try and claim ownership of generic words. According to their own language, it seems I'm violating their guidelines by writing this comment.
> Our Marks consist of the following registered, unregistered and/or pending trademarks, service marks and logos which are subject to change without notice: Dagger; Blocklayer; and other designs, logos or marks which may be referred to in your specific license agreement or otherwise.
> Blocklayer does not permit using any of our Marks ... to identify non-Blocklayer products, services or technology
Which would include Dagger, the dependency injection tool.
Other sections of note:
> Do Not Use As Nouns
(This one just reads amusingly to me, for some reason.)
> Do Not Create Composite Marks
This section seems to suggest that you can't use "dagger" in any shape or form, even as a smaller part of some other word or body of text.
> Websites And Domain Name Uses
>
> ... Any principal or secondary level domain name should not be identical to or virtually identical to any of our Marks.
>
> The website owner should not register any domain name containing our Marks and should not claim any trademark or similar proprietary rights in the domain name. For example, “daggertech.com”, “dagger-group.com”, “Meetdagger.com” are not allowed. Any violation of this policy may result in legal action.
>
> The website should otherwise comply with domain name registry policies and applicable laws regarding trademark infringement and dilution.
This would technically include dagger.dev, which again refers to the dependency injection tool.
---
Full disclaimer that I'm not a lawyer and there could be totally reasonable explanations for these provisions, but they certainly look scary to a layperson such as myself. All in all, the founders seem to be taking a pretty arrogant approach here, but it unfortunately seems to be a common one. I'm choosing not to support it, however.
---
EDIT: formatting
I'm a "devops engineer." Besides pipeline code and maintaining/building out infrastructure, I also write CLI applications to handle complex infrastructure tasks. At what point is a "devops engineer" not a software engineer?
[p.s., I prefer the term "platform engineer," personally]
There's an IJ plugin for cuelang but I'm not at my desk to know if it would be helpful for use with Dagger or not
Point was to break out of the Developers versus Operations silos and actually be more productive as a single team.
Back in the silo'd days... the ops teams would just flat out refuse to release any code on a Friday. Which doesn't work if you're twitter. Hence the switch.
A lot of my time as a "devops" engineer is reiterating that I can fix and automate as much as you want me too, but if your developers don't think about the infra the code eventually runs on in prod then your software is just gonna keep breaking --- "but it worked fine on my local machine".
So, while I agree the "Devops Engineer" name is a bit ridiculous, and not in the spirit of the original methodology, half of my job is writing software that tries to help developers not break production and the other half is attempting to change the culture, even slightly, towards "oh, wait, will this run in production?".
[0] https://developer.android.com/training/dependency-injection/...
GitHub action is a game changer. CICD finally has a proper UI were it belongs: on the repository.
GitHub actions are also much easier than gitlab runner. The basic actions are great and probably solve 90% of all normal use cases.
Then we/I see a big trend of going away from self build pipelines. Provide Standard Features through convention over configuration: the docker image build action checks if it need to run on the pipeline and runs isolated by default.
The build packs are providing something like this but they are to clumsy/complex.
Then you have the real issue no one is solving:
- fast sync between stages and builds. 1 gb of source code, maven cache etc. Still need effort to do it fast. Preferable a filesystem which is fast and allows to share snapshots you can mount ober the network (it's not a big issue but puts a latency limit on how fast you can provide build results and I like it fast)
- standard building blocks with retry mechanism. When a 3h build fails, restarting is annoying like hell. Pipelines have very little resilience.
- unified CICD output: lots of plugins support Jenkins Replacing Jenkins needs a new UI (GitHub has it now) which still is not unified. There is a thing going on to univy on GitHub through GitHub checks.
I think in the next few years GitHub will have solved those issues for us. They do exactly what you are looking for. Would love to work with them.
They even now inject the repository secret for you. They are slow (hello personal access token alternative we are waiting...) But steady.
Btw. Before someone says gitlab: the Auto DevOps thing was shitty and bad supported. A default case with java and postgres was/is not fixed for 3 years.
And those vulnerabilities. The last few years? Nope nope nope.
I feel like it isn't write once run anywhere until I can use the same configuration inside the IDE, from the command-line, and in CI.
The IDE would need to understand what the dependencies are in order to provide things like auto-completion, debugging and syntax-highlighting.
https://github.com/ysoftwareab/yplatform
It uses well-tested software (GNU Make, Bash, Homebrew, Docker) and there's not much to learn really.
* Incubated at Tobii (eyetracking, Sweden),
* it has been used for several projects of different sizes,
* tested on 12+ CIs !
* tested on macOS, Linux and Windows (WSL) !
* tested on 8+ Linux distros !
* comes with prebuilt Docker images !
Happy to help anyone integrate it.
So many days are wasted for me because of the ridiculous back and forth to the buildserver when adding or changing automation. My experience is mostly with Gitlab, and I dare to say it's an absolute business strategy for them to lock you in, into the buildserver. They've had the option to use local build runners, but that one lacks fundamental functionality which results in you having to change the Yaml file locally, temporarily, just to extrapolate templates and such. It's horrible. There are so many tickets asking for feature parity for this local-runner and the actual runner on the build server; it's simply ignored or false promises are made. Afaics, Github actions won't do any better.
I do expect Dagger integrates well with Gitlab such that I can download logs, artifacts and such from the Gitlab UI. But I'm afraid if something like Dagger catches on, the existing CI platforms will gatekeep functionality through pure yaml instead of allowing API calls from your pipeline. That would mean something like dagger will have to rely on compiling to a backend language, probably YAML. However, when did the 'one-ring-to-rule-them-all' approach ever work?
So, basically my question is, what's the incentive for the current CI providers to play nicely? And isn't it trying to follow multiple moving targets?
so does its worker run on a native machine, because we need to build docker images using CI/CD, which we can't do easily within a container?
also, is there a gui?
`dagger` is a client tool, which uses buildkit as its worker. Buildkit itself can run on a native machine, or inside a container. Either way, building docker images is one of the most common actions performed by Dagger, and it is very well supported.
Not at the moment, but we plan on offering one (optionally) as a web service.
That sounds horrible to me. Yet another abstraction layer and set of tools to do something that should be simple. Shell scripts inherently aren't bad. And I really don't see why they'll need more maintenance than using this new tool.
Well, the opposite is very much true as well.
A devops engineer's dream is usually a developers nightmare.
"A developer’s dream is a devops engineer’s nightmare" underines the failure of devops.
Isn't it actually devops engineers' job to make it so?
Then you compose declaratively in cue.
Here is an example of a package to deploy to netlify: https://github.com/dagger/dagger/tree/main/pkg/universe.dagg...
This is what CDK uses for declarative modeling, but gives the opportunity to use languages/tooling that most devs are already familiar with. CDK8s already uses it as a replacement for yaml (technically, the yaml becomes an implementation detail rather than actually replaced)
Not saying we should write one [1] but CI/CD are a commodity these days.
[1] https://aosabook.org/en/500L/a-continuous-integration-system...
- Build code and lint code at the same time - Run slow end-to-end tests in parallel
Is parallelization possible with Dagger, even in principle?
I guess my question is how this integration works in practice and what kind of complexity it generates.
- Concurrency is the breaking down of a program into pieces that can be executed independently.
- Parallelism is the simultaneous execution of multiple things (possibly related, possibly not)
Dagger is designed to be highly concurrent with minimal development effort. Compared to an equivalent configuration in a traditional CI system, your Dagger pipelines will be more concurrent, and require less lines of code.
Because it is highly concurrent, Dagger can be parallelized with relatively little effort. But as you pointed out, you still need to configure parallelization by setting up multiple nodes, etc. Dagger uses buildkit as an excution engine, so parallelizing Dagger boils down to parallelizing buildkit. There is a lot of work in this area (one benefit of building on an existing, mature ecosystem). For example, here is an example of Kubernetes deployment: https://github.com/moby/buildkit/tree/master/examples/kubern...
Note that, because of its highly concurrent design and because of built-in caching, a single node may get you further than you might think. For example, in the e2e testing example: instead of running every test for every new commit, like many CI systems do, Dagger will automatically cache tests with unchanged inputs. So, for example, a change in the documentation will not trigger the API tests; a change in the iOS app source code will not trigger the Android tests; etc.
It's not uncommong for CI runs to become faster after switching to Dagger, without any additional parallelism. As it turns out, most CIs are very wastesful and leave plenty of low-hanging fruits to pick. Dagger helps you pick them :)
Here is a talk that I gave last year: https://youtu.be/BJrxYuqG64Q
“Why saying ‘CI/CD’ is not enough”
Those include, off the top of my head: continuous integration, continuous deployment , configuration management, infrastructure management, PaaS, IT automation.
It seems like that is a DIY dagger. Since most of our precommit hooks are just docker containers themselves. I guess dagger does add parallel builds.
Hopefully this will be the last pipelining language I'll learn.
I regularly go over the main alternatives (TOML, JSON, XML), and they all have serious warts for config IMO.
Better formats exists (such as HOCON) but are much less popular. For a public tool I'd still stick with a YAML subset that everyone knows and can work with a minimum of fuss.
They are more similar in their choice of language than you might think:
* Dagger lets you write actions in any language: Go, Python, Shell, it doesn't matter. Then it runs it in containers, wrapped in a declarative configuration. This allows cross-language interoperability.
* Pulumi also supports existing languages, although it is more deeply integrated, with language-native SDKs (the tradeoff being less supported languages). Eventually they realized that they needed an additional layer to bridge language siloes, otherwise the Pulumi ecosystem is really an archipelago of incompatible components: Pulumi/Go, Pulumi/TS, etc. To solve this Pulumi introduced... a declarative configuration wrapper.
In summary: the best design for this problem requires two layers. One layer for leveraging the existing programming languages with maximum compatibility. And a second to add a declarative configuration model and reduce ecosystem fragmentation. Pulumi and Dagger are both converging on this model, each for their own domain of application.
I personally believe we will see many Dagger + Pulumi integrations in the future :)
1. You can use the same pipelines in development and automation. Dagger runs on Jenkins, but it also runs on the developer's laptop. This solves a common problem which is drift between the automation available in dev and CI.
2. You're not stuck with Jenkins. If you want to migrate a Jenkins-native pipeline to another CI, you have to rewrite everything. Dagger on the other hand can run on any major CI. This makes migrations much easier, as well as supporting teams with heterogeneous CI setups (which is quite common in larger organizations).
3. You can debug and test your pipelines locally. In theory this is possible with some CI systems. But in practice, the experience is very different. You can actually iterate on your Dagger configuration like you would on regular code: make a change, run, try again, run again. It's quite fun and addictive.
EDIT: there is a 4th reason, which is that Dagger uses buildkit under the hood (the hidden backend for "docker build"). So every action in the DAG is automatically cached when possible. This means that your pipeline is tuned for optimal caching out of the box: no manual tweaking necessary.
Thanks :)
https://developer.android.com/training/dependency-injection/...
Now good luck searching for using Dagger CI/CD for Android development.
Also, importantly, it can run on Github Actions itself!
https://github.com/google/dagger
Is it some weird SEO niche-squatting technique? IMHO we see it happen way too often
Are you serious?
Did you think Mailgun was a good name?
edit: Figures Paul Graham is pimping this trash. Good work Sykes, you found a way to float in rich-man-land for another 5 years. I'll never be there but I'm perfectly content living a normal human life. Try it sometime, it may give you some insight into developer tooling and how to profit off of it.