vladaionescu

I would add that 1. Earthly is meant for full CI/CD use-cases, not just for image building. We've forked buildkit to make that possible. And 2. remote caching is pretty slow overall because of the limited amount of data you can push/pull before it becomes performance-prohibitive. We have a comparison in our docs between remote runner (e.g. Earthly Satellites) vs remote cache [1].

[1]: https://docs.earthly.dev/docs/caching#sharing-cache

Nope. Only Earthly CI is.

Founder of Earthly here - besides the build debugging difficulty, I would add that modern CI/CD repeats a lot of steps: downloading, installing and configuring dependencies, making things much slower than they should be.

We built Earthly [1] to tackle these two problems specifically. We're open-source (10k stars).

[1]: https://earthly.dev

This is the main reason we built Earthly: run your builds locally, and get consistency with the CI.

Hello HN,

Vlad, creator of Earthly here! Earthly is an open-source build automation tool. We use containers to make builds consistent across any environment, thus eliminating difficult to reproduce CI failures, because you can run your CI build on your laptop every time.

Second, isolation means that dependencies between steps need to be explicitly declared, and with this knowledge, earthly can infer parallelism.

Third, containerization gives caching abilities. We can tell if nothing’s changed since a previous run, and earthly can cache steps across build runs or even across machines. This results in massive speed gains. 2-20X, depending on the setup.

We trended with Earthly on HN a few times before, but we made significant progress since we last had a Show HN entry, 2 years ago [1]. The biggest part is that we recently launched Earthly Cloud, which includes Earthly Satellites, super fast remote runners that work with any CI. Earthly Cloud includes a generous free tier if you'd like to give it a try.

[1] https://news.ycombinator.com/item?id=27785323

This article is slightly click-baity. The payments are delayed in some cases, not held out altogether. From what I'm hearing so far, Rippling has changed banks over night and has instructed all clients to update the ACH filters accordingly. Only payroll cycles caught in-between will be affected, but even then they are delayed, but not held.

Hey HN, author here. We made the switch to open-source to better align our incentives with the community. It was a tough decision in the face of tech-giant monopoly bullying, but I think the pros outweigh the cons in this case. Looking for any kind of community feedback on this. Many thanks!

Public / Private is interesting.

Private would not be callable from command line and not be listed in `earthly ls`?

The `earthly ls` idea is great. We do have shell autocompletions but that is not quite the same thing. I will add a ticket for that.

Cache misses can be a bit inscrutable. It could be the buildkit GC is running, because disk space is getting scarce, or that some arg or file change caused the cache to be considered invalid.

Caching is an area we will continue to improve. We have a proposal for extended cache mounts here[1].

Thanks for using earthly!

[1] https://github.com/earthly/earthly/issues/1399

Hello HN. Earthly is a tool for build automation and we just had our biggest release yet.

Earthly is a modern build solution that is similar to a sandboxed makefile. It works with your existing CI and the #1 reason people use it today is to able to reproduce a CI build locally.

Today we promoted a number of important features to GA status in our 0.6 release including:

    - WITH DOCKER : Earthly can execute any number of containers in parallel, via an isolated Docker daemon that it starts up within.
    - User-Defined Commands: Extract common repetitive build steps into a command you can use across your projects.
    - Shared Cache:  Earthly v0.6 now provides shared caching. This  extends our existing build caching to work in ephemeral CI runners by storing the cache in regular container registries.

We started working on this project in 2020[1] on GitHub [2] and while time has gone quickly the number of people and projects using Earthly now is truly exciting.

Let me know what you think! Feature requests always welcome :)

[1] https://news.ycombinator.com/item?id=22890612

[2] https://github.com/earthly/earthly

Haven't done this, but our legal team (WSGR) said that it's possible, if needed to add an angel or two later on. IIUC, most investors will want a price round if investing more that $2-3M though.

Press F to pay respects to IT admins, ops, SREs and systems engineers who have to deal with this on such short notice.

Hi mdaniel - just to clarify the blog post says that Confluent / Kafka follows an open core model. It doesn't claim that Kafka has a source available license. A few notable switches to source available licenses have been named in another part of the blog post though. Hope that makes it clearer.

Earthly team member here. Yes, we're working on native windows support as we speak. This has been highly requested.

The vision of this article is a similar vision to earthly.dev, a company I have founded to pursue the issues presented here.

We have built the build system part and are working on completing the vision with the CI part.

We use buildkit underneath as a constraint solver and our workflows are heavily DAG based.

Hundreds of CI pipelines run Earthly today.

I don't fully agree with all the assumptions in the article, including with the fact that the TAM is limited here. CI has been growing at 19% CAGR and also I think there are possibilities for expanding into other areas once you are a platform.

We've used Timescale at ShiftLeft since before 1.0. Recommend 100%

Hi there, we have developed Earthly (https://github.com/earthly/earthly) exactly for this use case. We hear about this use case quite often, but is a bit cumbersome with Makefile+Dockerfile.

If you also want to migrate off of Makefile and also want reproductibile builds, try out Earthly. Normal companies can't do Bazel because it's too alien and requires deep investment.

https://github.com/earthly/earthly

Disclaimer: I am Earthly's creator.

I write about lessons learned the hard way while building companies and about engineering management.

https://vladaionescu.com

Pretty sure that the only reason they copied that code was that they didn't know how to do it themselves.

Thanks man. To answer your question about load balancing, it is made via the 'leveroshost' agent itself. Each contains a HTTP/2 proxy that routes and loadbalances traffic as needed. Each RPC goes through 2 of these agents: one on the node originating from and one on the node going to. The decision of where to send the request next (load balanced / routed) is based on information from Consul.

Hey, at least I didn't use "cloud-native" or "12-factor app"! On a serious note, though, yeah I can't say I'm keen on the terms. I've known DevOps since it was called SRE 10 years ago, at Google. Just trying to use same terms other people on HN seem to use.

Absolutely, Python and Java are next. It's based on gRPC so adding new languages is relatively easy.

I wish someone made a Spotify playlist for this.

Very interesting. Is your project open source by any chance?

Security needs to be air-tight for sure, but we think we address it in the doc. Let me know if you have something specific that you think could be leveraged.

We talked with quite a few developers. There are many who want to launch their startup but have waaay less technical skills than you might think.

The way we thought of bridging the gap with mobile is by using Ionic and/or Apache Cordova, which use JavaScript.

Hey, thanks for the frank comment. Those are fair concerns and there will be a lot to address, for sure. The reason why we posted this so early (without any code) is that we wanted to get some kind of participation from the community from day zero. Your comment is the kind of participation we were looking for.

Now to actually address it, I think it's more than just a thin wrapper. Perhaps the example shows too little, but for many developers, writing server side code and all the necessary communication is a road block. For others it's just extra stuff to maintain. And we generally like less code that does more.

I wouldn't worry about V8 vulnerabilities, there are many node servers running in production today, in general, and they are doing fine. But your DDos point is pretty fair - I can see how that could be a problem. Feel free to submit a pull request to add to caveats if you want to take credit for it.

No, we're just starting out, pivoting from a previous similar idea. We wanted to include community from day zero, so anyone can contribute to the design as well.

IMHO, I feel like people who are not up to it don't agree with coding interviews. Also, in my experience, the employers that didn't check coding skills at the interview had the worst engineers I met (surprise!).

Now I can see why it's easy to disagree with interviews that check your algorithms 101 knowledge: that stuff is really really rarely used in real-life and you can just google it if you ever need it. But! Keep these in mind:

- Can you come up with a better process that scales with the number of interviewers in your company, but also maintains reasonable consistency and keeps reasonable costs? Maybe you think you can, but keep in mind that the tech giants have data-crunched their interview stats over and over and this process is what they stuck with. (Of course, with scale there's also the problem you occasionally have arrogant interviewers - but I think that problem should be decoupled from the coding/non-coding interviews problem).

- In places where they look for A* engineers, the point of the interview is often not to test what you know best, but how you get along with problems you have never seen before. An algorithm or data structure question often fits the bill.

- Geeks love geeky puzzles (like inverting binary trees). Companies often look for geeks in love with abstract stuff.

Also, IMHO, knowing only one language is a red flag for me too at 10+ years experience level.

From what I could find online, there are no plans to include websockets as part of HTTP/2 (according to this: https://webtide.com/http2-last-call/ ). HTTP/2 is meant to supersede all websocket use cases.