Hacker News Reader

Comment by stephenturban | Hacker News Reader

Is CI even worth it anymore? To be honest, I prefer testing on my local machine. This seem unecessary.

The CI shows the entire team that the tests are passing, and doesn't require other devs to pull it and run the tests themselves. It prevents devs from accidentally forgetting to run tests, or thinking "This is just a typo fix, it shouldnt break any tests". Tests might also run on multiple platforms/os's/architectures/versions, etc.

The CD ensures that the code is only deployed after all tests passed, it was approved, merged, and ensures a deploy isn't skipped accidentally. Depending on what your doing it could take a while to fully build new binaries, build for multiple platforms/versions/architectures, etc.

Ah yes, the "works on my machine" testing technique

Testing on your local machine won't catch portability issues (Linux vs macOS vs Windows, x64 vs Arm64).

Unless you take great care in writing hermetic tests, testing on your local machine won't catch problems hidden by non-standard versions of libraries and tools in your local development environment (e.g. you installed modern versions of bash and python on your Mac, but your users are using what the OS comes with).

And for large enough projects, running tests on a single machine is simply impractical: a full test suite run on a single machine would take hours; the only solution is to shard.

You can set up arbitrary environments locally too.

Yeah! And set up scripts so that you don't need to remember how to do all the setup/building.

And you could set up a trigger to build it when you make a commit so that you don't need to manually trigger every machine separately.

And then you don't need CI!

You can forget to commit a file and it will work on your machine but break everything for your coworkers locally and in production.

I see this class of issue often in early startups that don’t have robust CI implemented yet. Like literally once a month or more per company.

Maybe not for a small 1-person project. If you have a team project you definitely need automated tests to run regularly, and everyone needs to be able to see when/where they fail.

Seems weird to put these in conflict. I do both. There's some overlap, but they seem to serve different purposes.

Do you always remember to run every test? Does every other dev? Are there 100 other devs? If you check in bad code will it break something for 100 other devs? It all depends on the size of the org.

I'm waiting for a CI build as I write this - If I ran it locally it would take 3 hours, but the CI system can split across multiple AWS nodes and so be done in 25 minutes (the time of the longest build). I tried getting a more powerful machine, turns out my build becomes memory bandwidth bound at about 48 cores (building on a RAM disk makes the build take longer which is how I know it is memory bandwidth).

In my case, it would take about 45 hours to run on a single machine. Our distributed testing infrastructure gets it done in about 2. But there's a limited amount of machines, so you usually have to add 2 more hours queuing to that.

The whole point is to continuously integrate your work with that of others. If you are a solo operation then sure, no diff. On a team? Different story.

Yes it is very important, since not everyone has the discipline to run tests on their own machine, leading then to errors upstream.

Then CI cluster has 100x the core count of my laptop, so yes.

Some places it isn't possible to run the test suite on your local machine[1] (access to resources, sheer CPU power, etc.)

[1] Unit tests, maybe, but not integration.

Depends what you are making, how many people depend on your code working and how many team members you have

Reply on news.ycombinator.com