Hacker News Reader

Comment by jdfellow | Hacker News Reader

I've been wishing for one of my small projects (3 developers) for some kind of "proof of tests" tool that would allow a developer to run tests locally, and add some sort of token to the commit message assuring that they pass. I could honestly do without a ton of the remote-execution-as-a-service in my current GitLab CI setup, and be happy to run my deployment automation scripts on my own machine if I could have some assurance of code quality from the team without CI linters and tests running in the cloud (and failing for reasons unrelated to the tests themselves half of the time).

View on HN

You are right. Small teams absolutely do not need to execute code remotely, especially if the cost is having an always on job server.

My team writes test output to our knowledge base:

    bugout trap --title "$REPO_NAME tests: $(date -u +%Y%m%d-%H%M)" --tags $REPO_NAME,test,zomglings,$(git rev-parse HEAD) -- ./test.sh

This runs test.sh and reports stdout and stderr to our team knowledge base with tags that we can use to find information later on.

For example, to find all failed tests for a given repo, we would perform a search query that looked like this: "#<repo> #test !#exit:0".

The knowledge base (and the link to the knowledge base entry) serve as proof of tests.

We also use this to keep track of production database migrations.

> You are right. Small teams absolutely do not need to execute code remotely, especially if the cost is having an always on job server.

Super debatable. You should have some builds and tests run in a clean environment. Maybe you could do it in a Docker container. But otherwise, you want a remote server.

Devs mess up their environment too much for a regular dev machine to be a reliable build machine.

Git pre-commit hooks can run your tests, but that's easy to skip.

I don't know about a "proof of test" token. Checking such a token would presumably require some computation involving the repo contents; but we already have such a thing, it's called 'running the test suite'. A token could contain information about branches taken, seeds for any random number generators, etc. but we usually want test suites to be deterministic (hence not requiring any token). We could use such a token in property-based tests, as a seed for the random number generator; but it would be easier to just use one fixed seed (or, we could use the parent's commit ID).

Reply on news.ycombinator.com