93 karma · joined February 11, 2012
Let me also drop a link to the Pi Emacs mode here for anyone who wants to check it out: https://github.com/dnouri/pi-coding-agent -- or use: M-x package-install pi-coding-agent
We've been building some fun integrations in there like having RET on the output of `read`, `write`, `edit` tool calls open the corresponding file and location at point in an Emacs buffer. Parity with Pi's fantastic session and tree browsing is hopefully landing soon, too. Also: Magit :-)
It looks like there's two pytest plugins at least to support you with that:
- Mutatest: Python mutation testing: https://mutatest.readthedocs.io/
- Mutagen: https://github.com/timpaquatte/pytest-mutagen
Let me know how it goes.
Let me know in the comments if you think it's useful.
From the table of contents:
Our toolchain
py-cov-action: GitHub-native coverage
The two-workflow pattern for fork PRs
pytest-xdist: Parallel testing by default
uv: Fast package management
Six critical gotchas
Gotcha #1: Using coverage run -m pytest with xdist
Gotcha #2: Missing relative_files = true
Gotcha #3: Hidden files excluded from artifacts
Gotcha #4: Fork PRs can't post comments
Gotcha #5: Missing pytest-cov plugin
Gotcha #6: E2E tests with subprocesses contribute 0% coverage
Complete working example
.github/workflows/ci.yml
.github/workflows/post-coverage-comment.yml
pyproject.toml configuration
Verifying your setup
Check relative paths
Verify artifact upload
Test fork PR comments
What success looks like
Migration notes
From Codecov/Coveralls
From pip to uv
Next steps
ResourcesThis second challenge actually featured a different dataset with different hydrophones used etc. But even without retraining (which was rather trivial to do at that point; the hard work of finding the right hyper parameters had already been done), I would have still scored well above 90%. And I think Nick Kridler reported the same.
So overfitting yes, but not too much considering there was a different sensor.