I'm working with some researches on about ~40k lines of Python disease modelling code and these are the things that helped us becomes more productive:
- tests (unit, smoke, acceptance) (eg. https://mattsegal.dev/alternate-test-styles.html, https://understandlegacycode.com/approval-tests/)
- continuous integration (via GitHub actions eg: https://mattsegal.dev/pytest-on-github-actions.html)
- 10x performance improvements via cProfile (https://julien.danjou.info/guide-to-python-profiling-cprofil...)
- getting fast feedback on code changes by visualizing outputs with Streamlit (https://www.streamlit.io/)
- input parameter validation (eg. https://pydantic-docs.helpmanual.io/, https://mattsegal.dev/cerberus-config-validation.html)
- total automation of data processing job (eg. https://www.prefect.io/ - we don't use this but that's the kind of thing we have)