Hacker News Reader

Comment by vintagedave | Hacker News Reader

I've had to write some detailed prompts and examples to have AI generate the kind of unit tests I want in Python. I've seen the assertions on types alone too. I want assertions on values and more.

Even more than that, AI tends to mock _everything_. Mocking is useful, but the more real code a unit test invokes, the better, because the risk is not only the code itself but its interactions, the interface. Yet AI in Python will mock so heavily it barely tests even the code itself, with tautological statements.

I've prompted with heavy warnings against mocking and pointing directly at examples of thorough tests as examples. FWIW, Python does have excellent tools for injection and can write really nicely structured code.

View on HN

I wish I had 100 upvotes to give you. Weak, heavily mocked tests are my biggest pet peave. Test “quality” is important and not something a lot of devs pay attention to.

I’ve found myself preferring integration tests or unit tests with a “real” database set up because the tests are much more effective. If you design them right, they don’t even need to be slower.

They will be locally if you have to also run 3 virus scanners :)

When asked to write UI tests (playwright), I've seen Claude Code do essentially the following:

const elem = document.querySelector(".foo"); // actual element that exists elem.innerHTML = '<div class="bar"></div>'; const child = elem.locator(".bar"); // child we actually want to test for expect(child).toExist()

Gee thanks Claude, what a great test...

Same. Drives me up the wall. I’m writing my own coding agent now and I’m baking into it prompts against all the anti patterns I’ve see.

Mocked tests also make refactoring a pain in the ass.

This is why I heavily prefer integration tests

I'm curious how you structure your Python to be well testable. I have to admit, my own use of Python has been limited to scripts and (a long time ago) a game engine, not large codebases. So unit testing for those hardly came up.

It seems there's a couple of dependency injection frameworks but they're clones of what's found in Java, right down to the type names. One of them even calls injectable objects beans! (Rhazes)

Same as you do it in any language: you compose instead of inheriting, you avoid shared state, you generally think about how this thing you're implementing can be tested even as you are implementing it. Test-driven development tends to constrain your interfaces too early but you can get a lot of the same benefits with, let's call it, test-mindful development. That works in any language.

Most of my Python is web. Individual components, same as always - approach with a set API and not too many dependencies, and allow injection via some route if so. I also test web endpoints. One thing I really like is isolating tests that require data -- rather than mocking the database, for example, I'll create an in-memory SQLite DB used while running tests. That way I can test the full stack: a web API, see its results, and check what was changed in the database at the same time, all isolated from the 'real' stack.

I learned to write well testable code when I learned go. It pushes you to pass interfaces instead of direct implementations. There's also no inheritance, just composition. While there's no 1 to 1 translation to Python the concepts are still useful. It can be easier in Python thanks to duck typing.

Reply on news.ycombinator.com