330 karma · joined September 12, 2013
OK, but following this logic, shouldn't we disband the government altogether and have mafias rule us all?
When is it acceptable to hope that something is actually going to work well enough when handed over to the government?
If you are not smart or have no tests, you will not be happy.
If you are smart or have high test coverage, you may or may not be happy.
This situation reminds me a bit of ergonomic handles design. Designed for a few people, preferred by everyone.
https://devblogs.microsoft.com/dotnet/introducing-dotnet-sma...
In general, I see the idea of semantic matching instead of textual matching as one of the great, pragmatic applications of the current technology.
Somewhat related fun application of this concept is this: https://neal.fun/infinite-craft/ (the combination outputs are generated by LLMs)
Describe your data up front and generate schemas, API specifications, client / server code, docs, and more.
With TypeSpec, remove the handwritten files that slow you down, and generate standards-compliant API schemas in seconds.
Microsoft IntelliTest (formerly Pex) [1] is internally using Z3 constraint solver that traces program data and control flow graph well enough to be able to generate desired values to reach given statement of code. It can even run the program to figure out runtime values. Hence the technique is called Dynamic Symbolic Execution [3]. We have technology for this, just not yet applied correctly.
I would also like to be able to point at any function in my IDE and ask it:
- "Can you show me the usual runtime input/output pairs for this function?"
- "Can you show me the preconditions and postconditions this function obeys?"
There is plenty of research prototypes doing it (Whyline [4], Daikon [5], ...) but sadly, not a tool usable for the daily grind.
[1] https://learn.microsoft.com/en-us/visualstudio/test/intellit...
[2] https://link.springer.com/chapter/10.1007/978-3-642-38916-0_...
[3] https://queue.acm.org/detail.cfm?id=2094081
And what if this is done repeatedly for the junior engineer, and yet any initiative they show after that is still negligible?
You can also see a connection to a version control system like Git. Instead of keeping snapshots of all the contents of the repository after each commit, one can keep only the initial repository state and changes in each commit. Then to get to N-th state you say "Apply first N commits to the initial state".
In the bouncing DVD logo example the "function to compute next state" or "commit contents" is just easy and regular, to the point of being expressible via simple math functions.
This is also known as Event Sourcing.
> The 542 drone strikes that Obama authorized killed an estimated 3,797 people, including 324 civilians.
How about the AI never writing any code, just training "mini AI" / network that implements the test cases, of course in a generalized way, the way our current AI systems work. We could continue adding test cases for corner cases until the "mini AI" is so good that we no longer can come up with a test case that trips it over.
In such future, the skill of being comprehensive tester would be everything, and the only code written by humans would be the test cases.
Low-background steel, also known as pre-war steel, is any steel produced prior to the detonation of the first nuclear bombs in the 1940s and 1950s. Typically sourced from shipwrecks and other steel artifacts of this era, it is often used for modern particle detectors because more modern steel is contaminated with traces of nuclear fallout.
6:08 here: https://youtu.be/0kahih8RT1k
You put the logic that increments that variable into a pure function and unit test the input/output pair. Because it is now decoupled from the database, you don't have to deal with it.
> mocks allow you to isolate the unit to be tested.
You achieve isolation instead by refactoring to a "pure functional core and mutable outer-shell" architecture. Above I gave one example of refactoring to functional core. In the cases where you still need to deal with external dependencies in unit tests, you use in-memory implementations (aka simulators) instead of mocks.
Basically there are two subtypes of unit tests - small focused unit tests, testing input/output pairs of purely functional code, and "bigger" unit tests, that check how bigger units of internal business logic collaborate together. They use the in-memory simulators to ensure the tests maintain all the properties of a good unit test: runs fast, doesn't interfere with other tests, requires zero setup, is not flaky and makes zero assumptions about the environment.
So, overall - no mocks for unit testing.
> mocks are for unit tests only.
I do think the opposite is true, with very few exceptions. Mocking in unit testing is an anti-pattern leading to brittle tests with negative value - the cost of maintaining them is way higher than any benefit they provide. Too many false positives, too little true positives, too much rework needed when code changed but no bug was introduced, too unreadable code.
> integration tests is when you start integrating dependencies together. so maybe you have a dummy database like an in-memory db, but you're certainly not mocking it.
This is the only case mocking makes sense. In integration testing you want to test integration with one external endpoint. Hence you mock or simulate all the others. In unit testing there is no need for mocking as explained above. In system (end-to-end) testing there is no need for mocking because we test how everything integrates together.
Naturally, there is way more nuance to the simplified statements I made above. The book elaborates on that. There is also an article by the same author about it: https://enterprisecraftsmanship.com/posts/when-to-mock/