Every time you code (whether requirements are clear or not), you go through the following steps (often executed in the space of 2-3 minutes):
1) Think 2) Write code 3) Build/run code 4) Test and see the outcome 5) Go to step 1
Even if you are not writing tests, you are doing step 3 and 4. Moreover, if you are not writing tests, you are doing steps 3 and 4 manually.
Manual is only quicker if there are a few of these iterations. If you are doing this iteration every 2-3 minutes, you will be doing it hundreds of times a day and thousands of times in total over the course of a project.
Automating that process will save you a lot of time in the long run, and will still save you time in the short run, as well as leading to higher quality code.
If I'm writing a library to add two numbers, it's easy to create fast tests with no false alarms. Assert.equals(10, new BigDecimal(5).add(new BigDecimal(5)) - simple, fast, no false alarms, easy-to-maintain.
Now imagine you're making a website and you want to test with multiple browsers, and it uses caching and connection pools and sessions and whatnot, and the website has several dozen pages, and there are a bunch of configuration options and feature flags and you need to test with all of them.
Now all of a sudden you're administrating a pool of VMs with different OSes, writing web-scrapers, launching web servers every test cycle, waiting for tests of pages you haven't changed, running n tests * m pages * o server configurations * p clients, and all of a sudden you're spending more time keeping your tests working, or changing your software so tests run faster, and you're hardly spending any time working on features users want.
Naturally, to someone writing the first type of software, unit testing seems perfectly logical. But I can also see how someone working on the second type of software might get frustrated!
Part of the point of unit testing is that you keep the 'unit' under test to be as small as possible so you don't have these sorts of problems and can run tests as frequently as is useful. It doesn't remove the need for other types of testing suites, but they can be isolated and run at separate times.
You're right - it eats a lot of time and it shouldn't. It's not as easy as unit testing and it should be.
I actually wrote an open source testing framework to handle this type of thing precisely because I was getting so frustrated with having to rebuild the boilerplate code to do this everywhere I went.
This is a simple example test I made with it - it uses a browser (w. selenium), libfaketime, and a mock SMTP server to test a django/celery/postgres/redis app:
https://github.com/crdoconnor/django-remindme/blob/tests/tes...
And this is the harness - it runs 5 services and a browser in 50 lines of code:
https://github.com/crdoconnor/django-remindme/blob/tests/tes...
The framework is currently in pre-alpha now, but it should be a bit more stable in a few weeks. It should be usable to test virtually any kind of software that runs on UNIX, also (not just python apps).
Easily handling the "m pages * o server configurations * p clients" problem is on the roadmap. Different browsers and server configurations will be treated as "just another kind of fixture", and it will be easy to feed fixture groups so that you can run the same automated test under multiple scenarios.
Consider the problem of intersecting two arbitrary 3D surfaces to get a curve. There are an infinite number of possible inputs. Except for rare simple cases, for each pair of inputs there are an infinite number of valid approximate solution curves AND an infinite number of ways to express each of those curves. Also, you want solutions which are well-behaved -- no kinks, no singularities, etc -- and as simple as reasonably possible (but no simpler).
And even that's still better than something like MP3 encoding or digital musical instrument UI testing, where the quality of the result is arguably subjective...