Do they test for things like XSS or just to see if you have GD installed on your server?
Is it terrible that I am not creating unit tests for in-house php based web apps?
Let's say I write a piece of code to calculate a dollar value for an investment portfolio. Then, I want to validate it. I could poke at it and see how it behaves on some examples, but this would be a one-time thing. Better for me to write some tests which set up some sample data and assert that the code calculates the right results.
Later on down the line, maybe I find a bug -- maybe it doesn't work right when you purchased a stock, then purchased some more of it. I could fix it, manually test it, and call it a day, but it would be better if I first expressed the bug as a unit test (i.e. I set up some data which causes the code to generate a result different from what I expected, and so the test fails), and then fix the code. Since I also have my original test cases, I can be sure that I didn't break some other behavior when fixing this bug.
Similar reasoning applies to adding functionality -- leave the old tests intact and add new tests.
Tests are not about configuration. It may use the word "test" but "test whether XYZ is installed" is a different question from what unit tests are about.
Tools which look for problems like XSS are not tests either. I don't know what to call them. They are a good idea.
Selenium is a testing tool for validating web apps at the application level. Purists might call it a tool for functional or integration testing rather than unit testing.
A developer leaves, they've left behind tests that demonstrate that the code fulfills the requirements and perhaps shows how the code interacts with other areas of code.
A developer arrives, they can see the requirements and how an area of the codebase works by examining unit tests.
Of course, this presumes well-written tests that are actually thorough.
Unit tests also give you a measure of confidence that allows a freer hand when refactoring code. You can change code all you want and if the unit tests still pass, you haven't broken things (again - if the tests do cover all the expectations).
Some level of sanity is required around test-first, test once the code is written, how many tests you need, what level of depth, etc. A one-off app to fulfill a quick-and-dirty need doesn't need as much testing as a core platform that will be running business critical code for years.
"a unit test is any test that you'd be 99% happy to have if it ran continuously" (http://twitter.com/patmaddox/status/23988815286)
Followed up with:
"99% happy implies: fast; no false positives; flexible under code changes; catch errors as they occur" (http://twitter.com/patmaddox/status/23988868590)
In my experience, I can tell that I'd really like to be able to use them, but changing my apps to fit them would make them a lot "thicker".
Let me explain. When I make a page in a web apps which does something, the code is very thing. It has roughly 3 layers: sql database (static, no stored procedures), business logic including sql statements (java), and html template (in my case html+apache velocity). There is no POO dividing my apps horizontally.
In order to make this testable I'd have to basically put another layer on top of everything. I still have no idea how to do it without making everything horrible.
1. That the code you have written or are using behaves in a way you expect, for the inputs you expect.
2. As the code changes over time, a regression test suite helps ensure that assumptions from an earlier point in time still hold true.
If you practice TDD (Test Driven Development), you first write the test(s), then code to make those tests pass.
You're much less likely to end up with SomeClass.some_reference.a_really_bad_name(with,lots,of,required,arguments) when you first think of the caller, rather than of the method itself.
Once you have good coverage of your code base you can make large changes, run the tests and be confident the system still works without actually starting it up and checking manually.