I can't speak about 2005/6, but as a CL apologist in 2011:
My dad was doing TDD in common lisp in the 80s. Back then he just called it programming.
A. The major releases support things like unicode, networking, packaging, threading, cbindings. There are also 'standard' libraries for things like regexps, cross implementation threading, stuff like that. The Common lisp asni standard covers a areas that would be part of the libraries for most languages.
(Parsing a floating point number is easy once you have regexps by the way; confirm it is in the right format then do a read-from-string).
I don't know your point about character encoding or file system access? (with-open-file (stream #p"/path/to/directory"... ) ...) seems pretty easy to me. There is a lot of pathname stuff in the spec, that could be mastered, but as far as the basics, you can get by simply by having knowledge of a couple of key functions and the file-system that you are using.
b.) I don't know of any commercial PHP, Python, Perl, or Ruby implementations. I also don't recall professional C development environments having particularly cheap licenses either. (How much for a copy of visual studio? I think the cheap version is $800. How about the Intel C and Fortran compiler suites? They're at least as much as Lisp Works or Allegro).
In terms of free implementations, just pick one. I like SBCL myself, as it is fast, mature and has a lot of low level features (A+++++ great, would code with again).
No one is kept up late at night in a quandry over the choice between cython, jython and unladen swallow! Why should it be so for CL? And languages like python and ruby change in major ways every few years! Do i want ruby 1.8 or ruby 1.9.2 or python 2.6 or python 3.0? OH GOD!
And unfortunately the languages that near CL's flexibility are invariably slow.
c.) How does the size of the community really effect what I'm going to make in the language? (Here's a link to a web client btw http://weitz.de/drakma/ Weitzware rocks). There's a plethora of libraries, and now with quicklisp, someone is actually curating them so you have a good idea of which libraries will work pretty well.
After the initial learning curve, making things in common lisp really is easier than making things in a lot of other languages.
The problem is that there is a fairly large initial learning curve. even in 2011 it is very different from a lot of languages that people are already comfortable with.
The standard covers a huge range of ways of thinking about writing computer programs. (You can feasibly combine ideas from functional, imperative, OO, and DSL based programming, all within a few lines of code.
You can even do C 'bit-bashing' types of programming, if you really want to (and it can be pretty darn fast).
I guess my basic thesis is that Common Lisp doesn't suck.
Learning it (completely) sucks, because it involves learning a large number of the disparate things are done with a computer, which is a daunting task.