People are using linux on clusters of PS3s for real work, so it is obviously practical in some scenarios.

One of the SPE's is reserved for the hypervisor, but this is true for games too. There's plenty of power left (the main CPU and 6 SPE's).

AFAIK booting into OtherOS renders the GPU inaccessible. The cell processor (PowerPC core and the SPEs) is still fully usable.