Part of the value is in their utility as an anonymous currency: try taxing the guy that's paid in Bitcoins.
Another part is the integrity of the currency: the currency is backed by math, rather than men, and not subject to hyperinflation resulting from corruption (i.e. the US Fed can create money out of thin air, but there is no authority that can create Bitcoins out of thin air: they must be mined).