Ah, salt Peter has potassium chloride AND sulfur.

Yep, that was it. Or maybe just salt on your peter.

Saltpeter is potassium nitrate. It's a diuretic, not sure why it would have any particular effect on libido.

This is specifically about the military, but searching for "saltpeter prisons" gets similar results:

http://www.snopes.com/military/saltpeter.asp