The gypsum stacks are the other waste byproduct. But it's less problematic because it doesn't flow, it just sits. The low level radiation prevents it's incorporation into commodity products such as gypsum wall board.
The byproducts get conflated because for convenience the radioactive gypsum waste from phosphate mining is used to contain the acidic sludge from phosphate mining. But it's the acidic sludge that is the fast acting ecological disaster. The gypsum will give you cancer of course, but as Joe Jackson sings everything gives you cancer. [1]