"It’s so expensive to dispose of waste legally that many people attempt to do it illegally."
This is one of those things where we just have to collectively bite the bullet and make it easy and cheap to dispose of things properly. Otherwise you get people dumping toxic waste in storm drains, throwing old tires and appliances in ravines, mixing dangerous contaminants into regular trash, or just putting their trash in other people's bins.
Where I am it costs $20 to properly dispose of an old can of paint. It costs about the same to dispose of any electronic item that has a screen. Even regular household garbage has to be put in special bags that cost about $2/each. All of this is a counterincentive to people disposing of their waste properly. And every few years the local waste manangement bureaucracy raises the rates.
Just build it into the base taxes that everyone pays. Make it "free" and easy for people to do the right thing.