It has to do with swallowing something that can electrolyze anything wet (like the surface of your insides) into nasty chemicals that will eat holes in important things.
To top it off, swallowing presents flu-like symptoms until it's too late.
The lithium batteries are thrown away with a fair bit of energy remaining: Internal resistance will go up too far to be useful to power something, but the cell can still electrolize when inside the body.
Zinc-air primaries on the other hand tend to be completely empty when removed from the hearing aid; they have a really nice flat discharge profile.