My kids love playing with coins and before they turn 4 or so, love sticking them in their mouth. You have to be careful not to leave them around. We got a gumball machine that dispenses M&Ms, so at least now they think of coins as “something I should spend” instead of “something I should chew on”.
Coins aren’t that toxic… most of the time. An even worse threat is button and coin cell batteries. Those should really leave any parent worried. When disposing of them, they need to be wrapped up in something big enough they can’t be swallowed and then taped up so the coin can’t get out. Lately I’ve been disposing of them in a sharps container I have for getting rid of Stanley knife razor blades, another decent choice.