You need to grind your coffee into bigger pieces. With a good coffee grinder there should be no dust created. The coffee grinders with a rotating knife are just awful, throw it away and get one that works like a pepper grinder.

If you buy already ground coffee you need to tell the shop to make it coarser, tell them you need it for the French press they will know. If you buy the coffee online, you can usually chose the grade of grinding in the online shop.

It’s the opposite for an espresso machine, there you need it extra-fine.

Pre-packaged powder from the supermarket usually tastes like mud, because it’s not freshly grinded.

I mentioned this above, but the Aeropress is a great solution for French press simplicity and taste without the grounds ending up in your cup. It has tiny round paper filters and it's pretty much self-cleaning. After using it (all manual), you just unscrew the filter and eject the coffee grounds.

Someone linked the espro, which has a double filter system for fewer grounds.