> A dumb phone - small and minimal functionality, easy to forget about

This requires more diligence, but can't you create the experience you want on a smart phone (no social media, disable all notifications, etc.) without giving up utilities that provide actual value, such as GPS / navigation?

> Exercise - no headphones, a gym provides background music and enough human contact to keep boredom at bay

Interesting. I feel like I am more distracted and unfocused when I listen to the gym's music (sometimes their music just doesn't match the intensity of the workout) or when I socialize between workouts.