I'm with OP in their diagnosis of the problem. Other commenters seem to focus on "tricks". The problem with these tricks is that they can all be bypassed and you always go back to normal. Would you suggest someone who wants to quit smoking just keep cigarettes in their pocket and just snap a rubber and on their wrist when they want one? Of course not, that is just asking to relapse. Self-control is a finite resource. Eventually, you stop using the rubber band and just continue smoking as before.
I've long thought that what we need is a forked Android distribution without a web browser and with a whitelist of apps that can be installed. Something of an "addiction-free phone". I'd be curious to hear if anyone is working on anything like this. It's certainly something I would want. To be honest, I'd pay a lot for it.