I'll prefix my comment by saying I'm someone who is more on the sysadmin side of the scale of things. My code isn't pretty, but it generally works well enough to get the job done. I'm learning (probably the most important thing, I guess).
Contexts are one of things that I felt the same about. Until I tried really using them, rather than stressing over whether or not I was using them properly.
I made the mistake of trying to think of them as microservices, which bogged me down and for my purposes was complete overkill. I've now reached a kind of happy medium, which I've realised that I really should have already been using. I've kinda realised I suck at api design. And for that I'm very thankful to the phoenix team, because it feels like I'm learning.