Agreed. 'Just' is code for 'I don't understand the nuances of the topic at hand' in my books too, and it was here on HN in particular that I came to realise that.
I now actively try to avoid using that turn of phrase, and when I catch myself about to say/saying it, I check myself and remind myself that I'm also susceptible to this blind spot.