I would go as far as to argue that this isn't just your being "bad with" async but rather an inherent drawback of async: namely that it requires way more context switching!
Slack and other instant messaging tools may make it feel like you need to be always on and responsive, but instant messaging is at odds with async.