I mentioned in my other comment that the direnv plugin aims to resolve this. Personally I didn't get it working but I haven't been determined enough. It has been working for others.