Nice, I do this via a small script:

  env $(pass env/$1) ${@:2}