We (buildpacks team) get some of the way by ensuring that all secrets in our logs are redacted -- we actually wrote a rough-and-ready tool (concourse-filter[0]) for this purpose. It works on a whitelist principle. Any environment variable emitted to stdout or stderr is redacted unless it appears on a whitelist[1].

You're right that in the longer run, providing per-test keys will be the safest option. It's on our radar as part of the overall "3 Rs" effort[2].

[0] https://github.com/pivotal-cf-experimental/concourse-filter

[1] https://github.com/cloudfoundry/buildpacks-ci/blob/1c345c30e...

[2] https://medium.com/built-to-adapt/the-three-r-s-of-enterpris...