Why is that an issue for you? It seems like the downstream connection would be on AWS' internal network, so HTTP 2 doesn't have as many advantages.
(Though with HTTP/3 around the corner, I can't imagine anyone will bother working on HTTP/2 to backends.)
Request pipelining and multiplexing are good enough reasons.