Sounds like you're after the select() filter, which evaluates a boolean expression and passes the input to the output only if the expression is true.
So your example would be something like: select(.request.headers|map(.[0] == "X-My-Header" and .[1] == "my target value")|any)|<pick out some keys to display>
I realise though that your point wasn't the specific example, it's that the docs for these kind of cases are poor, and you aren't wrong.
I guess my main advice is to not think of jq as something like cut or sort - instead, it's more like awk or sed. You can do lots of crazy things with sed, but it's not immediately obvious how just from reading the man page. These kinds of tools require a little more time investment but are very powerful.
(edit for code typo)