consider also solutions like llms.txt, or variations of it using the same trick mentioned in OP

GET / accept: text/html -> passthrough to existing content

GET / accept: text/markdown -> respond with llms.txt or a stripped down version without html tags or other noisy bits