Yeah, relying too much on modified headers from requests to send the appropriate response isn't ideal, because request headers can be incorrect and largely unreliable.
Responding with a Cache-Control header with a max-age seems like a much more superior option for these cases.