On the flip side, I'd argue that the current centralisation of user agents (in the classical sense here) that benefit from programmatic content negotiation in form of a handful of harnesses like Claude or Codex is a great lever toward forcing the ecosystem to adopt better practices: If Anthropic added content negotiation as described in this thread to Claude, many sites would be incentivised to improve their web servers.