Hacker News Reader

Comment by 9dev | Hacker News Reader

Conceptually, I’m not sure I agree. There’s elegance in clients saying "I want this resource, and I’d like to get your markdown version of it. If you don’t have one, I’ll also take HTML." And if you couple that with optional "file extensions" at the end of the url to force a specific format (say, /foo for automatic negotiation, and /foo.html, /foo.json, or /foo.md for the respective media type,) you have a very easy to use API that adapts to the client; not the other way around.

I take the point that it makes caching harder, but I don’t think that should overrule ergonomics concerns.

View on HN

In general I think I just don't like the idea of one URL being able to return different content. Forces me to think about what each system I give that URL to may be sending in content negotiation headers. Would rather the HTML is returned and alternatives listed in HTML head.

But for HTML and Markdown in particular, there's been so much useful work done in the semantic HTML space and microformats, that I don't know why anyone interested in this wouldn't just improve their HTML markup and leave it to the agent to do the rest. Convert to markdown or extract the useful HTML before handing it to model.

> I just don't like the idea of one URL being able to return different content.

It's different content representations. A text in a markdown file is conceptually the same content as the same text in HTML (or PDF).

It also means you can’t easily know the set of formats that the server could respond with, which in turns makes e.g. archival more difficult.

That entirely depends on the server. A good solution would be to include Link headers in all responses:

  Link: /some-page      rel="canonical"
  Link: /some-page.json rel="alternate" type="application/json"
  Link: /some-page.html rel="alternate" type="text/html"
  Link: /some-page.md   rel="alternate" type="text/markdown"

That only works if the client looks at it. The current Claude fetch system does not.

Should we let vibe-coded agent harnesses dictate protocol design now..?

On the flip side, I'd argue that the current centralisation of user agents (in the classical sense here) that benefit from programmatic content negotiation in form of a handful of harnesses like Claude or Codex is a great lever toward forcing the ecosystem to adopt better practices: If Anthropic added content negotiation as described in this thread to Claude, many sites would be incentivised to improve their web servers.

For the LLM use, the challenge is that it will only discover those after first requesting and parsing the HTML version.

Maybe it will notice those, and maybe it will figure out the pattern for follow-up page requests, but there's no guarantee and it won't help the first request.

Not necessarily. They could also send a HEAD request to the URL first, to see the headers only and decide on the available alternates.

I am well aware that few sites are taking that much care of their API in terms of HTTP features, but all of the problems discussed here have solid and battle-tested answers.

The parent was referring to adding `link` tags in the `<head>`, not in HTTP headers.

I realised that after posting, but then I suggested sending them as header in the first place.

The ideas is that the URL references the resource and the content type requested is only asking for that content in a different projection or representation.

The content at a URL should always match, the format in which its represented can be different based on the request. Its a bit like buying a book in hard copy or paperback, same book different format.

Reply on news.ycombinator.com