The requested URL did not exist, so the server must return some response other than 200. It could be not-found, it could be forwarding, or it could be something else.
However, 200 (success) is simply wrong.
However, 200 (success) is simply wrong.
You can't write code relying on random third-party code doing what it's "mandated" to do by the standard, because malicious code exists and so do malicious services. So as a client, if you're connecting to services you don't control, you need to be able to handle misbehaving third-party code.
And what it means for a request to be "successful" is entirely at the discretion of the author of the software. I think it's fairly clear that the software looked for a response, found one, and successfully returned it to the client.
Which isn't to say that the code is optimal, necessarily, but unless you have a contract to say otherwise then I don't think you've any basis to claim that it must do anything in particular.
The relevant part of the spec says that returning a 4XX code is a SHOULD [0] not a MUST. That means:
> This word, or the adjective "RECOMMENDED", mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course [1]
[0] https://www.rfc-editor.org/rfc/rfc9110.html#name-client-erro...
[1] https://www.rfc-editor.org/rfc/rfc2119
So yes, there is no requirement to return a 4XX message, but you should think carefully and understand the consequences of not doing so.
One consequence is creating false positives in most vulnerability scanners.
Another consequence is potentially messing up various crawlers, including those that help search engines index your content.
As a client, you've got to be ready to deal with non-conformant implementations. As a service owner, there is no authority who will hunt you down for deploying a service that doesn't conform to the spec. Let me restate my claim slightly: acknowledging that impolite to claim to conform to a spec and then not do so, no-one has a right to claim that any arbitrary person must write code that conforms to any arbitrary RFC.
> ["MUST",] "REQUIRED" or "SHALL", mean that the definition is an absolute requirement of the specification. (my emphasis)
I may choose to return whatever response code I want in whatever circumstances I feel like, and there's absolutely nothing you can do to stop me. I probably won't, because that would be silly. But there's no must.
The whole point of the "MUST" vs "SHOULD" is that you can assume that you don't have to handle implementations that aren't compliant (e.g. don't do the things they must do to be in compliance) but do need to worry about handling edge-cases where implementations are in compliance but not doing things the way they should.
Both tools (the vulnerability scanner and Prometheus) should make changes but I would consider the Prometheus change to be a bug report (which they can clearly choose to not fix and still be compliant) and the vulnerability scanner would be a feature request to check file contents rather than just relying on the status.
But unless you control all of the implementations you're interacting with, you can't assume that they'll conform. You might well decide to handle non-compliance by declining to continue, but you still need to handle those cases.
I don't disagree with your assessment of the changes required to Prometheus or the vulnerability scanner, but I'd rate the Prometheus bug as really low priority, and the feature request for the scanner as a blocker for adoption: it might be correctly behaving as designed, but that doesn't mean the design is useful.
What makes it wrong? HTTP/HTTPS URLs/URIs don't mean files on the disk. You can return 200 as a default if you like.
The correct response code would be a 3XX or 4XX response code, depending on the client behavior that you want.
In this specific case, a 303 response that redirects to the homepage would be most correct.
Alternatively, you could return a 404 response alongside the default content.
The point is, "resource" is an abstraction that servers may implement as they wish. The only requirements, per HTTP, is that (1) it's something that can be identified with a URI (satisfied in this case), and (2) it has information associated with it that can be retrieved and/or managed via the HTTP protocol (satisfied in this case).
We might imagine that the set of resources should be finite, or strictly mapped to extant data records in our system – but these are not requirements.
Clearly they aren't requirements as I can think of many valid use cases that violate them.
To me, creating an infinite number of identical aliases for a single resource is clearly wrong. The only reason to do it is either incompetence or laziness.
Many responses here seem to be based on the misunderstanding that this is about "not following a standard is sometimes okay-ish", when it's actually about the fact that the standard does not rigorously define what a "resource" is and what it means for the resource to "exist", other than a circular definition based on status codes.
You talk about violating conventions and standards. The conventions are defined by the HTTP spec. The standard is the HTTP spec. Neither are, so far as I can tell, being violated here.
What if you want to confuse the client?
Isn't that just begging the question? What defines whether a resource exists? Is it the client's knowledge about the webserver backend configuration, or the server's? If a server returns a 200 response, the resource exists, by definition. It may or may not have data, but again, that's not up to the client to decide. If the server says the resource exists and has no data, then that's the authoritative answer.
If the client behavior you want is to display your styled error page, that has historically meant returning a 200 status code, because some user-agents prefer their own error displays.
It’s like saying “what makes it wrong to signal left but turn right?”
It makes the component you’re operating behave less predictably, resulting in a less stable system overall. Follow the spec.
> It’s like saying “what makes it wrong to signal left but turn right?”
No, it's not like that. Signalling left but turning right would be like returning a 200 for a resource that does not exist.