A new method to validate URLs in JavaScript | Hacker News Reader

A new method to validate URLs in JavaScript

stefanjudis.com

27 pointsfagnerbrack18 comments

> The only thing remaining is the question "What is a valid URL?" but I'll leave this one for another time! Because it's a tough one.

Is it? According to https://www.w3.org/TR/2011/WD-html5-20110525/urls.html, a URL is a valid URL if at least one of the following conditions holds:

• The URL is a valid URI reference [RFC3986].

• The URL is a valid IRI reference and it has no query component. [RFC3987]

• The URL is a valid IRI reference and its query component contains no unescaped non-ASCII characters. [RFC3987]

• The URL is a valid IRI reference and the character encoding of the URL's Document is UTF-8 or UTF-16. [RFC3987]

Except the RFCs are not the only URL standards. Browsers all implement the WHATWG living standard, which is subtly incompatible with the RFCs.

if they diverged from standards that's on them. What is WHATWG anyway, and what is a 'living standard', and is it a standard at all, or a lack of standard?

https://url.spec.whatwg.org/

Valid according to who? If you're automatically linking text that looks like URLs, this will fail since most people don't type the protocol when sharing a website URL.

agreed, as to normal user, I would think both in the post example are valid URLs

I detest that style of auto-linking. You get so many false positives—and sometimes troublesome ones at that. With the possible exception of text with a `www.` prefix, the entire thing is just a misfeature. Let people copy an entire URL if they mean a URL.

Validating absolute URLs is easy (edit: with a try..catch statement). I was hoping that this standard would introduce a utility for validating both relative and absolute URLs.

Here's how I do it: https://gist.github.com/eligrey/443d51fab55864005ffb3873204b...

As you can see, this implementation fails for URLs with - as their host. There should be a cleaner solution here that doesn't involve creating new URL instances with test scaffolding (the input base URL).

There's a second overload that takes a base:

https://developer.mozilla.org/en-US/docs/Web/API/URL/canPars...

That's quite useful! I'm going to integrate that into my code and fallback to the previous implementation if canParse() is not available.

Not fond of your function name isValidURL: it’s more about checking if a URL is represented canonically (… except that it also allows https://example.com with no trailing slash), rather than just if it’s valid.

What would you propose as a better name? I recognize that this doesn't align fully with the spec. This has been helpful for my specific use cases.

> Validating absolute URLs is easy.

I disagree.

Your approach does not address[0]:

  By convention, domain names can be stored with
  arbitrary case, but domain name comparisons for
  all present domain functions are done in a
  case-insensitive manner, assuming an ASCII
  character set, and a high order zero bit.

Nor URL's with user and/or password elements[1].

Nor the different, yet equivalent, path encodings such as "/has+space" and "/has%20space".

Nor does it normalize path segments before determining value equality, such as "/foo/bar" being equivalent to "/foo/blah/../bar" as well as be equivalent to "/../../../foo/../foo/blah/../.././foo/./bar".

So no, validating absolute URL's is not "easy."

0 - https://datatracker.ietf.org/doc/html/rfc1034#section-3.1

1 - https://datatracker.ietf.org/doc/html/rfc1738#section-3.1

The utility mentioned isn't the approach I would recommend for 'validating absolute URLs' -- for that I would simply use try..catch with the URL() constructor and no base URL.

My linked utility is specifically for validating that a URL is valid (won't throw an error when passed to the URL constructor) and doesn't need additional encoding. This helps with my use case which is a 'create URL classification' UI that allows users to input URL matchers in a multitude of formats.

For additional context, some inputs are invalid even with a base URL. e.g. new URL('//:0', 'https://-') will throw an error.

Your first critique doesn't seem relevant as this is mostly for checking if a URL is 'valid' (i.e. doesn't throw an error when used).

Also, for your second critique, the username + password is actually part of the origin as used by both of my snippets. For example, isValidURL('https://a:b@c.d/') and isValidURL('https://a:b@[::1]/') both return true for me.

Do you have a URL that gave a bad result? If so, feel free to mention it here and in the comments for the gist so that users of my snippet can be made aware of its limitation.

"-" is not a valid host.

That is a good point. It's also a potential source of input desynchronization vulnerabilities as 'valid' (does not throw an error when passed to the URL constructor) URLs can contain invalid hosts.

There is a distinction to be made between a parseable string and a valid URL in more complete terms. While it's nice to have a new standard function that checks whether a string is parseable as a URL, it can't, of course, validate that string in terms of whether the TLD is a real one, and not something like "https://super.hacker/".

So maybe the post should have called it what it is: validating parse-ability, since the method itself is called "canParse()".

super.hacker is a valid url.

Url validity isn't dictated by ICANN.

Reply on news.ycombinator.com