ECMAScript proposal: String.prototype.replaceAll | Hacker News Reader

ECMAScript proposal: String.prototype.replaceAll

2ality.com

56 pointsjoeyespo68 comments

I don’t understand the rationale behind throwing an exception if you provide a regular expression which lacks the g modifier. This seems really counter intuitive. I would expect replaceAll to replace all occurrences no matter if the regular expression is global or not.

> The assumption is that we made a mistake and should switch to .replace() if we really only want to replace the first occurrence.

Unlike the string case, the regexp case has a way of communicating that you want to match every instance versus only the first

That is an interesting way of seeing it. I’m not saying I disagree with the rationale, but I find it weird that a function which promises to do something needs a specially modified version of its input _not_ throw an exception. That does not sound like a good API design at all.

This feels like thinking about it the wrong way round: if you want to use regular expressions, use `.replace()`, which has full regex support and always has. There is no need to ever use `replaceAll` with regular expressions. In fact, you'd just be mudding the waters by using a function that implies "all" in combination with a regex that implies "single" (if you don't specify a `g` flag, as per regex convention).

However, if you just want to replace plain strings, `.replaceAll()` actually gives you a fundamentally better API: you can now use `str.replace(s,r)` for single replace, and the intuitive companion function `str.replaceAll(s,r)` for all-instances replace. No "learning/knowing regexp" required just to replace some plain text.

If there is no need in using replaceAll() with regexes, why would it even support it?

I agree that only replacing the first occurrence is the wrong solution. That would be surprising. The only correct solution to this is in my opinion auto-g-ing the regular expression, no matter how unusual that is. There is no way you would write replaceAll() as a mistake when you actually just meant replace().

Mostly to keep the signature for replaceAll in line with replace. You _can_ call it with a regex, but then you'll have to make sure it's a regex that uses the correct syntax for "replace-all" behaviour. If you don't, now you're doing something so weird that an error is the only appropriate response.

<<There is no way you would write replaceAll() as a mistake when you actually just meant replace(). >>

Have you ever reviewed the daily work of colleagues or contributors who haven't used JS day in day for the last 2 years? That's exactly the kind of mistake that happens _all_ the time. And people new to JS are more than happy to reach for regex well before there's a good reason to use them.

Is there a way to find all occurrences/matches of a regex that doesn't have the 'g' modifier?

Just tried `string.match` and `regex.exec` and could only get the first match.

I don't think so without modifying the string or regex each iteration. You'd have to do something like

    let flagified = new RegExp(original.source, original.flags + "g")

 or modify the regex standards as part of the proposal.

I tried to do this a while ago for a side project. There might be a really obscure way (and of course you can always write your own function), but I wasn't able to find anything built in.

It has to do with certain behaviours of RegExp objects, such as the `lastIndex` property, as well as consistency with `matchAll`. There was a lot of discussion[1] about what should happen with a regex that doesn't have /g, and in the end the most consistent choice was to just disallow them entirely.

[1]: https://github.com/tc39/proposal-string-replaceall/issues/16

The irony in that discussion regarding behavior of matchAll: "Once something has shipped in browser's we can't change it".

Chrome: deprecating and removing Custom Elements V0 in 2020.

*V0, this has always been explicitly unstable

Also, Custom Elements is in the area of W3C, not TC39, I believe. There could be different rules and people involved. Although I'm sure they'd like to avoid breaking the web too.

Indeed, and that's the point I failed to make. TC39 is rightfully concerned about not breaking the web. They are very mindful even if features that have reached stage 3 (because by this time everyone is already using them).

Browser implementors though... They take a much more callous approach.

And used by the biggest websites and projects on the Web (Google's own YouTube and AMP until very recently).

How many more websites are using it because it was out, and implemented?

Because no one else shipped it - “V0” was a terrible attempt at a spec invented by the chrome team and shipped without meaningfully talking to the rest of the community. This “throw a pile of terrible features at the wall and see what sticks” while berating other browsers for not also implementing the badly designed and poorly specified specs is pretty much part and parcel of chromes embrace+extend+extinguish approach to “open specs”.

I too was surprised by this, but for the opposite reason: I would expect replaceAll to replace all matches for the regex, meaning if the regex only matches the first occurrence of a pattern, then only the first occurrence is replaced. The fact that we both made perfectly opposite assumptions about what is "obviously right" is strong evidence for their decision.

Isn't that how .replace works? https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Perhaps - I rarely write JS. I'm speaking purely based on this function alone.

I disagree that this makes their decision the right one. If we can come up with so many "valid" versions of this, all with their own flaws, then this is the wrong API to build. This function should not exist in its current form at all. APIs should be intuitive, preferably without reading the documentation (although I agree that this is a utopian way of seeing it).

I agree. I would think replaceAll would behave like the following:

/* Usage:

  > 'abcdefabc'.replaceAll('abc')
  'def'

  > 'abcdefabc'.replaceAll('abc', '+')
  '+def+'

  > 'abcdefabc'.replaceAll(/abc/, '+')
  '+def+'

  > 'abcdefabc'.replaceAll(/abc/g, '+')
  '+def+'
 */

 String.prototype.replaceAll = function (re_or_str, sub) {
     return this.split(re_or_str).join(sub||'')
 }

On a side note, is it just me or does the linked polyfill for String.prototype.replaceAll [1] seem wildly complex???

[1] https://github.com/es-shims/String.prototype.replaceAll/blob...

Agree, seems much more complex than it should need to be, but I’m guessing there’s edge cases (but even then it seems more complex than needed).

Your implementation is missing features like capture-based substitions

     // Remove repeated characters
     'foobar'.replace(/(.)\1+/g, '$1')
     "fobar"

and overriding behaviour with Symbol.replace

You're right, my implementation was not a drop-in replacement for replace. How about this?

    String.prototype.replaceAll = function (n, sub) {
      let t, f = "g";
      if (n instanceof RegExp) {
        t = n.source;
        f = n.flags;
        if (!f.includes("g"))
          f += "g";
      } else {
        t = RegExp.escape(n);
      }
      let r = new RegExp(t, f);
      return this.replace(r, sub || "");
    }

How do you know when an entire language is ready to be thrown in the trashcan? When you are having a debate about what "all" means.

I would have make the "escapeForRegExp" function built-in (it seems to me the kind of thing should be built-in), perhaps called RegExp.quote. (I would also to add support for many features of PCRE.)

You can use String.raw:

  new RegExp(String.raw`\s`)

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

No, that is something different.

Yes, but it works quite well for this task.

It has nothing to do with this task.

  function replaceAll(str: string, pattern: string, replacement: string): string {
    return str.replace(new RegExp(escapeForRegExp(pattern), "g"), replacement);
  }

There's a proposal for RegExp.escape() which was rejected by TC39 for imho quite silly reasons: https://github.com/benjamingr/RegExp.escape/issues/37

I don't understand the reasons. What is the actual problem in this "even-odd problem"?

How do all the other languages that have an equivalent function (Perl quotemeta, PHP preg_quote, Python re.escape, Ruby: Regexp.escape, Java Pattern.quote, C# Regex.Escape, Go QuoteMeta, Rust regex::escape) avoid or deal with this problem?

They don't. The reasons are frankly idiotic.

The first couple of comments as well as the very last one from April in that thread all sum it up quite well.

This is insane

I wish the pipeline operator were standardised so we can stop fiddling with prototypes.

This would be nice, using RegEx's for such simple cases is ugly, especially if the replace string is dynamic (so can't be done with a literal).

I find myself using a combination of split and join a lot to avoid that.

And because regexes are regexes, so one might introduce bugs by not escaping what's meant to be escaped. (I didn't read the article so maybe he mentions this already)

I think JS has had enough of syntactic sugar like features. What it needs as a next step in its natural evolution is support for proper multi-threading with atomics, mutex etc.

What would you do with multi-threading in JS? I/O is already non-blocking which is a big reason for using multi-threading.

So much fuss for a small function

It being small does not mean it's not critically important to get the details right. In fact, if there's any correlation between smallness and frequency of use, then it's actually more important to go to the full lengths of fuss resolution.

Really you think in 2020 the software world should have an open discussion about a string replace function? Ok

For the world’s most popular language? Yes.

Mmmmm...syntactic sugar...

Not sure what you mean? This doesn't add or use any new syntax, let alone syntactic sugar. It's a new builtin method.

Could you provide an example of what this desugars to? I'm not sure I understand.

You can simply use string.replace() with a global regex.

Dynamic patterns require escaping, resulting in unnecessary complexity and overhead.

Yes, you are (of course) able to replace all occurrences today. However, you either need to call replace multiple times or pollute your code with a regex. This would certainly improve the readability.

Why would using regex pollute your code?

It depends on who you ask; however the thinking is often:

* They can indicate poor parsing (ie can't read xml with regex) and failure to use programmatic methods (e.g parsing command output rather than making syscalls)

* regexes are very easy to write incorrectly without being obviously wrong

* regexes are often overkill for a direct match

* regexes invite people to make it more complex rather than restructure surrounding code

* regexs are comparatively more expensive than alternative code

* to work on the codebase you are now required to understand regex in general and this particular implementations quirks as well

I often use regexes, and I often choose not to use regexes.

While this is a nice little addition to the language, personally I'd prefer to see more substantial changes like decorators, observables, class properties, optional static types, etc.

Class properties are already part of the language

My bad, I meant class fields which are stage 3:

https://github.com/tc39/proposal-class-fields

https://tc39.es/proposal-class-fields/

Exhibit $n why ECMAScript is not a well-designed language at its core. It ought to be as simple as the Lua core language but it isn't (the object type was a good start but then stopped being one). Boltons after boltons. For things as simple as this.

lua has s:gsub, which is basically the same thing... or are you saying js should've popped into existence with thousands of stdlib functions?

PHP at least is sane enough to provide the C stdlib, dito for Java.

JavaScript... well, remember left-pad? That's what happens when people are forced to re invent the wheels.

left-pad wasn't a fault of the language, but the modern javascript ecosystem and its tools. Javascript development got along fine for years when all anyone had was JQuery plugins, no one was complaining about the lack of a C style stdlib, and no single point of failure for the entire ecosystem created an internet-wide catastrophe every week.

Everyone complained about the lack of a C style stdlib. That's why jQuery was written and everyone used it.

People complained about incompatible versions of the DOM between IE and everyone else, and about how awkward AJAX and DOM manipulation was, and those are the problems JQuery solved. That's providing a common API and mostly forcing Microsoft to follow spec, not a "C style stdlib," though.

I don't think many people were upset that javascript didn't natively support, say, proper integers or static types or (ugh) "classes" or what have you, at least not until people started wanting to use javascript outside of its intended purpose.

You don’t count to thousand looking for:

- replace all occurences,

- remove n-th element,

- binary search for an item,

- sort an array without implicit string coercion

in a good stdlib. Lua is explicitly minimalist, so it is not a good example here.

I don’t understand your criticism. Are you criticising JavaScript for getting a feature you would assume it already had?

Couldnt we just have an object called String2 with a replace method that replaces all occurances without any regex at all? Like in any other language? We could fix entire Javascript just by adding two to all object names.

This isn’t String.replaceAll(str, match, replace), it’s str.replaceAll(match, replace). A function of the type, not a class.

This is such a terrible idea.

You right. We should have Javascript2 language.

https://xkcd.com/927/

This strip is about creating unifications based on two things that are not standarized. While the idea is to make a new JS branch that will solve issues gathered through years. So the strip does not apply here. Also, while it has some point with extreme unification efforts, there is a vast range of unification projects that actually delivered what they promised to.

Reply on news.ycombinator.com