ES proposal: arbitrary precision integers | Hacker News Reader

ES proposal: arbitrary precision integers

2ality.com

56 pointstosh49 comments

This is so Javascript. It adds arbitrary precision integers (which I would love) in a way that seems to me to be guaranteed to be hard to use and bug-prone.

The 'n' suffix is clumsy as hell.

You can't mix Numbers and Integers? That's crazy.

23n + 2 is a TypeError even though "23" + 2 isn't?

Ya the ban on mixing stood out for me too. Javascript and many of its frameworks are too opinionated these days (attempting to govern the way users code) even though that's inherently not in their purview.

Also I didn't see anything about the internal format of these numbers. Normally this shouldn't be a concern, but could affect memory usage.

Giving this proposal a vote of no confidence, even though the need for arbitrary precision is high.

> 23n + 2 is a TypeError even though "23" + 2 isn't?

The former is banned because it's error prone the latter is allowed because of backwards compatibility.

I'm going to need an explanation for how error prone it can be to add two numbers. Automatically promote the '2' to a '2n' and carry on. Where's the problem?

If you do 23n + x, you probably want it to raise an error if x isn't a bigint. Otherwise you end up with more situations similar to NaNs appearing in unexpected places.

It's arguable. There are at least two schools of thought, and both of them are valid. It just depends what you want to do. If you're using arbitrary precision, you're probably in a situation where it's better to be explicit than to have implicit coercion.

For example, what should happen if you do 23n + 0.5? You can't have 23.5n, since it's not an integer. You could convert it back to a Number, but you can't do that if it's 9007199254740992n + 0.5. It's logical to throw an error, if a bit counterintuitive.

Comparisons on the other hand probably shouldn't throw errors.

I don't see this as any different than coercion with other pairs of types.

I mean, sure, you can have that view. But again, what would you say should happen for 23n + 0.5? And for 9007199254740992n + 0.5? If you don't throw an error, you have to do something, and in those situations returning anything would be incorrect.

You could return 23.5 in the former case, but then you'd no longer have an Integer. You could throw an error in the latter case, but that'd be a lot slower for JIT tracing compilers to execute than simply verifying that the types are the same.

Treat it as a arbitrarily large floating point value, rather than an integer.

Good point, though in that case using arbitrary precision values anywhere in your program might tank performance. Execution speed still matters, especially now that we're at the physical limits of single cores, and automatically promoting to arbitrary floats seems like a poor trade of speed vs flexibility. It's way too tempting to "simply use arbitrary floats and be done with it", even in financial software, and then be in a situation where the entire program is slow. But it would be good to quantify how much slower it'd be.

Animats pointed out a potentially more serious issue, which is that you can DoS a program by sending it an arbitrary precision number whose size is gigabytes. But that's true of any arbitrary precision number, not just floats.

I think in that case, internally it can handle smaller values differently than larger values (like Ruby). If the performance of large values is truly important, something using the FFI can be written to a program using something like C or Fortran.

"You could return 23.5 in the former case, but then you'd no longer have an Integer."

This works rather well in Python.

    >>> a = 10
    >>> b = 1.1
    >>> c = a + b
    >>> type(a)
    <class 'int'>
    >>> type(b)
    <class 'float'>
    >>> type(c)
    <class 'float'>

There's a solution to all of this: TAoCP has thought through these issues carefully, and it would probably be best to study the tradeoffs Knuth presented.

Python handles common cases, but it breaks when you push it too far:

  >>> 1e100
  1e+100
  >>> 1e100+0.1
  1e+100
  >>> 1e100+0.1 == 1e100
  True
  >>> 9007199254740992 + 0.5
  9007199254740992.0
  >>> 9007199254740992 + 0.5 == 9007199254740992
  True
  >>> 9007199254740992 + 0.5 + 1
  9007199254740992.0
  >>> x = 9007199254740992
  >>> y = x
  >>> y += 1
  >>> x == y
  False
  >>> x = 9007199254740992 + 0.5
  >>> y = x
  >>> x += 1
  >>> x == y
  True

You could argue that handling the common cases is all that's needed, but stuff like this pops up in e.g. writing software rasterizers. It might also be one reason why PyPy can't quite match the speed of native programs, though that's just a guess.

You're right about those Python cases (and they are surprising!). But if you're dealing with numbers those big and adding decimals you would want to use the Decimal module:

>> from decimal import Decimal as D

>> 9007199254740991 + D('0.1')

> Decimal('9007199254740991.1')

It wouldn't work well in JavaScript because it already has double-precision floats as its datatype used for both integers and floats. Python doesn't have floats getting used as integers, which is what leads to this problem.

Actually, I'd argue that the former is also because of backward compatibility: if it wasn't, it would be easy to treat all literal integers as Integers and everything else as Numbers, similarly to what exists in Python.

I would dare say that even with this approach it would be possible to maintain backward compatibility with smart conversion, although I'm not familiar enough with ES internals to be more decided on that.

Personally I think JavaScript is already full of "smart" conversions and the trend now is to make new things simple instead of fueling XXI century's WTF.js (see http://brian.io/slides/dotjs-2012/ ).

Also why "n"? Why not "i" or "d"?

They had to pick one.

N seems like a bad choice for Integer when the thing it isn't is Number.

i is for sqrt(-1), d is for decimal, n is for "not not a bigint".

>This is so Javascript

It's not, it's the exact opposite. Instead of doing type coercion, enforces strict typing. It's what people have asked for for ages, now they get it.

This is almost exactly the opposite of how classic JavaScript would have handled it. Your comment of "This is so Javascript" seems ignorant.

There are some people that are saying not allowing Number and Integer to mix is error prone, but I don't really see it. For example, in Ruby if you do a calculation that's large enough, it just becomes a Bignum [1]. The easiest way to handle this would probably be to just allow arbitrarily large numbers, and if it overflows beyond the Number, turn it into a Integer automatically (and vice versa, if it becomes smaller than a Integer, turn it into a Number). In Ruby, the Bignum and Fixnum apis are also the same, with only the internal representation, as well as a few specific differences in how you handle a Bignum, as differences.

I can kind of understand the FAQ section about it, since JavaScript in the browser has to be backwards compatible, but are there really that many applications that depend on Integers acting like they do now at such high values that just transforming them into Numbers won't suffice? The only difference would be for bitwise operations, which are primarily used in Node, and you can explicitly document that change in a major version update, unlike in a browser.

Also, nitpick: Why not just call it BigNumber, or at least BigInteger? Integer as the name of an arbitrarily large number is going to be confusing.

Edit: newer Ruby versions treat Bignum and Fixnum as just Integer (since it was mainly an implementation detail), but my argument is still the same.

1: http://ruby-doc.org/core-2.2.0/Bignum.html

> In Ruby, the Bignum and Fixnum apis are also the same, with only the internal representation, as well as a few specific differences in how you handle a Bignum, as differences.

In current Ruby, "Fixnum" and "Bignum" are legacy-support aliases for the unified "Integer" class.

http://blog.bigbinary.com/2016/11/18/ruby-2-4-unifies-fixnum...

> Integer as the name of an arbitrarily large number is going to be confusing.

"Integer" in mathematics is the name of an class of numbers that is unbounded on both the high and low end; it is using "Integer" as the name of a restricted range that is counterintuitive.

The argument is still the same, now the user facing aspect is hidden (I don't know the Ruby source code, but I would guess they just rewrote the external facing code so the user doesn't have to care about Bignum or Fixnum, and just care about a single Integer).

As to your second point, why not just move everything to Number then and if it needs a decimal point, treat it as a floating point value? I argued in the original post that there can't be that many applications that depend on the current implementation, and those that do purposefully can be rewritten.

Edit: Integer -> Number

> The argument is still the same, now the user facing aspect is hidden

Well, the language has changed so that they are unified. The underlying implementation may use different representations at different sizes, and presumably CRuby still uses the same internal split that the language exposed externally, at least for now.

> As to your second point, why not just move everything to Integer then?

Because not all numbers are integers, so you still need JS's number type (limited range and precision reals) if you have arbitrary precision integers.

With this, can you send JSON with arbitrarily large integers to an API and make them churn on a gigabyte of digits? Has potential as a denial of service attack.

You could already send gigabytes of JSON garbage right now and DDoS them...

Also i don't think this has anything to do with JSON.

>"Integers in JSON data will probably be handled similarly to other unsupported data such as symbols:"

    >JSON.stringify(123n)

    undefined

I guess their proposal accounted for this by not supporting integers at all. But this could still be possible if you knew a service was coercing a string or similar into an Integer type (using proposed Integer.parseInt).

How is this attack vector any different from sending an arbitrarily large string?

In that they can restrict loading after a specific amount to KB or MBs -- but even a "small" 100-digit integer could have disastrous runtime behavior when passed to an API.

It's important to remember that JSON is "very close" to Javascript but is not actually Javascript. JSON allows arbitrary precision numbers. So, the number `1.0000000000000001` (or, similarly, an extremely large integer) isn't representable in standard Javascript, but if a JSON encoder emits that, a "proper" parser would actually parse that as an exact value (compare Python `simplejson.parse` with `use_decimal=True` vs `use_decimal=False`). I guess what I'm saying is that a proper JSON parser kinda already has this issue.

Arbitrary precision sounds potentially expensive. Will using them be faster for 64 bit ops than "manually" managing a pair of doubles?

That's an implementation detail, not something that needs to be spec'ed out.

Who does integer math in JS with numbers larger than 9007199254740991?

The examples in paragraph 1 are moot. An ID doesn't have to be represented with integers locally (it's probably because the server stores it as 64 bit ints) and a string is perfectly fine here. Furthermore, "financially technology" working with such large numbers in JS? Are you kidding me?

Edit: downvote without an answer to my question? very helpful.

A simple use case is incrementing a >53-bit integer id.

When you need incremental ids that large, you're doing something wrong. At the current rate (6000 tweets per second), twitter will run for >47600 years with 53 bit.

Sure if you only have one system generating the ids.

Once you deal with distributed systems, you need some "fat" to prevent duplicate ids from being generated. That's usually done with an N-bit prefix identifying the server plus a time stamp prefix.

There, problem solved for both the developers that need it:

https://www.npmjs.com/package/big-integer

The thing is that the API would still have to send them as strings even with this new ES feature because JSON doesn't support it.

Anyone dealing with data produced by other languages which uses 64-bit integers. Easy examples include tweet IDs (broken in JavaScript, because JS can't handle 64-bit integers), POSIX inode numbers (broken in node.js, because JS can't handle 64-bit integers), etc.

The lack of JSON support seems odd. One of the primary use cases for big integers is parsing big integers from another language via an API.

The lack of automatic number coercion seems lazy. I would prefer number approximations and warnings. Do any other primatives throw errors for operators instead of performing type coercion?

> Do any other primatives throw errors for operators instead of performing type coercion?

Yes, Symbols cannot be implicitly converted to strings for example.

Symbol has a toString method. It makes no sense to throw an error instead of just calling toString (like every other primative does). The whole point of making toString part of the Object prototype is that everything can be coerced into a string. Why are these new specs getting so error happy?

Primitives themselves to not have methods, maybe you thought about primitive wrapper functions?

Even then implicit conversion to string is special cased for each primitive (you can't call toString on null or undefined, see [0]).

> Why are these new specs getting so error happy?

I believe is called "fail fast, fail loud" principle. Remember Symbols are to be used as unique keys, if you accidentally mixed them with strings you'd lose this uniqueness.

[0]: https://www.ecma-international.org/ecma-262/6.0/#table-12

If you accidentally mixed an object with a string you would lose the ability to access it's properties, but that's not a good reason to throw an error.

---

> primitive wrapper functions?

I though that primatives would be unboxed so that their toString method could be called. Apparently not.

Symbols don't have any interesting properties to access.

The problem is this case:

  s[t + ""] vs s[t]

For all types they are the same because as we know in JS all properties (even array indexes) are converted to strings. But not for symbols, if you use symbol as "t" then implicitly calling toString would give you wrong result in first case. As this is clearly different from the rest of JS objects it's better to throw an error rather that silently proceed.

> I though that primatives would be unboxed so that their toString method could be called. Apparently not.

It's the other way around. Primitives do not have any properties so to call a method on it they are boxed before call.

An example:

  Number.prototype.getBoxed = function () { return this} 

  var x = 2..getBoxed()

X will be a Number wrapper object, not a primitive.

if (nth === 0n) when nth is never manipulated. Splendid example.

There's a -- before the nth. Did you miss that?

Reply on news.ycombinator.com