Hacker News Reader

Comment by airocker | Hacker News Reader

Can we not have one integer type in c that can grow like in js as required and become bigint if too big?

Haha, yeah JS integers "can grow". Kind of. And then they bite you in the worst way possible. Especially when combined with cryptography (e.g. nonces).

Try this:

  >> const x = 288230376151711740;
  >> x == x + 1
  true

Or this:

  >> 2\*1024
  Infinity

JS doesn't even have integers. It only has floats. JS is by far the worst language I know of when it comes to integer support.

If you want a better example of arbitrary precision integers, try Python, for example.

I was working at Twitter when we learned this the hard way. We switched to a separate service for generating tweet IDs (instead of MySQL sequences), which for various distributed systems reasons meant IDs started taking up more bits. And when we stuck those bigger IDs into JSON responses in the API... well, we learned the dumbest lesson possible about the intersection between JSON, JavaScript, and double-precision floating-point. Later that day the "id_str" fields were born.

Yep, I can totally relate. I found multiple bugs in a MessagePack JS implementation that were related to this issue.

I also worked on a project once where unique IDs of objects could get quite large (because they were random u64 integers). Those IDs were serialized and sent to a browser. Sometimes two objects were viewed as "the same" in the browser application because their IDs were truncated by the floating point precision issue.

Neat trick.

    Math.pow(2, 53) == Math.pow(2, 53) + 1

This is a bit clearer I think. The addition just barely overflows the 53 mantissa bits of the IEEE 754 double precision floating point number.

By the way, JS has BigInts these days, which are supported by all major browsers: https://caniuse.com/bigint

    >> const x = 288230376151711740n;
    >> x == x + 1n

    false

  2**53 == 2**53+1

^^ even more clear.

That's why Number.isSafeInteger(a) exists.

Integers up to +/- 9007199254740991 (Number.MAX_SAFE_INTEGER) are fine.

GP may have been referring to BigInt.

Although, C does have an equivalent to that: the GNU MP library (and probably others as well).

I was referring to bigint not js bigint, checking for overflows if optimized could be just 1 instruction.

We don't need arbitrary-sized integers; we need exceptions on overflow (or underflow, but I'll stick to overflow for the rest of this post) to be the default, or similar language features as appropriate.

I for one am tired of the chicken & egg issue of "CPUs don't support efficient overflow checking because nobody uses it, so it's slow" and "Overflow checking is slow because the CPU doesn't support it, so nobody uses it". For all the other good security work done in both software and hardware, much for things far more complex than this, this seems like an absolutely batshit insane oversight considering the cost/benefits for fixing this.

CPU would raise a signal just as null pointer exception? And it seems a lot of code (for eg, safeintadd metioned below) assumes no exception. Would not all that code get messed up?

Would it be possible to just silently replace with arbitrary sized integer and not break any code like safeintadd?

CPUs don't raise exceptions, that is a software concept. CPUs do have traps, like for division by zero, but those are not exceptions in the way you think.

Null pointers are also handled by the kernel, not the CPU. Its called a segmentation fault because you are trying to access a memory segment that the OS doesn't want you to.

I dont think kernel handles NPEs: https://stackoverflow.com/questions/63091446/what-happens-at...

From your link:

A NULL pointer in most (but not all) C implementations is address 0. Normally this address is not in a valid (mapped) page.

Any access to a virtual page that's not mapped by the HW page tables results in a page-fault exception. e.g. on x86, #PF.

This invokes the OS's page-fault exception handler to resolve the situation.

Yes, page fault exceptions happen within the CPU, they cannot happen in software. I think of it as C is close to assembly and does not check every memory address is referencing. It just compiles and CPU starts running it. If somehow a memory of 0 is dereferenced, it is already run by the CPU in its fetch/decode cycle. but of course once the excetion happens, Kernel is responsible for killing the actual process. So both CPU and kernel do it together.

It's worth noting that the term "exception" is in fact used for traps that aren't interrupts.

In C, you can't "just" replace with arbitrary-sized integers. They are fundamentally different memory shapes.

You may not be able to turn enforcement on for all code immediately. There's even the rare bits of code that depend on current overflow behavior. (Due to our human brains and the fact that we can easily name these bits of code, making the cognitively available, people often grotesquely overestimate the amount of code that operates this way. I'm sure it's only a matter of how many zeros belong in the 0.001%.) But we need this support to be available for code to be turn on easily and cheaply.

But what really boggles my mind, again given all the security work we've done, is that the reaction to this remains a combination of silence and "we can't do that!", when it seems to me the reaction ought to be "well duh jerf we all know that already." I don't get this. I don't get this attitude at all. This is a huge source of errors, a good fraction of which are security bugs, and nobody seems to care. Incomprehensible. This is, arguably, the number one thing that could be getting changed right now to fix security issues, and I just get slack-jawed "whaaaa?" in response to the idea.

One hundred plus one hundred is not negative fifty six! Here we are trying to hold together megabytes upon megabytes of security-critical code in a world where 100 + 100 = -56. Is it any wonder that writing any sort of code to maintain security invariants is tough in an environment where 100 + 100 = -56?

I feel like carefully planned with compiler level errors(not warnings), and some glibc non-backward compatible changes. we can achieve this in software layer.

We can do this in software because there's virtually no limit to the abstractions people make, but hardware doesn't work like that. At some point in the software stack we need to draw a line and say this is a 64-bit signed integer that hardware can understand.

But should it be everywhere including the fs/virtual fs layer? Could we limit it only to device drivers? Not a kernel expert here and would love to hear thoughts.

This doesn't have anything to do with filesystems or kernels.

The x86 assembly uses fixed width immediates. CPU registers are a fixed width. For any code to compile and run, it needs to make decisions about how large stack frames need to be, and how much heap memory to allocate.

This was the parent's point about abstractions. You can make a library that pretends to be a variable sized integer, but to implement such a library you need to make a decision about how much space to allocate and the width of variables in order to compile. There is no getting away from how the hardware works.

Yes totally, but FS is a Kernel abstraction, it deals with questions like: How to keep a directory? How to keep a file? How to keep synlinks. Etc,

These can be implemented in any higher level language or within C without using fixed width numerals if correct abstractions were available. Only device control parts should use fixed with numerals in my opinion.

The problem is syscall interfaces are facilitated through hardware circuits, and as such, you have to work with fixed-width integers. Not only that, but those integers usually have other requirements around alignment and endianness.

We can. But it would be very inefficient. There's no point to use C at all, if you can accept that level of performance.

No, because that would require implicit dynamic allocation, which would defeat the entire point of using C.

Only when limits are getting breached, otherwise it is one if statement extra on every access. This happens in Java and other languages.

It doesn’t matter if it only happens occasionally. It’s completely inappropriate for C, which has no implicit memory allocation. There’s not even a clear way to include implicit allocation in the semantics of C.

>that can grow like in js

I thought js "integers" are just floating point numbers?

I believe the poster you were replying to was talking about JS's new `BigInt` type.

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Good example of that feature is Python. Also, AFAIR, Scheme (and probably some other lisps) support it.

Yes, arbitrary precision integers could theoretically become a C language/library feature. But let's say that it gets proposed and accepted and it's integrated into popular compiler toolchains. The kernel wouldn't leverage it here or likely anywhere else because of its cost.

A newly designed OS kernel could perhaps take on this kind of feature. This would be the kind of OS that could be formally verified and would be willing to pay that runtime cost for arbitrary precision.

Perhaps such type of things have been put in Kernel before.

C doesn’t have the abstraction power to make anything with such a datatype.

I left myself a lot of latitude when I referred to this as a potential "language/library feature".

    typedef struct { /* TODO */ } arb_int;
    arb_int *new_arb_int(void);
    void delete_arb_int(arb_int *);
    arb_int *arb_int_add(arb_int *, arb_int *);
    arb_int *arb_int_sub(arb_int *, arb_int *);
    arb_int *arb_int_mul(arb_int *, arb_int *);
    arb_int *arb_int_div(arb_int *, arb_int *);
    bool arb_int_gt(arb_int *, arb_int *);
    bool arb_int_lt(arb_int *, arb_int *);
    bool arb_int_gte(arb_int *, arb_int *);
    bool arb_int_lte(arb_int *, arb_int *);
    bool arb_int_eq(arb_int *, arb_int *);

Reply on news.ycombinator.com