Hacker News Reader

Comment by WalterBright | Hacker News Reader

Here's my proposal for fixing buffer overflows in C:

https://www.digitalmars.com/articles/b44.html

It's simple and it works, we've got 15 years experience with this in D.

I mean, sure, fat pointers are the "right" solution, but the point of "low-fat" pointers in the original paper [1] is for greater compatibility with existing C code, including at the ABI level. Adding fat pointers to C (which is what your proposal is) is incompatible with those goals. Page 2 of the paper discusses this.

[1]: https://www.comp.nus.edu.sg/~gregory/papers/ndss17stack.pdf

My proposal does not impact existing C code, and does not make existing C pointers into phat pointers.

It's purely additive, which means you can incrementally adopt it in places where it matters.

It makes a subset of what are currently C pointers—arrays in argument position—into fat pointers.

The phat pointers only appear when they are specifically declared as phat pointers. Existing syntax does not change behavior.

Yes, and that's a laudable goal, but it's not the same problem as the one the "low-fat" pointers proposal is trying to solve. The "low-fat" pointers proposal attempts to improve the safety of existing code.

Is your complaint that because we don't have a perfect automatic solution we shouldn't bother offering any solutions, not even incremental opt-in ones?

No, pc is simply saying that the fat pointer solution is not in line with the goals of the solution presented in the paper.

I never said fat pointers shouldn't be added to C. The point is that they're irrelevant to this paper, which demonstrates a weakness in a mitigation designed to improve safety of existing code.

I still think that the Deputy's project (from UC Berkeley) approach (https://barnowl.org/research/pubs/07-hotos-linux.pdf) is the most practical for adding bounds checks to existing C code. Basically annotate functions and data structures with the, usually available, bounds information, e.g.:

  f(int *a, int n) -> f(int *count(n) a, int n);

  struct { int len; int *values; }

  struct { int len; int *count(len) values; }

And other annotations for unions, etc.

This actually works quite well in practice - the cited paper involved us applying these annotations to a complete, bootable linux kernel. You do have to be willing to tolerate limited code changes and a few "trust me" casts

Disclaimer: I worked on the overall project, though not significantly on the Deputy part.

Depends on if one would rather write:

    int s[3];
    int *p;
    f(s, sizeof(s)/sizeof(s[0]));
    f(p, n);

    f(int *count(n) a, int n);

(which has a mistake, n should be size_t not int), or:

    int s[3];
    int* p;
    int[] a;
    f(p[0..n]);   // convert pointer to phat pointer
    f(a);         // or use existing phat pointer
    f(s);         // static arrays just work

    f(int[] a);   // the declaration of f()

The point was not to design the "nicest" way of adding safe arrays to C, but a way that captures existing uses and preserves binary compatibility, including preserving memory layouts. For instance:

    uint16_t count;
    uint16_t *COUNT(count) buffer;
    uint16_t *BND(buffer, buffer + count) pos; // pos is within the specified bounds

which captures that buffer is a known-size array, and that pos ranges within buffer's contents. Of course, writing from scratch, one might prefer

  uint16_t[] buffer;
  uint16_t pos; // An offset within buffer

but that's not much help when incrementally modifying an existing code base, or interacting with some external library one either doesn't have source for or doesn't wish to modify.

> that's not much help when incrementally modifying an existing code base

I am doing exactly that now that I converted the Digital Mars C++ and the DMD D programming language compilers to D - converting the pointer based array code to [] based arrays. Doing it incrementally is very doable and easy.

It's not viral like const is. A slice can be trivially created:

    a[] = p[0 .. length];

and the other way:

    p = a.ptr;
    length = a.length;

I agree with the framing here, and I agree a small bit of syntax would go a long way.

FWIW, you will see a class called StringPiece copied all over a lot of open source Google projects, including Chrome, Ninja, RE2, probably Android, etc.

That is essentially a pointer-length pair. I agree it would be nice to have a dedicated syntax for it, and not a class that everybody has to copy around.

C++ 17 names this string_view (long overdue!)

I just Googled and apparently there was a problem with array_view ? That's a shame.

https://www.reddit.com/r/cpp/comments/5ya5pe/whatever_happen...

The trouble with a library type is the compiler is not aware of it and cannot make semantic use of it.

The compiler does this with other “library” functions like malloc and printf. There’s no reason a compiler couldn’t convert to a built in for this new type.

C doesn't have member functions and doesn't do implicit conversions to/from struct types.

That’s obvious and also completely unrelated to what I was saying.

The optimizer can convert a library type to a builtin type if it still behaves as specified in the standard.

C isn't powerful enough to have the library type behave as desired (with implicit conversions, etc.).

aye, if it is defined in the same standard, there is really no technical difference between a language feature and a library feature.

I mean, C++ kind of solves this issue with templates:

  template<size_t N>
  void foo(int (&array)[N]) {}

How does this fix it? It automatically includes a size_t with all array types but does it also rely on the programmer setting the size_t value correctly? If so there will be the same problems. It seems like this is exactly the same as just passing a size argument along with all array-type arguments, which is already common practice.

> does it also rely on the programmer setting the size_t value correctly?

For static arrays, no, the compiler does it. For turning pointers into phat pointers, yes, the user has to supply the length. But he only has to do that once.

> exactly the same

It isn't, for the reasons:

1. The compiler knows about it, so it can be used for optimization purposes, and to optionally insert bounds checking or not.

2. Consider why C has structs. It's because it's a lot less error prone to group related values together, rather than having a collection of independent variables. It makes the code easier to read, too.

I wish somebody could bring Walter into C standard committee

I remember when C added function prototypes. It was an unmitigated and welcome improvement. If I was to add two features with the most impact, simple to implement, and no backwards compatibility issues, it would be:

1. phat pointers for arrays

2. nested functions

(Nested functions have a surprising ability to untangle ugly control flow, including getting rid of most uses of goto's.)

They do not want to break single-pass compilers. Nested functions with no forward declarations will require a multi-pass compiler. I think your D compiler is multi-pass already.

> They do not want to break single-pass compilers. Nested functions with no forward declarations will require a multi-pass compiler.

Why is that exactly?

Nested functions are done in one pass. As you might infer, they cannot be forward referenced within the function.

Reply on news.ycombinator.com