A gentle introduction to pointers using C (2019)

> If pointers are so "simple", why are they so confusing?!? Because pointers are one of those concepts that's easier to use than to explain.

I disagree (the rest of your comment I fully agree with, FYI). What I personally found difficult is to understand that: when you dereference an invalid memory address, then the dereferencing operation is working, but the OS/runtime environment won't allow you to do it!

Wait, what? Why?! Who cares my value is wonky, why is my program giving an error? Just obtain the wonky value! (is what I thought when I first did it)

The reason it's so difficult because at this stage you're still using printf for debugging, while you really should use a debugger, to really understand what's going on under the hood. Preferably, one also learns a modicum of x86 to really see it immediately.

The concept of dereferencing the wrong values isn't explained a lot. How to troubleshoot segfaults isn't explained at all in beginner tutorials, not even in my own (I just wrote an explanation for fun as a comment in the same thread [1]).

Also FWIW, when I heard the concept of pointers I found them intuitive immediately. I tend to think and reflect a lot, and the explanation of referencing and dereferencing almost sound like some proverbial life wisdom to me.

I found their use a lot harder, in part because I found code syntax hard. In another part, because I used to find "doing things" and "building things" (of any kind) an insurmountable task.

---

With that said, I do agree with the rest of your explanation. I suppose it also depends on who teaches pointers to you. I got a classroom explanation first, a few examples and then some exercises. Then suddenly, I had to write a 500 line C program. Then all the complexity storms at you like a rabid zombie army.

But if you're used to writing small C programs all the time and picked up on pointers that way, but then suddenly have to explain it. Then, I can see how that's hard.

[1] https://news.ycombinator.com/item?id=23550040

> pointers are one of those concepts that's easier to use than to explain.

My experience is the exact opposite.

A pointer is just an address for something. There is a "reference" operator to get a variable's address (pointer) and a "dereference" operator to get the data at a given address. That's it.

The hard part for me has always been using them correctly - keeping track of what points to where in my head, especially for void pointers. I haven't done any significant development in C since my early programming years, but that's the part I remember struggling with.

They're sort of like GOTO in that they're easy for me to understand, but they can quickly make my code hard to follow.

> pointers are one of those concepts that's easier to use than to explain.

Strongly disagree. The rule of pointers is simple: an integer pointing to a memory position. However, using it correctly and effectively is hard. Dealing with pointers is like working with math. The rules are simple but there can be difficult problems coming out of these simple rules.

> easier to use than to explain

That is a nice explanation for what I noticed, which was that learning assembly before C meant that pointers were not mysterious to me at all.

Just lucky to do it in that order, I guess.

I agree. I really can't understand what is complicated about them. I find the same with programmers who find the DAG in git difficult to grasp. It's just linking stuff together. It's how the the web works. It's how linked lists work. What's difficult about it.

I think exposure to assembly demystifies all of this stuff about C. Most of C is just making stuff really convenient that you would do anyway in assembly. A pointer is just a memory address, but in C it also has a type. The only thing the type is for is so the compiler knows how many bytes to add/subtract when you increment/decrement the pointer. That's it. In assembly you'd just have to remember how many bytes to increment/decrement yourself.

Reading Knuth also helps with this. It's one of the reasons he uses a synthetic assembly language rather than a high-level language. There are no pointers. There is just memory, and you need some way to find stuff in memory, otherwise it is just noise.

Same here. After I read K&R C's memory model seemed pretty simple and straightforward. Whenever I have trouble figuring out something I usually draw a little diagram with the memory space, where things are allocated and how handles and pointers relate to the memory space.

When I got started I found the syntax of it confusing, like why doesn't de-referencing look more distinct kind of thing. I got used to it eventually but it didn't feel very clear at first.

I could read and write commodore 128 assembler before I started to learn C, so I knew about adresses, adressing modes and how to use them. Despite that, I still had trouble wrapping my head around the concept of pointers at first.

There are multiple things that can make it confusing, and they kinda collude to make it harder to get.

In order to understand them at all, you have to know

- what a variable represents (a region of memory)

- what an adress/reference is (the index of said region in memory)

- how a value is stored in and read from said variable (by putting it into or reading from the memory beginnen at this adress)

- that you actually can treat an adress as a value, i.e store it in another variable

Thats already four deeply related, but still different things you have to wrap your head around.

To make matters worse, the way pointers often are used (and most of the time introduced), is to point to another variable. Now you have two adresses, and two values, one of wich is one of the adresses. If that sentence confuses you, that is how beginners feel about it as well!

Dereferencing is not straightforward either: It makes a pointer behave as if it were the pointed to variable, which in the usual introductory toy examples makes it look like a totally unnecessary and useless operation.

Also the size of the pointed to variable is encoded via the pointers type, and does not have to match the size of the actually pointed to thing. Another mindblower that one.

Lastly there is the linguistic problem that we tend to say a variable IS its value. Which is actually misleading in multiple ways if a variable contains the adress of another one.

Ultimately I had to understand what pointers are NOT, in order to finally grasp what they are.

Your version isn't very true though. Most operating systems have virtual memory.

You need a bit more than that if people are going to understand you... what is a "position", and what does "next position" mean?

Wait till they start learning about C++. A notion of ‘reference’ there that is separate from that of ‘pointer‘ confuses things even more.

> A pointer is an integer that indicates a position in

RAM, ROM, NVRAM, flash, etc.

A pointer stores a number that represents an address, without making reference where is located. Can be 0 (NULL).

"p+1" can be the next byte in RAM, or the next 4KB if p is of type "4KB struct".

Trouble is pointers don't have to be integers. The standard lets you address by fruit and veggies if you have such a machine.

That ends up being misleading though as it ignores optimizing compilers. A pointer isn't simply an integer address of memory. It's tagged (by the compiler) with information about e.g. which allocation unit it came from and what extent it has (i.e. how far you can move the pointer forwards or backwards).

The long and short of it is that doing `p+1` can be UB. Also two pointers can point to the same actual memory address but not be equal.

Pointers are, in practice, a complex abstraction. It can be hard to understand all the subtleties, especially if you're used to the simplicity of memory addresses in assembly.

And that's the way they're shown in K&R2

I agree that it is confusing. Here [1] is something that can help new C programmers to understand these kind of declarations. It gets easier after a while.

[1]. http://c-faq.com/decl/spiral.anderson.html

Yes, I think the confusing syntax of types is one of the actually bad things in the C programming language.

True, but you do not need to know assembler to have a general idea of what computer memory looks like; but more importantly, C's notion of a pointer is more complex as it is tied to the data type and its 'sizeof' and is further complicated by association with arrays.

To quote K & R "In C, there is a strong relationship between pointers and arrays, strong enough that pointers and arrays should be discussed simultaneously" ... however they also say "Pointers have been lumped with the goto statement as a marvelous way to create impossible to understand programs". I use both these quotes in my lectures.

> The C language has more than one way of accessing arrays. Other languages don't.

Like this?

    procedure ArrayDemo;
    var
        data : array [1..20] of Integer;
        ptr : ^Integer;
        i : Integer;

    begin
        ptr := @data[low(data)];
        for i:=low(data) to high(data) do
        begin
            WriteLn(ptr^);
            ptr := ptr + Sizeof(Integer)
        end
    end;

There is no magic in .h files, you can think them as plaintext. The actual magic happens in .c files. Since C requires objects to be declared before using them, you need to declare objects used in multiple units together. Let's say you have a function `int f(int)`. You want to use this function both in apple.c and pear.c. How would you do it. Well you need to write a f.h such as

    int f(int);

then in apple.c

    #include "f.h"
    int apple() {
        int x = f(10);
        // rest of the code
    }

You can think this as a copy paste of f.h such that, apple.c is equivalent to:

    int f(int);
    int apple() {
        int x = f(10);
        // rest of the code
    }

but what "is" f. To "define" f you need another .cpp file that implements f, let's call it f.c:

    #include "f.h"
    int f(int x) {
        return x*x;
    }

now you compile f.c, apple.c, pear.c. These will create f.o, apple.o and pear.o object files. Then you link these together to create your application. e.g. if f.h is in include/ dir:

    gcc -c apple.c -I include/ -o apple.o

    gcc main.c -I include/ apple.o pear.o f.o -o main

Here, assume that main.c defines int main().

Imagine you have a function f in f.c:

    void f() {
        puts("hello, world!");
    }

and you want to call it in main.c:

    int main() {
        f();
    }

this is an error because the compiler treats each .c file separately, so when it’s compiling main.c, it isn’t looking at f.c, so it has no way to know the function f exists. So you need to “declare” it at the top of main.c. A declaration is similar to a definition, except it has no code, and just tells the compiler that the function exists:

    void f();

But now imagine f.c had many functions. It would be extremely cumbersome to repeat all those declarations in every file that wants to use them. So you can collect all the declarations in some file f.h, and include it from main.c or anywhere else you want to use functions from f.c. When the compiler sees #include "f.h", it operates as if the contents of f.h were copied and pasted at that point.

You might ask “why not just include f.c from main.c instead?” Because then you would have two _definitions_ of f in your program (one in f.c and one in main.c), which is illegal, as opposed to two declarations, which is fine.

Another good trick for seeing what headers (and the pre-processor) does in general is using the -E flag (for gcc / clang) it will show the file with the headers included so you can see what happens. This is really useful when using macros and include guards to see what you are getting. clang -E myfile.c > pre.txt will give you an editable file to see what is going on.

Perhaps that's because C pointers are usually (always?) exactly that - a hard coded char array, which starts at memory offset 0.

Depending on the circumstances, it may be easier to think about it as int array, or some other type, but the easiest on-ramp for pointers is still char[] = 0x0.

> how to e.g. read strings from say a CSV file and parse it into an array of arrays of strings > > Does anyone have any pointers here? (sorry)

I think this kind of thing isn't discussed too often because there are so many different choices that you can make (and once you understand the language & your constraints, you can certainly make those choices). I agree it would be useful to hilight all the possibilities. Let me try to illuminate:

First, do you really need an array of arrays of strings? That's possible, but not all that common. If you don't really need such a thing, then you do something like strtok or strsep: implement a function that you call in a loop to extract fields one by one. Now you just need a single buffer large enough to hold a single field (or one row if you prefer to make it a little simpler). Your function locates the terminating comma, replaces it with a NUL, and returns a pointer to the start of the buffer (and stores the start of the next field -- that is, the address of the byte after the NUL -- somewhere for the next call).

This approach is popular among C programmers because it usually means you don't need dynamic allocation, as long as there's a reasonable maximum field size. Or if a maximum field size cannot be imposed, then you get away with dynamic allocation (and re-allocation, when needed to grow) of a single buffer.

What next? Ok, maybe you really do require "all fields at once." Your average C programmer still won't allocate arrays of arrays of strings. Instead, they read the entire row, extract fields out of it (as above), and store the pointers to the start of each field somewhere. If the caller knows how many fields they want to deal with (very often you do!), then they can provide you a fixed length array for these pointers. Otherwise, you can dynamically allocate a buffer to hold the pointers. Either way, now you need two buffers (and depending on your needs, one or both or neither may be statically sized): one for the row, one for the pointers. Now the buffer containing pointers is exactly like argv in main(), while the fields are in one contiguous buffer spliced by NUL bytes.

Ok, maybe you really do require all rows and all fields at once. At this point C programmers will hate you because you're forcing them into dynamic allocation (or an unreasonable fixed size limit on the file). Otherwise, you do as above, but you also terminate lines with NULs and now you have some choices as to how to lay out your row + field pointers. For example, you could allocate one array for row pointers, which point to field pointers (which could be allocated separately for each row, or, perhaps preferraby, in a single flat array containing pointers to all the fields). If you expect the caller to iterate (rather than random-access) rows, you could use a single mixed type array containing row metadata + field pointers.

Another very common approach would be to allocate field pointers + data for each row separately and then link rows in a linked list. Again this favors iteration but you're less likely to need to resize huge arrays.

Sometimes you deal with memory that is best to keep immutable; in that case, splitting the data into rows and fields with NUL bytes is not an option and you must take a different approach. Either duplicate the data (more mallocs, required if you also need to do things like unescaping) or keep it where it is and store pointer + length tuples.

See, there are lots of choices, and lots of variables, depending on how much memory you want to use, can you use fixed size buffers, what kind of access pattern the caller expects, does the caller expect to be able to free individual rows, etcetra.

There is no best practice but generally C programmers gravitate towards using iteration & fixed size buffers unless more is required (this is simple and lets the caller persist data if they want to, but doesn't force them to deal with dynamic memory if they don't want or need to). If something more specific is required, then you'll know what is required and implement something that supports just that.

The Python style divide-and-conquer where you first read an entire file, then chop it into lines, then chop lines into columns, etc. is not very popular.

It is easy to believe that you understand something while in reality things are more complicated. What is an address? Is it like an integer value? (Is it really? Or can it be converted to one? Even if so, how many bits wide is it?) Or, if not, what is its structure? Back in the day, a question would be is it a 'far' or a 'near' pointer or we are living in a flat address space? Also, how is pointer arithmetic affected by alignment?

> In those 3 examples, the * is not a dereference operator, it simply indicates that it is a pointer

This is one of the confusing parts for beginners, but in actuality, C declarations are meant to mirror the use of the variable. In that sense, you can think of the asterisk in the declaration as the same unary operator:

    int *a; // dereferencing "a" gives you type "int"

This is a really good explanation. You understand what's going on with pointers.

This is similar to the idea that a number isn't always a number. A number can be categorical, in which case comparison operators don't make any sense. They can be a label or a name, in which case addition or other mathematical operators don't make any sense. The same way a number can be an address, which has its own sets of operations that operate on the idea of an address.

As an example, statistics has the idea of ordinal values vs interval values, where they have different properties that make them not comparable to each other and shouldn't be mistakenly be confused when working on them.

I would recommend working with https://godbolt.org instead of gdb. It abstracts away the unnecessary stuff and highlights lines corresponding to asm from source and vice versa quite conveniently

Useful link: Use cdecl to explain confusing-looking statements: https://cdecl.org/?q=++int+*****super_pointer

I'd rethink the this part of the explanation:

And this is how many times we're allowed to use the dereference operator: one time

I can see what you meant, but I initially thought this was trying to say that a pointer declared with a single asterisk could only be dereferenced once and never dereferenced again afterwards (as if there was a "one-time-use only" property to pointers). In the same vein, the "super_pointer" example could be misinterpreted like "Oh, so if I want a pointer that will let me dereference it on 5 different occasions, then I'll need to declare it with 5 asterisks?" I think a better way would be to replace the "how many times we're allowed to use the dereference operator" explanation and start with explaining the idea of a double pointer

  int **doublePtr

in depth first. With a solid understanding of a double pointer (an address that points to a memory location that contains an address that points to another memory location), making the mental jump to understand

  int *****super_pointer

should be easier.

Also, I think low addresses aren't used for various reasons, but I believe the kernel is stored in the top, high-address region of the address space. Otherwise, I think this is a really solid explanation of pointers!

> And this is how many times we're allowed to use the dereference operator: one time

Then you come across

  extern int *(*f)();
  int *g = f(0), h = *(******f)(*g);

> that would make the char pointer the 'first char in the array' then?

That would make the char pointer to point to the 'first char in the array'.

It's long been the convention here, and it's useful for people to know how recently it's been published.

With an article title like this, it could easily be anything from 1 to 30 or more years old, and showing the date makes it easier for readers to know if they've already seen it before clicking through.