Avoid NULL
foundbit.com
foundbit.com
Generally speaking there is really only one case that you need to think about: what do you do when a function needs to return a value and it has no value to return. This often happens in error conditions, but it can also happen as a result of poorly designed code (for example the ubiquitous "doUpdate()" function that sometimes returns a result and sometimes doesn't, based on what was passed to it). I will maintain that initialization is a special case of this (I haven't initialized it yet, so I have no value to put in it).
However, more important than what you use to represent "I don't have a value" values, is what you do with it if you get one. In several of the examples in this article, a boolean is returned to indicated whether or not you have valid data in the parameters passed by value. This may seem "safe" because you don't crash, but there are often much more serious problems than crashing: data corruption, for example.
Let's consider the case where I gather some data from somewhere and write it to my database. In the case that my data collection failed, I certainly don't want to write random data that was returned in the parameter. What happens if the calling code neglects to check the boolean return value? This check is identical (and could literally be identical) to checking for NULL. The only difference is that if I try to an uninitialized, but valid data structure to my database, it will work. If I try to dereference NULL, it will crash. In many cases, crashing is preferred.
Even when you return a NULL Object, you still need to think very hard about what you want to do with it. What happens if someone tries to write a NULL Object to the database? We know things are very, very wrong. We could just skip it, but many times it would be prudent to halt execution of the program.
Too often I deal with programmers who blindly accept the advice that NULL is evil without actually understanding the problem that NULL is intended to solve. I have no problem with avoiding NULL (and in many cases it is a very good idea). It is important, though, to understand that you still have a lot to do to write robust code.
NULL is just a tool. It's just a way for you to express some property of your API.
That said, some of the strategies outlined in the article could be very useful for avoiding use of pointers. Use of a class type implementing an interface as a "null type" is a very good strategy. Such a type could guarantee successful instantiation, and so avoid the need to throw exceptions or otherwise crash entirely. You could architect an application with those strategies and completely avoid the use of NULL and nullptr entirely.
But where your program is dependent on a finite resource, you must have a way to represent the absence of that resource. NULL and nullptr are convenient ways to do that without a ton of extra code. So no, don't avoid NULL.
It's been more than a decade since I did any C/C++, so I wonder if you were referring so something else that I'm not aware of.
But modern hardware is frequently designed so that dereferencing a zero pointer causes a crash. Even embedded systems will often treat the first couple hundred bytes or so after 0 specially, since it catches so many bugs.
I think it's generally safe to assume you'll get a crash if you actually dereference a null pointer.
I do agree that there's no fundamental difference between using a "might-be-valid pointer" vs. a sum-type (Option/Maybe/etc.) to indicate nullability. However, it's usually easier to get static analysis of the sum type—most type-inferencing compilers can ensure that when you destructure a sum type, you provide match-clauses for every possible instantiation of the type (i.e., for an Option type, they ensure you do the equivalent of a NULL check, and handle both the NULL and non-NULL cases), and so forth.
What I'd really like to see, though, is a language with data-parameterized "types" that unfold (as a last resort, when the concrete type cannot be inferred at compile-time) into runtime data-comparison branches. You should be able to have a raw may-be-valid address with no metadata tag, and then treat that data as having a separate type (a Just or a Nothing) depending on whether the memory address itself is nonzero. You write match clauses; the compiler turns those into NULL checks.
(As an aside, such a language would also allow for things like a type for "Float that only has an integer component", or "Even number", or "Positive number", or "Number in the range 5..23", or "String of length 5", or "String that encodes a valid Unicode codepoint sequence", etc. The type system would become your swiss-army knife for expressing invariants.)
In Rust, the compiler optimizes pointer types wrapped in Options to nullable pointers. Is this the kind of thing you meant?
That's sort of what I mean, here: the nullable ptr as the "canonical, machine-level" representation, with the type serving as a language-level abstraction, effectively a "lie" because it has no real effect on the type system, and only serves as syntax sugar for generating runtime checks.
Depending on the type of semantics you are attaching to the pointer, you can use something like this[1] function, which does exactly what you're asking. It is a runtime no-op, but it coerces the unsafe pointer type so that memory safe code can use it (based on your word that it can be interpreted in this way).
[1]: https://doc.rust-lang.org/std/primitive.pointer.html#method....
Such an FFI function declaration really can't require that the thing returned inside the Option be a dereference of the pointer: in the happy case, it's still an opaque FFI handle, so dereferencing it just exposes an opaque struct you shouldn't touch; and in the unhappy case, it's still an invalid pointer, just one that's invalid because somebody else (like the library that gave it to you) freed it. The raw pointer itself, even if non-NULL, is still "tainted"; you still want to apply manual runtime checks to it before casting it to a Rust reference. (And, past that, in both cases you still probably want to deal with the pointer mostly just by passing it back to other FFI functions from the same library, treating the pointer itself as an identifier, a key the API associates with something.)
I'm emphasizing this distinction because the NULLness of the pointer is, from what I can see, "outside" the lifetime of the reference, and even outside the "identifier-ness" of the handle. Knowing the validity of the pointer requires knowing about ownership/lifetime; the pointer might point to memory that has been free()ed. But you can know that the pointer is intentionally NULL (and therefore not a valid handle/identifier from the FFI's perspective) without attempting to construct a reference.
Effectively, a [nullable] raw pointer is the moral equivalent of a tagged data structure. There's a piece of metadata you can get from the tag: when it's zero, the contained raw pointer is intentionally invalid; when it's nonzero, the contained raw pointer is only maybe-invalid. You can destructure and act on the "tag" without doing anything else to the [non-nullable] raw pointer "stored inside".
I wouldn't expect most languages to care about a distinction this fine, because if a language has references, it really only wants to think about "pointers: maybe invalid" and "references: always valid". But Rust is in the unique position of trying to work "directly with C" without the impedance mismatch of glue like SWIG. And C provides many libraries that return nominally opaque handles, but expect their consumers to notice if those handles are NULL (which happens often, when e.g. a factory-function from such a library is called with NULL arguments, or unexpectedly receives a NULL from a library it itself consumes.)
The function I linked takes a possibly null raw pointer, and vouches that:
1. It points to null or something. 2. The object it points to (if it is not null) will live at least as long as the current scope. 3. The object it points to will not be mutated as long as the current scope is active.
Practically speaking, as long as you don't do anything with the resulting Option<&T> other than check if it is empty, you're safe. In addition, Rust will preserve the internal pointer address, so you could bring it back to the FFI. So you could treat it consistently as such, you'll be fine. I'm not sure if the language spec guarantees that, because that's not how Rust references are designed to be used, but it won't matter in practice.
For what you want, you could just create a generic structure that holds a pointer to the given type, and lets you ask if the contained pointer is null. That way you won't add any false guarantees to the pointer (e.g. immutability, lifetime, etc.).
> Use of a class type implementing an interface as a "null type" is a very good strategy.
Have you ever programmed Objective-C? It uses the null object pattern (called "nil") for all Objective-C objects. This makes it really easy to put big logic errors into your code, since calling upon a nil will just give you more nil. So now you have to wait even longer until a fatal error pops up, and then find where on earth the original nil came from.
> NULL and nullptr are convenient ways to do that without a ton of extra code.
How do you figure? If malloc can return NULL and you want a sane error message, you need to write a NULL check whenever you do a malloc. Otherwise, your user will just get a segfault, instead of a notification that it was memory they ran out of. So why not do that by default, and for the few places where you have prepared yourself to handle out of memory conditions use a different malloc which returns a value that must be explicitly checked?
NULL breaks the type system. Now every pointer type is actually two types: the normal type, and the NULL type. You can dereference or call methods on one, but not the other, and only a runtime check will tell you what you're dealing with. Any time you see:
Object *
This causes three big problems. One is that it's way too easy to forget to check for NULL. The second, related problem is that without a way to express a non-nullable pointer type, you have to rely on documentation and convention to express the difference. The third is that since only pointer types are nullable, you have to reinvent the wheel any time you you want to express the concept of "null" for a non-pointer type.
To illustrate, you point out malloc which returns NULL when it runs out of address space. What's the appropriate return type?
void *
void *
void f(void *ptr)
NULL is "just a tool" but it's not a very good tool. It breaks the type system in a fundamental and painful way.
Edit: whew, HN's markup makes it kind of hard to talk about C pointers!
Part of programming is about memory management. You can hide it with a sophisticated compiler but it's always going to be there, and there are always going to be two types of memory: that which is available, and that which is not.
In the context of C and C++, dereferencing a null pointer does not guarantee a crash, or anything else for that matter. In fact, not only does it not give you any guarantees, it nullifies any guarantees you might otherwise have had, because the behaviour of a program that dereferences a null pointer is undefined. Now, an implementation is of course free to make guarantees about programs that have undefined behaviour, but none I know of does.
>Part of programming is about memory management. You can hide it with a sophisticated compiler but it's always going to be there, and there are always going to be two types of memory: that which is available, and that which is not.
This is not true. It is perfectly possible to write useful programs that never have to deal with unavailable memory or even memory management at all past compile time, even in C. This is quite common (as are implementations that don't crash programs which dereference null pointers) when working with embedded systems, for example.
Here's another potential problem. Let's say you write a little bit of code to copy an array into some dynamic storage, basically strdup for non-strings:
void *memdup(const void *source, size_t length) {
void *destination = malloc(length);
memcpy(destination, source, length);
return destination;
}
if(destination == NULL) {
log_failure();
abort();
}
Using an optional type would do away with all of these problems. It would guarantee that dereferencing a null pointer would actually halt execution, rather than just "probably," and it would highly encourage distinguishing between "an error occurred" and "no error occurred, but you didn't actually ask for any memory."
Yes, the underlying NULL value will always be there. Any reasonable system will represent the "null" value of an optional pointer type using the underlying NULL value. But in terms of the type system, NULL is terrible and optionals are much better. There's no real downside to it, and you can rest easy knowing that you're still working with 0x0 in the end.
Null-checks only make sense when you expect a null value. If you don't, and you get one, then either your code should be expecting one and has to do some processing differently, or something else is broken.
If the answer is "no," what happens if you pass NULL anyway? The compiler certainly won't catch it, and who knows what will happen.
In all likelihood, a crash when it tries to do something with whatever the pointer should be pointing at, in which case you should be asking why your code is giving it a null.
Beware programming errors which reveal themselves as "in all likelihood," because that hides an entire universe of difficult-to-debug behavior, and sometimes behavior that can be subverted.
Optionals solve these problems, and solve them far better than the "be careful not to do that" approach you describe here.
But option types do the same thing. The option type is actually two types. You can use one, but no the other, and only a runtime check will tell you what you're dealing with.
It may be more convenient than dealing with NULL, but it's doing the same thing...
If you have an interface which returns a pointer that's never NULL, you still have to return a nullable pointer. The non-nullability of the value can only be encoded in documentation or naming conventions. Because of this, the language has to make it easy to use pointers without first checking for NULL, otherwise all these interfaces would become a giant pain in the ass to use. And that means that when you're working with an interface which can return NULL, you have nothing that forces you to check for that. You just have to remember. It's similar for function parameters.
Optional types also work for non-pointer types, which C's NULL simply cannot do. That's a major plus, beyond the problem of all pointers being nullable.
You're right that an optional type is actually two types. But it's designed such that you can't get at the underlying runtime type without first checking to see what it is. That's a huge difference.
It's also error numbers. Yeah, they're just an int as far as types go, but they also have nullable semantics, with 0 being "no error". So it's not just pointers (quite).
> You're right that an optional type is actually two types. But it's designed such that you can't get at the underlying runtime type without first checking to see what it is. That's a huge difference.
Agreed.
If you want an example of a non-pointer nullable type, float and double probably qualify. Both can be NAN, which is effectively NULL for floats. Like NULL, NAN behaves completely differently from any other value and only convention and documentation tell you whether any particular place might produce or accept it.
not_null gives you the illusion of creating compile time invariants based on dynamic values. But that's impossible. You can still pass a nullptr to a not_null parameter indirectly which will assert at runtime. It just prevents a user from passing nullptr directly as a function argument. It provides no advantage to pass by reference. If you pass a null reference (you gotta go out of your way to do that), it will fail at runtime just like not_null. not_null does provide a consistent way of doing assert(value != nullptr) but an assert is clearer imo, not_null hides the assert away from the programmer and adds ambiguity. You can forget to assert just like you can forget to specify a parameter as not_null.
int *x = nullptr;
int **value = &x;
func(*value);
If the program state cannot proceed when one of the functions is passed a nullptr then assert in the function and make it clear, otherwise check for nullptr condition and proceed accordingly. You can forget to do that just like you can forget to increment a value.
Both resources below say something opposite.
http://bit.ly/1SEnJQe - see optional<T>::optional()
and http://bit.ly/1lh7eyv - see default constructor
(I'm just curious)
Edit: I've tested it, you are wrong. Default constructor of optional doesn't call Student constructor (VS2015).
program will fail in run-time. Sometimes it's not an acceptable situation, sometimes it just makes debugging other problems more complicated.
I disagree. If I were to make a list of all the bugs I had to find over the years, sorted by difficulty, a nullpo (usually caused by something else) would be near the bottom. They're quite obvious precisely because of the crash that occurs when they're used, and you can usually easily trace back to the origin of that value.
Seeing all these clumsy "solutions" that increase complexity just to avoid something so simple is both sad and somewhat amusing. It's a bit cargo-culty. E.g. if you use the "Null Object Pattern", with the example given you still have to presumably check whether the returned value is an instance of the null object or not... so not only do you still have to do the same thing you'd have to do if you'd just used a simple 0, you now also have to make an additional class and figure out how to compare with it. It's obfuscatory.
I think avoiding "NULL" makes as much sense as avoiding the number 0 - i.e., none, null, \0, nada, zilch. ;-)
In all the things I've debugged that's almost never been the actual error, but rather a symptom of something else (running out of memory, failed file opening, etc.) that needs further investigation. In my experience the "check for null, skip processing and continue if a value is null" way is a far worse option since it causes errors to propagate further, whereas a crash is an unmistakable call to action.
But null is a useful concept in SQL.
Embrace NULLs in outer joins.
Let's say that I have a bunch of contact information, from multiple sources, to synthesize. Sources include guest lists, registration forms, notes from agents, and other hand written information. Some of the information is illegible, pages are ripped and damaged, etc.
So let's consider email addresses. In some of our sources, there's no place to enter them, but we know that most people have at least one. The appropriate value for those email addresses is "NULL". That's also appropriate for email addresses in other sources that are illegible.
Conversely, in reliable sources, missing email address means either that there is none, or that it shouldn't be used. In those cases, it would be appropriate to leave the column empty, or use "NONE", or even "nobody@dev.null".
That's how I learned to handle "NULL", anyway.
"Easy to handle" means you don't have to manually interpret the values every time you deal with them. That's the default! That's the hardest a thing gets to handle...
You're talking about an innately fragile process. This algorithm is not general, and it's not generalizable, and thus it represents an upper limit on the effectiveness of our ability to write good software.
You've got a method for solving multiple problems here; you gave me one example. Now give me an implementation of that algorithm, and you'll find it doesn't bother to employ nulls at all. You're just mapping other concepts onto null because that's how you imagine a database to work. You're familiar with a hammer, so you're employing nails all the time.
>Conversely, in reliable sources, missing email address means either that there is none, or that it shouldn't be used. In those cases, it would be appropriate to leave the column empty, or use "NONE", or even "nobody@dev.null".
A properly normalized database would put emails in another table and reference them using a key. You'll have no empty columns. You'll know there is no email because there is no value, not because there's a NULL value. If you want to send emails, you do a join on these two tables and get a list of people who have emails. This is the proper way to model such a domain.
In the mean time, you'll just have a fragile model. Then you'll use that fragile model to make fragile decisions, and your database will be completely useless in 40 years.
(Granted, your DB implementation might combine these tables as an optimization, but then the optimizer has a clean notion of what NULL means across your entire schema, and you shouldn't be exposed to it at all.)
A table with 2 columns (key, value) is logically equivalent to a nullable column in the parent table. There are times when a separate table is a good solution. But there are also times, for example if you have a lot of optional columns that cannot be grouped into sets, that such a thing would be massively over-complicated, space wasting, and poor performing.
NULL means I couldn't store a valid value in that column. Why? Well that depends on the system, just like the contents of every other column.
>NULL means I couldn't store a value in that column. Why? Well that depends on the system, just like the contents of every other column.
No, it means you stored a NULL value in that column. Why you did that rather than throwing an error or generating a more informative value is a mystery. And it is not made clearer by changing the name of your column. And someone writing code to handle that data can't know how to handle it, because they have no clue what went wrong, or even if it is wrong.
Why would you think it's automatically an error or that a more informative value could exist?
I might have a form with yes/no or date answers and if a question isn't required and the user doesn't provide an answer then it's stored as NULL. Either way the meaning is obvious, it's read and written perfectly fine, and it's not an error.
I don't understand the implication that storing NULL means something went wrong? That's not at all what it means to me. I'm going to assume something about your response: you seem to conflating the concept of stored null value in database vs. returning a null reference or pointer in C.
There's no way to programmatically determine which of these cases it is.
>I don't understand the implication that storing NULL means something went wrong?
It doesn't mean something went wrong. It also doesn't mean something went right. It means nothing by itself.
NULL simply has no consistent semantics in a DB model. Its only semantic purpose is to make joins consistent, which often has nothing to do with your problem domain. That's why higher levels of normalization tend toward fewer uses of NULL.
Imagine you were using a database to log processes on a supercomputer. You log access to the machine whenever a process starts, and you write a NULL value to the end_time field for the process. What does NULL mean, here? Does it mean the process is still running? Or does it mean the process failed to write it's end state (for instance, if the machine crashed?) Does it mean the process is non-terminating? If you read this value from the database, would it be correct to wait for the process to finish?
You'd have to go and look at what processes are running. Which means this field no longer models the end time of the process, it models some dirty hybrid thing that you have to manually verify every time you read a NULL value. That's the problem with NULLs: they are bad at modeling your domain. It's a big middle finger from your data telling you to verify it yourself. It is one of those things that makes computers stupid, because you can't get past it without involving a human brain. (Which would make it very hard to make a computer learn how to build database schemas.)
There is no need to programmatically determine which case it is. You're way over thinking the problem.
> NULL simply has no consistent semantics in a DB model.
Yes, it's the lack of a value. If you have an end_time field and you don't have an end_time and column doesn't allow NULLs, what do you put in there? It's a simple question. If you put any kind of valid value in there (which you have to do) then you'll be wrong.
> Or does it mean the process failed to write it's end state (for instance, if the machine crashed?)
You sure as hell better be using transactions to maintain consistency. This has nothing to do with the problem. If you do it this way that's just wrong. Has nothing to do with nulls. If you had a perfectly non-null FLAG you'd have the same problem.
> Which means this field no longer models the end time of the process
While the process is running it doesn't have an end_time, so what do you put in this field?
Thank you for saying the dumbest thing I've heard all week. This conversation is over.
God dammit why do we use this? Oh right, because every attempt to "fix" it throws out the relational algebra baby with the sql bathwater.
I understand why SQL chooses to treat null as unknown for comparison, but I hate it with every fibre of my being and wish SQL dialects would join the 21st century and let me define alternate concepts of nullity like we can with modern type systems.
If you have (1,2,3,4,NULL) as data in column foo, and you query
select foo from tbl WHERE foo!=1
2
3
4
Redshift (and I think Postgres generally) will also, very confusingly if you're not aware of it, return no rows if your subquery returns any NULL values in a NOT IN scenario.
If bar is (1,2,3,4,5)
SELECT * FROM bar WHERE foo NOT IN (SELECT foo FROM tabl)
But are you saying that my query doesn't resolve the NULL issue? I know that I've done just that with tables containing NULL values. Maybe I screwed up the format.
SELECT * FROM bar WHERE foo NOT IN (SELECT foo FROM tabl WHERE foo IS NOT NULL)
SELECT * FROM activation_code
WHERE code NOT IN (
SELECT activation_code_used FROM user
);