Let the callers of your API control allocations
dave.cheney.net
dave.cheney.net
For example a lot of C libs will have an API along these lines:
typedef void* my_api_t;
int create_api (my_api_t* api); // almost certainly allocates
typedef struct {
// yada yada yada
} api_result_t;
int get_result(my_api_t api, api_result_t* result); // does this allocate? I have no idea
int get_result(my_api_t api, api_result_t **result);
...
api_result_t *result;
get_result(api, &result);
typedef api_result_t *API_RESULT;
...
API_RESULT res;
&res; // is this a single or a double pointer?
res; // is this a pointer at all?
typedef struct {
void* some_data_pointer;
} api_result_t;
int get_result(my_api_t api, api_result_t* result) {
result->some_data_pointer = realloc(foo_size + MAGIC_VALUE);
// ...
}Yea that's what I meant. But this sort of gets into the weirdness of docs, if there is a `free_result` somewhere then authors should make sure to put a "see also free_result()" in their doc comments, especially when you don't have the niceties of constructors/destructors.
If it isn't - and especially if you're dealing with a closed source library - you have to assume the worst and even then you're often unpleasantly suprised :/
Effectively what we have here is code allocating memory and handing it to you for care and feeding.
It's not unlike an unwanted gift. For some people this is an important and aggravating dynamic in their lives. I'm sure I've heard people complain about some aunt who "forces" plants or god forbid pets onto hapless relatives.
I almost always write pool based memory allocators for my projects (CG, games), so I typically log that info myself. What would other folks use to profile that info?
https://www.agnosticdev.com/blog-entry/ios/profiling-memory-...
As a nice bonus, it can also look for memory leaks by running what is essentially a conservative garbage collector on your heap and reporting unreachable allocations.
"Let the callers of your API control allocations"
vs
"Don’t force allocations on the callers of your API"
There are another couple of benefits, though:
1. A call which allocates can necessarily fail, although in garbage-collected languages like Golang, memory allocation failure are normally handled implicitly with something like panic(). (That's because the main benefit of garbage collection is not that you don't have to debug segfaults, but that you can treat arbitrary-sized data structures as if they were atomic integers — with immutability, even when they share structure. This allows you to program at a higher level. You lose this benefit if you have to know which calls allocate and which do not.)
1½. By the same token, dynamic allocation almost never takes a deterministic or bounded amount of runtime, though see below about per-frame heaps.
2. In many cases, you can do the allocation on the stack rather than the heap, with potentially substantial implications for garbage-collection efficiency. (You could see the MLKit's region allocation as an automatic version of this pattern.)
3. In environments like C++, it may make sense to use a different custom allocator, like the per-frame heap commonly used in game engines, or the pool allocator provided by the APR, or an allocator that allocates from a memory-mapped object database like ObjectStore. The alternative is something like the rarely-used allocator template parameter that makes all our STL compile errors so terrible. Even in ordinary C programs, it might make sense to use a static allocation for these things some of the time.
On the other hand, putting the allocation on the caller ossifies the amount of space needed as part of your API, and prevents that space from being dynamically determined at runtime by the algorithm. So there are times when letting the caller allocate all the space is undesirable or even impossible. And, as Cheney points out, it does make your API more of a hassle to invoke — but that's easily fixed with a wrapper function that does the allocation.
A third alternative, used for example by Numpy and realloc(), is the one mentioned in https://news.ycombinator.com/item?id=20888920 (though it erroneously implies that C has optional parameters) — have an optional buffer argument telling where to store the result, but if it's not passed in, dynamically allocate a buffer and return it. Taking advantage of this parameter to avoid extra allocations and cache misses commonly doubles the speed of Numpy code, in my experience, but it really hurts readability. As a general API design technique, this seems like the most constraining to future implementations, and I think it makes more sense to separate the underlying stable non-allocating API from the allocating helper wrapper, as described above.
I believe the (poorly conveyed) implication was that the parameter was nullable.
From a systems point of view, the fact that modern crypto doesn't require heap allocations makes it one of the easiest domains to work with. (The maths are exacting, but the good APIs are a joy to use).
(A little example posted by Bartlomiej Filipek showing a vector neatly using a stack buffer)
#include <iostream>
#include <memory_resource>
#include <vector>
int main() {
char buffer[64] = {};
std::fill_n(std::begin(buffer), std::size(buffer)-1, '_');
std::cout << buffer << '\n';
std::pmr::monotonic_buffer_resource pool{
std::data(buffer), std::size(buffer)
};
std::pmr::vector<char> vec{&pool};
for (char ch='a'; ch <= 'z'; ++ch)
vec.push_back(ch);
std::cout << buffer << '\n';
}It is important to have the option to deviate from “objects” in projects that can require significant memory. I would not allow objects to wrap absolutely everything, being passed all around the system, to the point that it’s impossible to control the biggest users of memory.
As a simple example, imagine an object that strictly encapsulates storage of “64-bit int plus bool”. At a small scale, the wasted space is an implementation detail but for a few million objects you have a pretty ridiculous memory footprint. A fancy encapsulated object probably gives no option to avoid this, such as “put your array of million 64-bit values over here, and your buckets for bits over there”. When this happens enough times, you have holes all over the place and it’s almost too late (where do you start to fix it?). That is also the type of design that can consume shockingly more memory when moving from 32-bit to 64-bit, e.g. some class that had moderate holes with 32-bit pointers has bigger holes with 64-bit pointers.
The justification is largely the same, i.e. that the caller generally has more context and thus better ability to decide when and how side-effects should happen.
Because it's letting the callers control side-effects! :)
It would indeed be possible in that scenario to return the buffer, and to return the error in a subsequent call, but this seems to change its behaviour into one that (to me) is surprisingly different.
The same is true for writing; write(2) can write some, but not all of the passed in data, and then err. What kind of API would one want tosee here?
If OS "writes some bytes and then errors" - then for the current call write(2) will return the number of bytes written, and then on the next call you'll get the returned error.
In C it's almost always Length | error_code
[0] Passing buffers around is much less necessary in Lisp than in C-like languages. This is about the times when it still makes sense to do so.
>First they must record the number of bytes read into the buffer, reslice the buffer, process that data, and only then, consult the error.
ObjC is very verbose partly because it went with the "signature encoding" approach, which has its own tradeoffs, but I do appreciate the API consistency.
https://developer.apple.com/library/archive/documentation/Co...
foo * bar(foo * useme=0);
where one can optionally pass memory to be used, but if nothing is passed, it will be allocated by the function (placement new can be helpful here!).
In C++, a better thing would be to take advantage of overloading directly:
foo *bar(foo *); // doesn't allocate; non-null required
foo *bar(); // allocates
foo *bar() { return bar(new Foo); }
// Definition
foo * bar(foo * useme);
// Allocate foo for me pls.
foo * result = bar(NULL)
// I'll allocate the buffer myself ktkxbye.
foo * result = bar(malloc(…))You are over-nitpicking. It's obvious that the GP meant that you can pass NULL by default if needed.
But the way the overloads are collapsed into a single function, there has to be a run-time check for that null.
Using a null to indicate "argument is not there" is a C API trick. That parameter won't be optional in C.
If you're especially unlucky, that garbage data will happen to be NULL at first, but then become non-NULL when you recompile with debugging turned off, resulting in truly fascinating ways to spend your Friday and Saturday nights the month before shipping.
(I know you know this, Kaz. I'm mentioning this for the benefit of those who haven't yet spent their Friday and Saturday nights this way.)
void foo(void* p=0); // has exactly the same ABI as
void bar(void* p);
bar(); // error: missing parameter
foo(); // shorthand for foo(0);
// void(*)(void*) is a different type than void(*)(void*=0)
// but they can be implicitly converted the same as
// eg void* and char*The pattern is more useful when there are more arguments to foo and you don't want to have bar(int a, int b, double c, foo * useme) and bar(int a, int b, double c) in the API.
double * doMathOperation(unsigned N, const double * in, double aux1, double aux2, double * out = 0);
By nature of being a buffer means it can probably be used in subsequent calls. You dont need to read everything all in one go
This type of findings are pretty much the daily routines of engineers a decade or longer ago...
Moreover, modern CS education teaches a lot of things that weren't taught in the past, and it likely only gives a hand-wave to things like memory-management etiquette in low-level C (or Go) API design. This is likely because (1) it's not that difficult to read up on the specifics if you're otherwise a competent engineer and (2) other concepts were valued higher. Perhaps the course focused on writing code with borrow-checking in mind instead.
For the most part, it was the other way around, we mostly learned abstraction, so the lessons in class focused on how to encapsulate and hide state. At a higher level than that, education really focused (and still does, I think) on engineering for functionality over engineering for performance.
In some sense, the problem with APIs that allocate is larger now than it ever was before, and growing larger. Data sizes are bigger now, and memory pressure relative to what the processor can do is going up.
At the application layer, I really don't care to utilize a partially filled buffer, I want a result set that has what was read. Of course, nobody is being stopped from creating such things as abstractions. StreamReader for C# as an example.