The article mentions the address sanitizer keeps track of allocations and their sizes. I wrote my own allocator, how do I integrate it with the sanitizer? I added malloc attributes to all allocation functions but they don't seem to do anything.
In those functions, you need to implement the sanitizer yourself, and have it panic if it detects an invalid access (address sanitizers are usually implemented with shadow memory). You'll have to give your sanitizer enough shadow memory during initialization, and also add your own alloc/free sanitizer callbacks to your allocator (so the sanitizer knows when things are allocated/freed).
I have an example [1] that implements a basic ASAN and UBSAN in a kernel (it's written in D, but could be easily adapted to C or C++). Hopefully it's helpful!
[1]: https://github.com/zyedidia/multiplix/blob/master/kernel/san...
https://mcuoneclipse.com/2021/05/31/finding-memory-bugs-with...
https://interrupt.memfault.com/blog/ubsan-trap
I've tested the UBSan, it worked with some limitations but helped me catch some old hidden bugs.