You can but it's definitely not easy to do correctly.
There's also a lack of traits that can describe what's required of a specialized allocator. For example, std::map only needs to allocate 1 object at a time in practice while std::vector needs to allocate a contiguous count of objects.