I'd replace the underlying data structure used to represent buffers (https://www.gnu.org/software/emacs/manual/html_node/elisp/Bu...) with something more sophisticated, like ropes or piece tables.
https://coredumped.dev/2023/08/09/text-showdown-gap-buffers-...