RMS's dislike of Common Lisp really caused a colossal amount of damage. Imagine an Emacs written in Common Lisp instead of Emacs Lisp, with decades and decades of improvements. Having written a reasonable amount of software in both, Common Lisp Emacs would be very, very preferable, in part for the reasons you list and in part for others (e.g. a full-fledged, built in object system and improved extensibility).
If Emacs were written in Common Lisp, only people on big mainframes would have been able to use it in 1985, and it probably wouldn't have become popular in the first place. Also, implementing Common Lisp would have taken much longer. RMS has said that his main goal with his LISP dialect was to keep the programming language as small and performant as possible, since at that time, machines had maybe 1MB of RAM (if you were lucky) and no virtual memory. This is why for instance he decided against lexical scoping (and still, Emacs was known back then as "eight megabytes and constantly swapping").
[0]: https://github.com/lokke-org/lokke [1]: https://gitlab.com/python-on-guile/python-on-guile
[1] https://nyxt.atlas.engineer/article/why-building-nyxt-instea...