You can have a full neovim experience with all sorts of modern extensions without using a single line of vimscript. Some people even replace their init (neovim's vimrc) with lua, but I am of the opinion that it is a step too far, as lua isn't particularly adapted to writing configuration files and the result is too verbose to my taste.
I use lua when examples are in lua or when I need some "logic" (such as assigning defaults to a var and then passing that around/overriding). And that's embedded in vimscript.
I don't really like either. VimScript has always been a horror to me, eventhough I've been using vim for some 20 years now, almost exclusively. Lua is "that thing that I should really sit down and learn. But not now, I've got stuff to finish".
What does lua -for an end user- offer that something like yaml+python cannot offer? I really won't mind setting all sorts of flags, defaults and vars from some init.yaml, and then have some init.py to handle the few places where I do need actual logic. Why was lua picked, why did vim build its own language and not move to an existing one for its config? Am I just weird for never sitting down and learning lua? Or vimscript? Or both?
Why Bram build is own language and then doubled down on it with vimscript9 I don't really understand.
[1] https://neovim.discourse.group/t/why-was-lua-chosen-for-neov...
What language would have you chosen in 1991 instead of creating vimscript? The vimscript language is a very natural extension[0] of Bill Joy's "ex" language, that was used in vi since 1976. The history of vimscript is not weird, it's just a fairly natural continuation of existing practices. Vimscript9 is a least-friction update for modern times.
[0] https://en.wikipedia.org/wiki/Vim_(text_editor)#Vim_script
According to your link scripting was only available from '98, at which point Lua and Python were already around, but then I don't know how viable that would have been. Funny you say that about ex, because I find running ex commands from vimscript the most clunky thing about it.
In my opinion the least friction for the ecosystem would have been to follow neovim and adapt Lua.