Lua's biggest thing is that it's pretty easy to embed inside a C/C++ application safely, for relative definitions of "safe" spanning "preventing the Lua code from segfaulting your application" to "preventing the Lua code from doing anything you don't explicitly give it the ability to do".

It's why you see it used as so many game's plugin architecture and why it was selected for NeoVim.

So there is a very large ecosystem of Lua code that just lives in Lua 5.1 and doesn't really push forward at all. I haven't checked in on the situation for some time, but most people who write Lua are writing Lua for something like NeoVim or WoW or some other plugin for a larger project. Those projects have no interest in dealing with the slow reference VM for Lua >=5.2 at this time.

Lua 5.2, 5.3, and 5.4 are more analogous to Python 2 to Python 3's level of change, rather than Python 3.9 to Python 3.10. These more recent versions of Lua 5.x are somewhat incompatible with each other, rather than just adding new language features.

Most products settle on Lua 5.1 because that is what luajit, a very fast just in time compiling implementation of the Lua VM supports. Plans to further support to new versions for luajit are...well they exist.

I can tell you from my own experience that luajit, which also targets 5.1 compatibility, is _significantly_ faster than Lua proper. In using it I've never missed 5.2+ features. So I imagine luau has the same thing going for it.

Lua 5, unlike other foobars, is a complete language from the start. You may see 5.x as 5.a, 5.b, 5.c and so on rather than as “evolution” versions. The second digit is more like flavor, and as e.g. 5.2 or LuaJIT user you don’t miss 5.4 features. Moreover, some people, me included, find some of 5.2, 5.3 and 5.4 experiments questionable.

Why is it so? Usually, when a language unlike Lua “evolves”, it just lacks a flipping lot of core/meta features from the beginning, crutch-adding them later. So users long for 1.x+ version so much, because 1.x is still bullshit, for any x. In case of Lua, its 5.1 version had some flaws, but it already had (arguably) more powerful execution/data model-wise than e.g. python or js, and these flaws were related to what the latter two haven’t had at all (because to them Lua is Lisp, in a sense of blub paradox). Coroutines, environments, metamethods, etc. 5.2 solved all of the remaining problems one way, LuaJIT another way. Upgrading to >5.2 doesn’t really change much in a way it works, but it’s enough to break your existing codebase.

So, “especially now” doesn’t have much weight in this case.

Type annotations are the main benefit. It's also much faster than Lua 5.1, but I don't know how it compares to 5.4.