From Python to Lua: Why We Switched
distelli.com
distelli.com
- Python 2.4? Really? In 2014?
- Yes, writing compatible code with a 10+ year old release of Python is going to be hard. Who uses 2.4 still? You have to use the lowest common denominator, I mean I don't think Python 2.4 even supports context managers!
- Yes, in 2014 you should be supporting Python 3, or at least have it on your roadmap.
- The ethernet address (MAC?) via uuid.get_node(). Or for something a bit more advanced you can use netifaces[1] (or even better psutil.net_if_addrs[3]). One of Pythons core strengths is the number and variety of the packages available. Took 2 seconds of googling to find a cross platform solution.
- Calling C from Python is really really simple. Use the built in ctypes library, or something better like cffi[2]
I'm obviously firmly in the Python camp so perhaps I'm a bit biased, but I don't see any clearcut reason to switch other than "the current code is not optimal, let's re-write it in X", where X could be anything. I would actually say Go would be a much better fit than Lua in this situation. I guess the memory reduction/CPU usage is a point, but really your program seems to be network orientated. What's wrong with asyncio? How is replacing everything with a C library actually better?
1. https://pypi.python.org/pypi/netifaces
2. https://cffi.readthedocs.org/en/latest/overview.html#simple-...
My solution was to use pyinstaller[2] to basically ship my own python, first 2.7, and now 3.5.
[0] https://access.redhat.com/support/policy/updates/errata/#Lif...
Then create a folder for your project, 'virtualenv -p /usr/bin/python3.5 venv' to create the venv in that directory. Activate/deactivate the environment as you wish when you want go between the system Python environment and the one in your folder.
I'm unsure how what I said:
> If you want another Python, then you need to install it yourself (somehow) and then point virtualenv at it.
Differs from what you are saying:
> 'virtualenv -p /usr/bin/python3.5 venv'
I know there's the argument that by straying you don't get security updates, but I can't really imagine Python 2.4 getting much security updates nowadays.
It's trivial when you have a handful of machines, all under your control. It's somewhat less trivial when you are dealing with 1000s of machines under the control of dozens of different people/departments.
I still use a few RHEL5 machines but nobody would ever dream of pushing new software to them.
Telling customers that they are doing things wrong isn't really a great way to win customers.
Why everyone is relying on python that's installed there for RedHat's needs (e.g. yum).
Unlike other languages, Python is probably the best one that can have multiple versions coexist together.
RHEL 5 is really common.
Environments where the build of your server is strictly controlled and conservative...banks, mission critical apps, that kinda thing where stability from upstream is hugely important.
To my mind, it's usually a good idea not to use system Python, both packages and the interpreter, if you're not also controlling them. This makes building your deployables more involved, but makes the actual deployment more painless.
How did the switch to Lua not break this in the same way?
Sorry for the naive question, I'm just not following how it can be possible to change languages, but impossible to change versions. At first glance, the latter seems like a special case of the former.
(EDIT: LUA->Lua)
Deleted comment
2) Deployments with the new agent were noticeably faster, (like 1 second deployments that used to take 20+ seconds). However, this speed up was largely due to the Python codes design which had a "continuation" loop which was polling based, but the Lua code used coroutines and simply continued the threads when the steps were complete.
Overall, I think the choice to use Lua (or more specifically luvi) was a great decision. Note that it did take a few iterations in order to come up with the optimal way of using Lua + libuv. Originally I was taking the Node.JS approach of using callbacks, but that approach has two problems:
[a] It is difficult to properly implement "back pressure" so that all queues between producer and consumer are bounded.
[b] It is difficult to handle errors properly.
Lua coroutines allowed me to write a simple "green thread" library that encapsulated these challenges.
That was your problem, you just made a huge mistake and depending on how big your python code base was, you have now alienated most of your developers by moving to a language they are not already comfortable with.
Do your customers already have lua installed on their servers or are you bundling it now? You could have just a easily bundled a newer python interpreter. Lua is a nice scripting language, if you want to embed it into another app or to create a plugin system. It's not a nice general purpose scripting language, mostly because of the vast pypi repo.
The same thing could have been achieved within Python, but from an end user perspective I'm guessing this is an improvement for most of their customers.
The cost of effort their developers will have to put in to be as effective as a LUA developer as they were as a python developer is really not worth this move. There are more to moving to a different platform/language than just the end users.
Being as the size of the team is so dependent on a small number of high performers, I don't think it's unusual that they picked a language one of the main people in the company is extremely fluent in.
I've worked with Brian before and he is a beast. I don't know if 10x engineers exist, but if they do I would certainly consider him one.
I don't think what you say is completely true. People who are more experienced tend to be able to produce features much more quickly than those who are junior, just because they've already run into issues before so can anticipate them or solve them more quickly. If anything I would argue people like that actually write code more quickly that is more maintainable.
I don't know if he can ship 10x as many features; I didn't think term 10x engineer was used literally. I just know I've worked with quite a number of people and he is one of the best developers I know.
But then, we'd need to know how good your assessment skill/method is. Maybe you misjudge all the developers you work with? What feedback do you get?
@Chris2048 - Distelli is hiring. If you want to make your own assessment of my skill, then join us :).
Are you in a position to speak for Distelli developers?
"The cost of effort their developers will have to put in to be as effective as a LUA developer as they were as a python developer is really not worth this move."
I suppose this could be true in a poisonous environment where scheduling does not accommodate the need to learn a new skillset.
Unless you are familiar with the employer this is an incredibly hostile presumption. I.E. I would not work for such an employer nor would I want to be their customer
There will still be a minimum effort (e.g. in hours) that cannot be reduced by the nature of the employer. A dev will still have to put in the hours to learn a new language.
It seems the cost being described here is cost to the employer, not to the developer.
I also need to mention that Lua is the easiest language I've ever encountered, so any competent developer can learn it quickly, it's not alienating anyone.
Given that, I agree that LuaRocks isn't so vast as other languages' repositories and this can be a drawback of using Lua. But why again are you judging a language by it's package manager? If it's a good language wouldn't it be better to suggest more people to write packages instead of suggesting people not to use it?
I read it more as "relying on a runtime that must be installed on the user's OS instead of included in our executable is a nightmare". The dramatically reduced conceptual footprint of the language is a nice bonus though.
* Some companies have absolutely no in-house tech staff to speak of, and so they want to invest in technology exactly once and never have to deal with it again.
* Some companies have in-house tech people, but are entirely marketing/sales driven with little or no input from the tech staff. So they focus exclusively on writing new features and never clean up technical debt.
* Some companies choose a platform to bet the farm on, develop a huge amount of in-house code for it, and then realize that since they wrote it for specifically the exact version of the platform they first installed, any upgrade involves a complete refactor/rewrite, which they then dismiss as too expensive.
The first case can be solved somewhat by subscription-model managed services where the upgrades just happen at the discretion of whoever's managing the platform.
The other two are failures of management and planning, and absolutely should be laid to rest at the feet of the management staff involved. Preferably in an expensive way, pour encourager les autres.
What companies often learn too late is that there are basically two options:
1. Make maintenance -- both in terms of upgrading underlying software, and addressing accumulated technical debt -- a regular, required part of their process, or
2. Occasionally face a potential existential threat from the unbounded cost of staying fixed in perpetuity to a stack that "already worked" once upon a time but no longer does due to lack of upstream support and available developers to keep it limping along.
The genuine problem is that "what does this do for next quarter's results" is the only concern, which means long-term thinking is actively discouraged.
python35 will be soon be coming to EPEL also.
RHEL4 = Python 2.4 RHEL5 = Python 2.5 RHEL6 = Python 2.6 RHEL7 = Python 2.7
netifaces: ...would require me to build a C extension which would mean going down the rabbit hole of building and bundling python and this library.
Why? What's wrong with 2.7?
They wanted something easy to deploy (single executable), easily portable, very small CPU footprint, and very small memory footprint.
Lua is well regarded for all these things, and the company succeeded in meeting all these objectives.
Congratulations Distelli. Nice job. And thanks for sharing.
It may be true that they meet their stated objectives, but some of those objectives could have been achieved without a rewrite, and some of those objectives don't make sense.
I've written plenty of single executables for multiple targets in Python. Figuring out how to do that would certainly have taken less time than the rewrite.
Small resource usage may make sense, but they didn't relate it to any actual customer need, and given it was already running on RHEL, I doubt this was a reasonable goal. Unless it also needed to run on pocket calculators, the resource needs of a Python program are well within reason for most target environments.
To be clear, I don't think Lua is a worse choice than Python for a new project; my objection is to the rewrite. There are problems with both languages. Sure, now packaging is easier, but when he runs into a hard problem that would have been solved by an existing Python library is he going to rewrite again?
It's a junior developer mistake to blame issues on your tools and rewrite instead of getting things to work in your existing environment. Mature tools like Python (or even to a lesser extent Lua) are rarely the problem.
There are certainly exceptions where there are serious problems such that changing tools makes sense. But none of the problems mentioned in the writeup are that.
The bad part of the rewrite is that there is no possible way to know what was lost. There are edge cases which were discovered and handled in the old product that will have to be rediscovered in the new one.
It sounds like they haven't lost any business over this, so it will be okay in the long run. But I suspect they could have done this a lot easier and cheaper. Just because it turned out okay doesn't mean this is an exemplary choice we should learn from.
Had the title been "Why we Chose to use Lua" it probably would have garnered more positive comments.
Something like http://roscidus.com/blog/blog/2013/06/09/choosing-a-python-r... where the author actually considers multiple candidates and performs a controlled comparison would be far more valuable. Given that the goals sound very similar I'd be very interested to see a comparison between Lua and OCaml.
You end up with something that is superficially better, but misses a lot of edge cases that the ugly legacy application covered. Fortunately, you have usually moved on to save other projects, leaving lowly maintenance developers to fix the problems, thereby creating job security. It's a win win situation, really.
1. I started the company as a single founder and wrote the first version of the agent. See this HN post that gave me the encouragement to keep going (thanks HN!) - https://news.ycombinator.com/item?id=6059481.
2. I wrote the original agent in python 2.6. It was great because I could use things like python-requests (which was awesome!). However my first encounter with an enterprise customer showed me that was not feasible because they were running only python 2.4 and would not be upgrading for another 2 years.
3. I had the choice of either not getting the customer or back porting the code to Python 2.4. I chose to back port the code to python 2.4 and get the customer. This was a lot of work but worth it because I was able to then raise a round with a16z. - https://www.distelli.com/blog/distelli-series-a-funding
4. When Brian joined Distelli, he actually worked on the Python code for many months and made many improvements and added many features. When it was time to add the streaming logs feature he built the prototype in Lua just to prove that the idea would work and then we talked extensively to figure the direction we should go in.
5. We decided to move to Lua because Brian was very familiar with it and it was a lightweight language and we could ship a single download with zero dependencies. We also looked at using Go. At the end of the day the decision was Brian's to make and own as the senior engineer on the project. The passage of time has convinced me that was the right decision.
I'm sure we could have stayed with Python with all the suggestions on this page, but we decided to go with Lua. It was a good fit for what we needed and honestly I felt that my original code was not in great shape and would need a lot of work to refactor and a rewrite felt like the right decision at the time. Looking back I still think it was the right decision and I'm very happy with the end result.
edit: grammar
I think your company made a good decision. The only reason not to have done it was because your existing code was valuable, but as you've said it wasn't in great shape and required tons of work to refactor and rewriting anyway.
[1] Lua has LuaRocks, but it's just not the same.
* variable name typos (undeclared vars are nil),
* accidentally confusing array-style tables and hash-style tables (or vice versa),
* tables with "holes" (nils) in them,
* multiple assignment with wrong number of vals or vars (mismatch silently ignored),
* variables are global by default.
Have you found this to be the case at all? Is it more difficult to avoid bugs when writing in Lua?
Good questions.
- variable name typos (undeclared vars are nil),
There's a linter to catch undeclared variables. https://github.com/mpeterv/luacheck There's also some runtime checking techniques you can use. http://www.lua.org/pil/14.2.html
- accidentally confusing array-style tables and hash-style tables (or vice versa),
I've had that happen about as often as confusing Python dictionaries returned by different types of functions. (Which is - not often enough for me to remember the last time that happened)
- tables with "holes" (nils) in them,
When I'm building a vector and I want holes, I use false to fill the holes.
- multiple assignment with wrong number of vals or vars (mismatch silently ignored),
For me, when I'm thinking about multiple assignment to function returns, it does mean having to read documentation for a library function more carefully than if I was reading the same for another language. If it's as simple as "local a, b, c = 1, 2, 3", then it's no issue.
- variables are global by default.
If I'm using linting, when I can't have global variables implicitly, then it's like C or Java where I must declared all my variables and by default they are local to the scope.
Otherwise it's just like javascript where it's best practice to always declare variables using "var".
Lua, in and of itself, didn't solve the problems. Shipping their own, known, Lua interpreter + extensions solved them. (which they could have done with Python)
For example my local Python venv is about 130 MB. It symlinks in another 52 MB of standard Python libraries. And I know it won't run without some unknown amount of additional dependencies in C-libraries. If I wanted to send that to someone I'm starting at 200 MB. After compression that might be 40 MB or so. You're going to have to work to strip that down.
By comparison their download including Lua, code, and libraries is under 6 MB.
I have a minimal running Python application that was started recently and has done nothing. It takes close to 90 MB of RAM just to start.
Theirs is generally under 10 MB while it is running.
So at every step Python requires 5-10 times as much data. If you're trying to get other people to put you in their containers, this can matter a lot.
A minimal Python app on my machine takes 6MB RAM.
There are use cases where 200mb of disk space matters, but for the overwhelming majority of applications, a 5% (say) increase in developer productivity would be well worth adding 200mb to the install.
And did you try spending a person-week or so of dev time trying to minimize the size of the Python version? That would be a much smaller one-off cost than the costs of migrating everything to Lua, and I could easily imagine you'd get it down by one or two orders of magnitude. (You could do the same for Lua too I'm sure, but if we're talking about say 5mb vs 200k then that's "who cares?" size for most customers).
From where do you get this impression that Python is the only high productivity language in the world? There are plenty of languages that can compete with Python in terms of productivity and a lot of them offer better abstractions for it. Lua is known for being one of the best escape hatches for scripting.
My opinion is that Python is more productive than Lua, but if you disagree by all means use Lua. But if you're talking about saving 200mb of disk space as though it makes the difference, then I think you're using the wrong criteria for choosing your language.
> I ended up introducing incompatibilities with older versions of Python when I tried using a finally block with an exception block—that syntax was not available until version 2.6
Lua changes major parts of the language in minor revisions. 5.1, 5.2, and 5.3 are not compatible, and in much larger ways than Python 2.6 and 2.7. Lua 5.1 is currently the most recent version supported by LuaJIT which he specifically mentions. Lua 5.3 introduces integers!
> For example, I needed to add support for detecting the Ethernet addresses of the host computer, which can be done with the netifaces native library (or shell out to ifconfig and parse the results, which would not be compatible with Windows).
You're going to have exactly the same problem with Lua, except that that library won't exist so you'll have to write it.
Lua's great, but this isn't a case of Lua being better than Python for your use case. It's a case of you redoing your deployment in such a way that you happen to have more control.
The Lua creators are very careful and deliberate in how they break compatibility.
One such thing they do is they avoid subtle changes that can lead to incorrect/different results, and prefer hard things that lead to obvious compile time errors.
But most of the time, the changes between minor revisions aren't a big deal. On the Lua side, authors have been pretty good about figuring out how to make their scripts run across all versions because the changes haven't been that big. On the C side, the API changes can be a little more dramatic, but authors have been good about handling the differences.
For example I wrote many games in Lua 5.1, that actually work in Lua 5 if I wanted to, and also work in Lua 5.2, despite 5.2 not existing when I wrote them.
I don't checked yet the changes on 5.3, but I suspect my games might work on 5.3 without changes too.
Plus me (the author of the Lua agent) was already familiar with Lua, so the easiest way to get from A to B was for me to use a known-good tool.
I've only ever done game development in Lua and have never considered it for anything else honestly. Very cool use case here.
I don't think they could have created a statically-linked Python binary that contained the entire Python installation and core libraries, and the entire application in less than 5 MB. The entire app in a self-contained executable. No futzing with PYTHONPATH, LD_LIBRARY_PATH, virtualenv, installation, conflicting with the system-installed Python, etc. That's what Lua let them do.
But don't take it from me. Here is Guido van Rossum last year:
> The final question was about what he hates in Python. "Anything to do with package distribution", he answered immediately. There are problems with version skew and dependencies that just make for an "endless mess". He dreads it when a colleague comes to him with a "simple Python question". Half the time it is some kind of import path problem and there is no easy solution to offer.
You can also zip an app (think .jar) since 2.6, with the zippap module available in 3.5.
I haven't seen any of problems mentioned in the blog post.
I'd like to add, that blog can be dismissed outright. It's also out of date in regards to PyInstaller. It does support Python3 now. 2.7;3.3-3.5. I've used PyInstaller for years and prefer it over the other packaging solutions, it's really good.
I wouldn't be digging up old blogs as a source for dirt and relying on any information within without verifying it was or still is true.
At a very high level luvit looks like node.js for Lua, with some extra goodies thrown in like being able to bake your apps into single binaries. This is awesome in itself and the article discusses these things.
That said, I think the major innovation over using javascript is that it allows the support for coroutine-based I/O. This means you can get the performance benefits of non-blocking node.js without the headache of callbacks/promises. I think the linked article only touches on coroutines briefly but they are a major piece of Lua's awesomeness that you shouldn't miss.
Though you aren't required to use coroutines, the package management server which hosts packages for luvit is called "lit" and it seems to demonstrate this ability nicely:
"Lit is written in lua and uses the same system platform as the luvit project, but is has a very different I/O style. Instead of using callbacks and event emitters, it uses coroutines so that code can be written with simple blocking I/O, but still maintain the abilities of a non-blocking event loop." (from the README on https://github.com/luvit/lit)
There's Lapis, by leafo http://leafo.net And I'm also improving Nginx experience on the framework I develop, Sailor, because it was originally developed for Apache http://sailorproject.org
And outside of scientific computing, python doesn't offer any real edge, to me at least. Ruby has better tooling libraries, java/go/c++ have better performance, JS has more powerful website frameworks.
Why should I bother with that packaging mess after having it done once already?
I'm Lua fanboy, but I've to admit, that I'm missing clean deployments and upgrades with virtualenv and pypi available in Pyhton. I've never had any issue so far, am I being just lucky bastard? It has been like that for many years already.
> Ruby has better tooling libraries
On the other hand, I've honest to maintain one Redmine instance (which is written in Ruby) and this is real nightmare for me. Each time I've to upgrade, I've to drink a lot day or two in advance :-)
Well that is your problem right there - don't drink in advance, drink afterwards to try delete the traumatic experience from your memory!
My solution was to docker all the things. It made my life so much better; you could simultaneously run multiple services on the same box without awful fragile hacks.
There's a few python libraries out there which just crash the entire stack, like Python-Mysql (or was it PyMysql, or one of the other very similarly named libraries?). And I might be biased, but talking to a mysql-db shouldn't be hard.
>On the other hand, I've honest to maintain one Redmine instance (which is written in Ruby) and this is real nightmare for me. Each time I've to upgrade, I've to drink a lot day or two in advance :-)
I rather mean tools like rake as a task-runner, guard, rubocop, thor as a templating system, rspec, chef as config management, kitchen, food-critic, sinatra for easy web services. It's a whole bunch of stable tools and tooling libraries (and they are integrating with each other).
When are dynamic languages finally going to learn from these problems and fix them? Smalltalkers were making these exact complaints to me in person in the 90's and I know they predated my involvement by a lot.
What if someone created a Smalltalk-like uber-debugging-godlike-power-over-the-runtime alternative compiler target for Golang? Then you'd have all the good parts of dynamic languages with the dead-simple distribution model of Golang.
Packaging and deployment aren't problems in Go OR Python if you're only solving trivial problems.
With a larger project, you might have an easier time packaging and deploying Go code, but that's largely because there are no libraries to package. Admittedly packaging can be a pain in Python, but that's usually because of poor choices in dependencies. Packaging Python with a few mature dependencies isn't hard in my experience. It's the projects where some idiot has pulled in every 0.x versioned library in pip that are hard to package. When Go has as wide a variety of libraries as Python people will run into the same problems in Go.
You could argue that at least for now Go doesn't allow you to shoot yourself in the foot that way, but I'd rather have the option.
I'm not defending Python in particular here. I'd say the same things I've said here about any mature language with extensive libraries.
If you're espousing Go because of easy packaging and deployment, I strongly suggest that you consider whether that's actually a feature Go will have long term, and whether you're currently paying for that feature by having no libraries available to you. The only lesson I would take away from Go's easy packaging is to only use mature dependencies that pull their weight.
I stand by what I said: give it a decade and Go packaging will be just as miserable as packaging in any other language.
Not that I don't lua. It's like the best parts of javascript, mixed with some AWK and scheme, with all the crap removed. OTOH, from-1 indexing is evil.
Yet, I still admire the Lua creators for making this bold choice. Why should we use from-0 indexing for all eternity? Just because it's always been that way? This is a real question if you have the answer.
Well, we obviously can use 1-based indexing. Ada/VHDL allows you to define the ranges of your indicies even beyond 0 or 1 based (very useful in fixed point mathematics where you sometimes want to index your bits by a negative power of 2).
The big disadvantage is that you lose the algebraic properties of mapping to the ring of integers modulo N when your indicies are 1-based.
This leads to a bunch of off-by-one bugs in various places. One of the most pernicious is that while you can get to the next index with "next = prev % N + 1" you actually have to do "prev = (next - 1) % N". Note that the previous REQUIRES the parentheses due to precedence of operations.
0-based may be convention, but it isn't ill thought out. FORTRAN(1-based) and C(0-based) coexisted a LONG time. Pascal was(is?) 1-based. Most people had experience with both.
People who designed programming languages CHOSE the 0 convention. Overwhelmingly.
Here is the answer: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/E...
Lack of arity checks is definitely much more problematic than this, but still... if you can count the number of tragic design flaws in your language on one hand, you're in great shape compared to the rest of what's out there!
This is a straw man. You're not forced to index[1]. You can create tables that index[0], no worries.
$ lua
Lua 5.2.4 Copyright (C) 1994-2015 Lua.org, PUC-Rio
> x = {}
> x[0] = 1
> =#x
0
> =x[0]
1
> x[1] = 2
> =#x
1
>
$ node
> x = []
[]
> x[-1] = 0
0
> x[0] = 1
1
> x
[ 1, '-1': 0 ]
> x.length
1
I agree that 1-based indexing isn't as much of a downside as people make it out to be, but saying you can ignore it and index from 0 "no worries" is just being silly.
> x = {}
> x[0] = 2
> x[1] = 3
> for i, v in ipairs(x) do print(i, v) end
1 3
> x[2] = 4
> for i, v in ipairs(x) do print(i, v) end
1 3
2 4Cars don't require you to drive on the right side of the road either (or whatever side your nationality prefers), but you're going to have a bad time in traffic if you try to switch.
Lua standard libraries do not suffer this problem. If you have a legacy / cross-platform issue about it, well then .. its a good thing one is 'wise enough to know array[0] is a thing', but its hardly relevant to the question of whether this 'inferior accessing of arrays in Lua' is anything more than fallacy.
Idiomatic Lua involves intelligent use of pairs/ipairs/table index schema, and metatables if needed, to solve the data-access problem. And guess what? A competent Lua developer either has had zero problems with this issue (because they are competent and know how to wire up a C/C++ data structure to the Lua vm), or they have infinite, multiple "problems with Lua", of their own devising, nevertheless, but hey .. you can't "blame Lua" for that. Lua gives you the keys to the universe. If you crash into someones mailbox, well then ..
- http://nuitka.net/pages/overview.html
- https://code.google.com/archive/p/shedskin/
It would result in a similar binary, but save the full code rewrite.
For example. In numpy:
x = numpy.array([1,2,3,4])
y = 1/x
x = torch.Tensor({1,2,3,4})
y = x:apply(function(n) return 1/n end)
It sounds like the author just wanted to move it to Lua to me. I would only go with Lua over Python if you needed something more embeddable. Like the game scripting usecase it's known for. The lost libraries and language features may make some things harder. I package up Python and my code, deploy it to other platforms all the time with under 5MB footprint in a single executable using PyInstaller.
That said, it'll work out fine with Lua. My question would be who will maintain something years down the road. There's far more Python devs than Lua. The blog said he joined in 2014 so it may end up being someone else.
But you're right, I don't think Lua is unique in being able to fix this issue. I think it's more that the author was already very good with it and liked it.
[a] Here is the complete syntax of Lua:
http://www.lua.org/manual/5.2/manual.html#9
...compare that to Perl where there is no "complete syntax" of Perl. This shows how simple it is and thus it is easier to write tools for and teach others how to use it.
[b] The complete default API consists of ~200 functions:
http://www.lua.org/manual/5.2/contents.html#index
[c] The defacto book on learning the complete language (including the C interface) is 366 pages:
http://www.amazon.com/dp/859037985X
Simplicity is beauty IMHO. I would expect any competent developer to get up to speed on the Lua language within a week, it would then take them another couple of weeks to get up to speed on the libraries we use with Lua.
The real power in Lua comes when you couple it with competent APIs that do the "real" work. The Distelli Agent uses libuv for the OS interface, libcrypto (part of openssl) for the crypto primitives, zlib for compression, LPeg for parsing, sqlite for an embedded DB, etc.
The job of the Lua code is simply to "tie" these core components together.
I can't speak to the "Lua standard library" but I can say that I believe every single statement above about the POSIX C standard is incorrect as written.
POSIX Sockets: http://man7.org/linux/man-pages/man2/socket.2.html
POSIX Signals: http://man7.org/linux/man-pages/man7/signal.7.html, http://pubs.opengroup.org/onlinepubs/009695399/functions/sig...
POSIX Timers: http://www.2net.co.uk/tutorial/periodic_threads, http://pubs.opengroup.org/onlinepubs/009695399/functions/tim...
Nonblocking IO: http://man7.org/linux/man-pages/man7/aio.7.html
That doesn't mean that the POSIX features are best in class or easy to use -- rather just that the features exist.
Thanks!
That said, anyone who has written Lua knows that scaling is difficult and relies on really high quality code. I'd be wary about replacing python scripts with Lua, especially if the problem with python was users wanted more advanced features. Still, my use case isn't yours and I'm glad the migration went well.
Since that was a long time ago, can anyone chime in about the state of unicode support in Lua?
http://www.lua.org/manual/5.3/manual.html#6.5
Full unicode support is rarely necessary in my experience. Lua "strings" are raw byte arrays. It is the job of the programmer to validate the byte arrays are in the character set of your choosing and doing appropriate conversions and validation when necessary.
One can always use library functions to process the binary string as unicode.
language preferences aside, sometimes it's all about the right tool for the job.
* not literally _all_ of them, but lots and lots and lots.
It's a life saver.