ChaiScript – Easy to Use Scripting for C++
chaiscript.com
chaiscript.com
Why not just use Lua, one of JavaScript engines, Python, etc? They all have C++ bindings, all relatively easy to use. All are already used in many C++ apps for this exact purpose... (I guess it's not commonly known in all coder circles, but all those languages can just as easily be embedded in a C++ codebase as this.)
In my case, it is Wren. Some people prefer Chai, AngelScript, Lua, mruby, etc.
Wren has exception-like error handling and concurrency. And the object model is very similar to C#.
EDIT: Sorry, I wrote this a little too quickly and ended up writing a bit too coarse of a description. I meant JS and Python are both heavyweight to include in a project, and Python is slow both to initialize and to run. I didn't mean JS is slow to run. I don't know how fast it starts up either.
Also then, if this would be the selling point of ChaiScript, I would want to see some benchmarks (I have a hard time believing it will outperform lua with the built-in JIT or one of the faster JS engines with optimizations (it's not like performance is a novel desire in embeddable scripting languages, tons of manhours have been put into them already))... The website doesn't seem to have any such benchmarks.
Why do you believe it's ridiculous? Being able to integrate a third-party library by just adding a few source files to your source tree is as simple as it gets.
> The state of C++ build tools is very poor, and it is harming the language as a whole.
This assertion makes no sense, particularly in the light of this discussion. Installing a headers-only library is a solved problem, and even template-heavy libraries such as Eigen are already distributed and installed quite easily with standard linux package managers.
- Adding compile-time build steps, e.g. for code generation.
- Adding dependencies of your own, even on, say, a tiny library of helper functions – unless you want to copy and paste it into your header.
With a package manager, those things 'just work'. Package managers also make it much easier to update to newer versions of the library as they're released.
And let's not pretend that in some platforms such as pretty much each and every single popular linux distro already package and distribute C++ libraries and offer packaging tools and also package repository services to distribute any dependency.
I'm starting to suspet that those who complain about these sort of issues have little to no experience with C++.
If that were sufficient, we wouldn't see so many header-only libraries.
> Even if we ignore this fact, there are also package management tools such as Conan which handle thjs case quite nicely and also supporting cross-platform deployments.
Conan would be a decent solution if everyone used it. I tried it briefly last year; I ended up not using it for that project because it didn't support certain features I wanted (namely, compiler toolchain management), plus Bintray was having issues at the time. But my overall impression was that it was... fine. I may end up using it in the future.
For now, though, as a library author, most of your users won't have Conan set up; as a library consumer, most of the libraries you use won't be on Conan; and in either position, most people building your software won't know how to use Conan. Until that changes, it doesn't really solve the library friction problem.
> And let's not pretend that in some platforms such as pretty much each and every single popular linux distro already package and distribute C++ libraries and offer packaging tools and also package repository services to distribute any dependency.
If you have a library which is popular or a dependency for something that's popular, then yes, the N different Linux distributions and package managers for other operating systems will all handle packaging your library themselves. If you just want to be able to upload some code and have it be immediately reusable by the wider world, well, it's not particularly feasible to make packages for N different distributions yourself, and even if you do, the Linux distros most people use are months to years behind the bleeding edge of software releases, so you'll have a while to wait before those packages actually reach people.
Where do you see a difference?
>This encourages header-only libraries even when they are not really appropriate, increasing compilation times etc.
Thus assertion makes no sense. Headers only declare interfaces, and you only require headers-only libraries if you're deep in template and template metaprogramming land. Evenso it's quite trivial to package and distribute those libraries just like any other library
Yes, I realize that some header only libraries support using the header as either header or implementation, but this still blows up compile time.
Boost is really the only example I can think of where small utility comes at the expense of huge compile time increases.
If you stick to Linux and Debian derivatives for example then you've got all the tools for dependencies, building and packaging.
$ time python -c 'exit()'
python -c 'exit()' 0.01s user 0.01s system 98% cpu 0.021 total
And though python is no speed daemon, for a scripting/embedded language, it's likely to be fast enough.
Now try importing some stuff, and also testing a little more thoroughly...
> python3 -c "import sys, timeit; n = 10; print('%.2f ms/spawn' % (1000 * timeit.timeit('subprocess.call([%r, %r, %r], stdin=subprocess.DEVNULL)' % (sys.executable, '-c', 'import re, argparse, subprocess'), 'import subprocess', number=n) / n))"
Ubuntu (native): 20.82 ms/spawn
Ubuntu (WSL): 43.16 ms/spawn
Windows: 89.45 ms/spawn (without imports: 74 ms/spawn)
> python3 -c "import sys, timeit; n = 10; print('%.2f ms/spawn' % (1000 * timeit.timeit('subprocess.call([%r, %r, %r], stdin=subprocess.DEVNULL)' % ('php', '-R', ''), 'import subprocess', number=n) / n))"
45.14 ms/spawn
I also don't agree that 10ms is ridiculous. That's barely any time at all. That's less than a frame, at 60Hz. And, how many interpreter instances are you going to start up? Surely, no more instances than you have cores; that means, worst-case scenario if you have a ryzen threadripper or a maxed-out POWER9 (both have 96 cores), a second. For most people, it's going to take under half a second; considering most programs take tens of seconds to start up, I think that's fine.
Not to mention that if you have 'complex initialization code', that's going to take a long time regardless of language; what we're testing the the language core baseline startup time.
And the number of interpreters you spawn isn't a function of cores either. It depends entirely what you're doing, and nobody was talking about parallel execution. If you run a bunch of Python scripts in sequence you'll multiply the startup overhead...
I guess we'll just have to disagree on 10ms being ridiculous for a program that does nothing. To me it is. And half a second is orders of magnitude more so.
P.S. All of this is my timing on a pretty darn fast CPU. Try running on a more typical laptop CPU, switching to battery, and you'll get even worse timing.
And yeah, I have only a crappy laptop and your benchmark takes more like 50ms/spawn for me. Which I still think is reasonable because there's no reason to make a new interpreter state for ever script.
For an embedded application, of course you'd try to keep the interpreter state if it's possible. Sometimes you can't because of global state changes; sometimes you can. And sometimes, even with a single interpreter, the rest of the C++ program is fast enough that the embedded Python initialization time would dominate its running time, rendering the standalone vs. embedded distinction moot.
In any case... all you're doing is amortizing slow startup cost over the lifetime of the program. The startup is still slow. You were wondering "how" the startup is slow, so I wrote you benchmarks and ran them to illustrate. If what you really wanted to argue was that startup time is irrelevant, maybe you should say that instead of asking "how is startup slow" and making me spend my time writing a benchmark for something you don't care about.
And I did give you an example of Python that doesn't import anything, and as you can see (and could have easily tested yourself) it was still a lot slower...
Now compare with Lua:
$ python3 -c "import sys, timeit; n = 10; print('%.2f ms/spawn' % (1000 * timeit.timeit('subprocess.call([%r, %r], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL)' % ('lua', '-'), 'import subprocess', number=n) / n))"
1.09 ms/spawn
$ python3 -c "import sys, timeit; n = 10; print('%.2f ms/spawn' % (1000 * timeit.timeit('subprocess.call([%r, %r], stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL)' % ('newlisp', '-'), 'import subprocess', number=n) / n))"
1.25 ms/spawn python3 -S
It uses a header-only approach, which makes it easy to integrate with existing projects.
It maintains type safety between your C++ application and the user scripts.
It supports a variety of C++ techniques including callbacks, overloaded functions, class methods, and stl containers."
Shipping header only is really admitting that modern C++ still has a lack of technique, sidestepping the lack of modules and packaging for instance.
Furthermore, it's ignorant to conflate headers-only libraries with a problem or a shortcoming.
The fact that Header Only is one of the five bolded features of this project indicates that "modern C++ development" includes basically giving up on many desirable features of software projects. It's probably fair enough, but I wouldn't necessarily hold it up over other less "modern" designs like providing a C ABI, which you can still do with C++ using C++17, following basically all of CppCoreGuidelines in the implementation, using state of the art tooling, etc.
Some C++ libraries are header only because they were designed for highest performance possible.
Classic example is std::sort versus qsort, C++ usually wins because inlining.
My favorite example is Eigen, they use template metaprogramming to save RAM traffic. When you write x=a+b+c for matrices or vectors, the library doesn't compute a+b intermediate vector. That C++ expression returns a placeholder object of a weird type, CPU computes a+b+c in a single loop over them, reading a,b,c, and writing to x.
For instance, on some platforms system headers use preprocessor variables called "major" and "minor". If your code is in source files, you have strong control over whether you care. If your code is all in headers, your don't since all your code is affected by unrelated textual inclusions.
For cases where it matters, C++ can do that just fine.
Look at Direct3D, DirectWrite, Media Foundation. They're written in OO C++, and they use IUnknown ABI from COM. They're not COM objects, the ABI is the only feature they took from there.
Apparently, C++ library developers don't care enough.
I agree. But in some cases, you also gain performance not achievable with stable interfaces.
It’s about tradeoffs.
I think modern C++ needs stable ABI and well-isolated modules less than it did 10-20 years ago. 20 years ago people wrote everything in C++. Now C++ lost the majority of the market to higher level and safer languages, it’s only used for components where performance really matters. For such components, runtime overhead of stable ABIs sometimes matters.
I'm tired of that nitpicking. These tools happen to be outside of the language for historical reasons and probably C compatibility. It's not in the language because apparently not enough people care.
And I think that the ideal way is standardizing of the OS package management first, not having a package manager (or several) for each language. But it won't happen for OS business reasons.
There are properties of C++, like textual inclusion, that make shipping and maintaining code hard. Piling up header files and expecting downstream users to understand how the include search path will work is the current approach. I would just label it realistic more than a best practice. If it's a best practice for packaging, that is only because the current options are so disatisfying.
And all of that is basically orthogonal to whether you use a language specific package manager or an OS specific one.
C++20. __Twenty__, man! In the third decade of the language existing.
But I agree that it helps OS package managers.
But I stopped using it because it was insanely slow (in both compile times and runtime performance). Several hundred (if not thousand) times slower than the equivalent Lua code.
That was a few years ago, but skimming through the release notes, it doesn't seem to have been fixed.
You can put all your ChaiScript-related code into an isolated implementation file, and try to avoid touching that file as much as possible. But that only works well for long-term projects, where once you get all the bindings you need set up, you can coast for a while without touching those files.
For short-term projects it's a nightmare because you're constantly touching the ChaiScript files.
There's no inherent reason a header-only library should significantly impact compile times, aside from the fact that the authors usually don't have the foresight to make it efficient. It's also very difficult to find information about how to make templates more efficient at compile time.
If any hypothetical compilation time problem concerns you then rest assure that C++ enables you to develop submodules that wrap and instantiate your templates, eliminating the need to recompile them every single build.
Many of his videos / tutorials are worth a look if you are interested in C++.
I am worried though how memory management is handled under the hood. Is it possible to use traditional C++ memory management with any of the variables created in ChaiScript?
I'm not saying that I don't trust the implementation, I'd just like more control over it in some cases.
I believe the dynamic library loader requirement is a showstopper for AVR.
Codecov: 72%
Reality is often disappointing...
In all seriousness, looks pretty nice.
72% is more than high enough for me to believe them, but it's not 100% hence the gag
https://www.etymonline.com/word/chai
Edit: apparently the original source is China. Etymonline fail!
See: Tea if by sea, cha if by land: Why the world only has two words for tea
https://qz.com/1176962/map-how-the-word-tea-spread-over-land...
edit: For example, there's a fairly clear distinction in a game between engine code and gameplay code. Engine code is run all the time, and should be memory and computationally efficient, and robust. It won't likely need to be changed much, and is OK to spend some extra effort writing it to get all these properties. So C++ is great for this.
Gameplay code doesn't tend to have these requirements, and you may not want to pay the cost to write C++ if you don't need the benefits. And additionally, gameplay code often needs to be rewritten frequently while you experiment with gameplay to find what's fun, or do expansions, or support mods to the game, so a language that's very easy to write in helps a lot.