Hacker News Reader

Comment by bsenftner | Hacker News Reader

Why interpret at all? Back in the mid to early 90's I started embedding C++ compilers into the game engines I wrote, where the "game scripting language" was just #define macros hiding the C++ syntax so the game level developers, who worked in this script, could be basically anyone that could code. Their "script" would compile to a DLL that was hot loaded. What they were doing in their scripts would compile in under 5 seconds, and they were good to go. If they ran into problems, one of the game engine developers would just run their "script" in the IDE debugger.

Borrowed this idea from Nothing Real, the developers of Shake, the video/film compositing system.

View on HN

Totally valid take. The answer is: it depends.

The advantage of a lot of scripting tech is some form of REPL, which is really just a super-fast code-compile-run loop. In your example, "why?" boils down to how useful/painful those five seconds-per-change are. Maybe that adds up and slows the coder down, or maybe it's no big deal. It all kind of depends on the workflow and how fast you need to be able to iterate on code changes. Moving to a scripted interpreter would eliminate that wait period at the cost of runtime performance, which might be a valuable business tradeoff.

FWIW, that "script" solution sounds awesome for the time. I'll add that five seconds to build a hot-loaded DLL in the 90's is really, really good performance for that solution, regardless of its role as a scripting alternative. Today, that would probably be mere milliseconds to compile - impossible to distinguish from an embedded LUA or JS solution.

> Why interpret at all?

a tiny part of points that come to my mind:

- education

- iteration on writing simple functionality

- loading and trying out several APIs to see what's possible (I use it frequently with Elixir / Erlang for example)

It makes life easier for newcomers to wrap their head around something and produce a good solution rather than a "working" one

But the "interpretation" is kind of in name only. When you run your program there's still going to be a compilation step, it's just the interpreter will merge it with the run step, and it will do it every time you run the program. I'm with the GP, I don't understand the advantage of this approach over traditional AOT compilation (actually this isn't even JIT, it's just deferred AOT).

And, then there’s

https://liveplusplus.tech/

https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledC...

https://learn.microsoft.com/en-us/visualstudio/debugger/hot-...

I don't suppose modern os would let you do that today, sounds like a security nightmare

This is just JIT'ing. It's used in python and lua commonly

I wish, JIT and Python still isn't something to brag about.

Python 3.13 has an experimental JIT compiler https://peps.python.org/pep-0744/

As I said, not something to brag about.

It's just loading a dynamic library from an arbitrary file, as traditionally done for third party software addons (e.g. Photoshop plugins since the early 1990s and VST instruments and effects since the late 1990s).

Yes, any modern OS lets any process load into its memory space binaries from anywhere the user has permissions, even if those are binaries it generated just now. It can be a security problem if the binaries are generated from untrusted sources (e.g. you download some, say, Haskell, compile it and run it fully automatically).

Windows has a private heap system where you can disable code execution from the pages allocated to `HeapCreate`, if you don't* set that flag:

https://learn.microsoft.com/en-us/windows/win32/api/heapapi/...

> HEAP_CREATE_ENABLE_EXECUTE

> 0x00040000

> All memory blocks that are allocated from this heap allow code execution, if the hardware enforces data execution prevention. Use this flag heap in applications that run code from the heap. If HEAP_CREATE_ENABLE_EXECUTE is not specified and an application attempts to run code from a protected page, the application receives an exception with the status code STATUS_ACCESS_VIOLATION.

I think POSIX has equivalent memory protection calls, but no equivalent to HeapCreate

But you can still call VirtualAlloc(), VirtualProtect(), and LoadLibrary(), so this isn't really a security mechanism, but more of a safety mechanism.

I don't think Windows provides a mechanism to disable creating any further executable pages, although I've seen Chrome do it by hooking those functions (and I know it because I've had to bypass it :)).

App Sandboxing and VBS enclaves go onto that direction.

I wouldn't expect windows to prevent creating further executable pages; there are legitimate use cases for creating dynamically allocated executable memory. It just means that whatever foreign data you load into those pages can't execute, which is a security mechanism (for example, game save data can be loaded into these heaps so that you can load all game state but without the save file potentially running foreign code)

There are legitimate uses, but the point would be that the process could ask the system to lock it down with whatever executable code is already present. This could be used to prevent already running code from tampering with the process' behavior by loading new code, or to thwart code injection.

>It just means that whatever foreign data you load into those pages can't execute, which is a security mechanism (for example, game save data can be loaded into these heaps so that you can load all game state but without the save file potentially running foreign code)

But malloc() and all the other standard memory allocation functions already return pointers into non-executable pages, anyway. Perhaps those functions call into this one internally, but using this over whatever your language offers by default offers no additional protection.

The benefit of heapalloc and heapcreate is that it makes serialization of data simpler because you can define the bounds of the heap so you can easily save pointer offsets. There are other flags available as well. It's a win32 feature so you would only use it if windows is your target platform (and then write other bindings for other platforms)

I still don't get it. Why can't you do the same with a regular buffer?

All of this calls VirtualAlloc behind the scenes, and you can do that yourself as well for manual page allocation. Each page can have options set with VirtualProtect to allow or disallow execution of code within the pages as well.

iOS won't let you do that. Or at least Apple won't let you do that on iOS.

What's actually happening is that the SDK doesn't expose the system calls necessary to do it, but I can guarantee that if you can get a native binary to run on the device, you can have it do whatever you want. If that wasn't the case, the few apps that do support JITting wouldn't work.

Reply on news.ycombinator.com