iOS 12 uses Lua code downloaded from Apple's servers | Hacker News Reader

iOS 12 uses Lua code downloaded from Apple's servers

twitter.com

It’s nice that Apple isn’t tying this functionality to OS updates. It was pretty bad a few months back when typing prediction went haywire and we had to wait weeks for a fix.

Separately, I’m surprised Apple didn’t do this with Swift. I thought one of the goals of Swift was to be a good language for low level code up to high level scripting.

Or JavaScript.

Presumably both of these have more overhead than Apple wanted in this case, but it does seem odd.

lua is much easier to execute and much easier to sandbox than JS.

I don't see how Lua is easier to sandbox.

I'm a bit surprised at the use of Lua, given that JavaScriptCore is already there. But I wouldn't be surprised if Lua was just something that the team was already familiar with. Lua is tiny, so it's not like it adds a significant dependency.

As you said Luna is tiny, which makes it easier to sandbox. A smaller system with a more limited set of behaviours and features is much easier to test and verify.

I guess, but (1) JavaScript is already there, so using it adds no new attack surface; (2) JS engines get more scrutiny; (3) sandboxing isn't as important anyway since the interpreter only runs trusted code.

There was a topic a few months ago about running JavaScript code in ring 0. People used similar arguments to claim it was safe enough. A v8 engineer chimed in to explain the reality of JavaScript engines.

https://news.ycombinator.com/item?id=17186842

JavaScriptCore has a huge surface area. It is measured in tens of megabytes, whereas Lua is measured in hundreds of kilobytes. Wikipedia chose Lua because it was small enough that they were able to audit every line of code for security. Verisign approached Lua the same way for their high-reliability servers.

Consider the complexity of JavaScriptCore. Their new-ish "FTL" optimization engine stands for "Fourth Tier LLVM", which means there are at least 4 levels of runtime optimizations JSCore performs on code, which is a lot of non-trivial complexity. And JIT means they cannot utilize no-execute bits in hardware.

Then there is the memory and power-consumption. Lua's footprint for a VM instance is tiny. It is also a very efficient language, even in interpreted form. JavaScriptCore is comparatively heavy in both regards.

Remember that Apple Watch still does not have Safari and JavaScriptCore is not available for watchOS developers. Presumably this decision is partly influenced by memory and performance constraints. Lua is trivial to get working well on watchOS.

For a core feature, it makes sense to have minimal dependencies with a minimal footprint so plenty of room is left for user applications. Remember that Apple probably has a bunch of skunkworks devices in their labs they hope to productize someday. And many of these might be tiny portable devices with minimal hardware capabilities, like Apple Watch (or remember all the different iPods?).

Lua's tiny implementation and extreme portability (written in pure ANSI C) make it well suited for this role. Apple won't have to port Lua to every new experimental CPU architecture they try, whereas JavaScriptCore has a lot of platform specific #ifdefs which have to be dealt with for every new platform.

> There was a topic a few months ago about running JavaScript code in ring 0. People used similar arguments to claim it was safe enough. A v8 engineer chimed in to explain the reality of JavaScript engines.

I've worked with SpiderMonkey and have known the team for about a decade now. I'm aware of the complexity :)

> Consider the complexity of JavaScriptCore. Their new-ish "FTL" optimization engine stands for "Fourth Tier LLVM", which means there are at least 4 levels of runtime optimizations JSCore performs on code, which is a lot of non-trivial complexity. And JIT means they cannot utilize no-execute bits in hardware.

FTL is gone, I believe. But anyway, JSC is not just a JIT. If you're worried about the JIT, just use an interpreter.

> Then there is the memory and power-consumption. Lua's footprint for a VM instance is tiny. It is also a very efficient language, even in interpreted form. JavaScriptCore is comparatively heavy in both regards.

JSC uses a hand-written assembly interpreter. I would be shocked if it beat Lua in performance.

> For a core feature, it makes sense to have minimal dependencies with a minimal footprint so plenty of room is left for user applications.

JSC has been shipping since the first version of iOS in 2007. Tons of apps use embedded WebKit already.

> Remember that Apple Watch still does not have Safari and JavaScriptCore is not available for watchOS developers.

This may well be the reason.

One purpose for making Swift was because Objective C is dynamic, and they thought they could get a speed boost by going fully static. The other is because they were having a hard time evolving a C-based language because C is full of plot holes. Swift is meant to be efficient but still allow abstractions, which C is really bad at. (The best you can get with C is leaky abstractions.)

This should be dynamic, but it shouldn't be LUA. It should be something like JSON. Although Lua is sandboxed, so it's safer that way, it is still an easier and bigger attack vector for hackers than JSON would be.

Do you really believe that if Apple wrote their own interpreter for their own custom language that serializes into JSON it would be safer than using one of the battle-tested LUA implementations?

I was suggesting that they don't have executable code at all, that they use static configuration. But it may not be possible within their problem domain, I don't know. I always prefer static solutions until dynamic becomes necessary.

Between code signing and sandboxing, this probably isn’t any more dangerous than the JS that Safari downloads and runs every time you open a webpage.

Lua, not LUA. It's Portugese for moon.

C gives you technically perfect abstractions, but it's a pita: you have to mess around with unimplemented structs (`typedef struct Bla Bla`, but never actually define Bla in the public interface, and then have functions taking Bla*), and void pointers, which generally makes it significantly less comfortable than in higher-level-languages.

Swift has a lot of goals, but having a small compiler isn't one of them as far as I'm aware. Swift has always leaned heavily on compiler heroics :)

To be fair ...

For any given programming task, there is a fixed (minimum) amount of work to do to solve it correctly, and the programmer's design choices merely decide the distribution of that work among the environment, the compiler, the standard library, the third-party libraries, and the programmer herself.

Meaning, that in order to use the programmer's time most effectively per unit of task difficulty, the language itself should be such that the compiler does as much work as it possibly can. Such a compiler simply won't be small, and probably won't be fast either. (imho the best it can or should do in the face of a syntactically-valid non-program, is to give as many errors at once as are applicable, so that the programmer can fix them all in one go)

Rust, of course, in the grandly oversimplified view, takes this tradeoff in the same direction as Swift, and I'm sure you're familiar with all the ways that Rust is better for it ... and even a few ways that Rust is worse for doing the opposite. :)

(Though, then again, there's miri, for those that value correctness over performance. Unless it too has changed names.)

Swift is more like the next C++ than the next javascript. But is supposed to be easy enough to use that you could use it for a shell script. And the 'shell interpreter' is just compiling and executing your code.

Or they could do what Google have done, and separate more OS components into applications delivered out of band via the app store, like the browser (Chrome), the keyboard (Gboard), etc.

Behavior changes of the os should not be quiet and magical, they should be intentional and the user should be notified (if they care).

Apple's goal is to have as many people as possible on the latest version of the OS. It makes support, etc. easier and therefore less expensive. In order to do that, people need to be enticed to update regularly. This comes in the form of updates to Maps, Safari, Emoji, etc.

The example is interesting (a suggestion to contact the organizer of an event you're running late to), and supports my suspicion that much of the "smartness" ascribed to modern devices boils down to lots of hand-crafted pattern-response rules. Pay no attention to the programmers behind the curtain.

I’m not really sure anyone should be surprised that much of the smartness is rule-based with maybe a bit of “AI” to help with speech recognition and fiddling with the words a bit. I think this is good because it tends to lead to misunderstanding being biased towards not doing anything instead of doing something particularly wrong. The downside is that the rules never seem to cope with enough inputs.

My hope is basically that many more rules will be written and they will slowly move towards handling more general problems as well as a very large variety of specific cases.

Looks like the code is not obfuscated and even includes comments. Does anyone here have the URLs? Might be an interesting read.

Is the “not obfuscated” part potentially just because the OS is still Beta? Seems somewhat similar to not encrypting the kernel, from iOS 10 (https://techcrunch.com/2016/06/22/apple-unencrypted-kernel/).

It's like a WoW addon. They should make the entire UI scriptable and create an addon store.

They won't do that because they want a consistent look for their devices. If you jailbreak, though, you can inject code that hooks into random methods and changes the ui; but that's not officially supported.

That's something that Apple forbids app developers from doing (unless the lua interpreter is running in a webview).

https://developer.apple.com/app-store/review/guidelines/

  > 2.5.2 Apps should be self-contained in their bundles, 
  > and may not read or write data outside the designated 
  > container area, nor may they download, install, or 
  > execute code which introduces or changes features or 
  > functionality of the app, including other apps.

The wiggle room is in the "introduces or changes features or functionality of the app" line, they've given themselves vague discretion to reject things that download too much code, but there are tons of apps that do OTA updates that haven't been rejected because they aren't changing fundamental features/functionality.

Also if there was no wiggle room here, then any app that downloads and displays webpage content would be in violation due to embedded javascript.

For a long time, downloaded code could only run inside Apple’s WebKit or JavaScriptCore, but since June 2017, you can use any language and interpreter. Apparently, almost nobody noticed: https://www.theregister.co.uk/2017/06/07/apple_relaxes_devel...

It's good that they revised that ridiculous restriction, but I think they should go further still. It was disingenuous to advertise the iPad as a "computer replacement" when having an environment for Python coding (widely used for beginner programming) was essentially banned. There really should be a way to work in even compiled languages on iDevices.

I thought that apps that used their own interpreters weren’t allowed to download additional code after installation.

Having a self modifying app is a nightmare from a security & privacy standpoint.

Wait, am I understanding this right? Apple is doing exactly what they forbid iOS developers from doing for years? Downloading code from the internet and executing it, in order to make runtime decisions or to allow live upgrades? That's just hypocritical BS right there. But the comment thread in the tweet is wrong about it being dangerous. Lua can be sandboxed like any decent VM in 2018. "Danger" isn't the problem.

You aren’t understanding it right. That isn’t forbidden.

Of course, it is their platform. The reason they prevent runtime/live updates is because 3rd party developers can and will easily inject malicious code.

Apples owns the entire platform, I have no problem with them having "root" privileges. Honestly I'd rather have a closed platform with strict guidelines than the wild wild west that is Android.

What. Apple has no issues trusting Apple 100% completely, nor do any of their users. Or more specifically, Apple itself is simply an inseparable part of the core trust foundation since they control the entire stack of hardware, firmware and software. You calling this "hypocritical BS" is just nonsense, it's like acting shocked that Apple doesn't allow any random developer to issue microcode processor updates even though Apple could. Of course Apple can, there is no equality there. Apple develops private code and dogfoods it before making it public. Any platform developer by definition has to do low level stuff as part of making higher level stuff. This is not difficult.

"Hypocrisy" gets thrown around way too often on the Internet, and if you do so you are basically always wrong, either because it's not actually hypocrisy at all (the word is not a synonym for "anything I don't like") or because it's a meaningless thing to say anyway vs more substantive complaints.

It's also worth noting that the bundle that includes the Lua code is signed and its signature gets verified by the OS before running.

> "Hypocrisy" gets thrown around way too often on the Internet, and if you do so you are basically always wrong, either because it's not actually hypocrisy at all (the word is not a synonym for "anything I don't like") or because it's a meaningless thing to say anyway vs more substantive complaints.

Another thing - hypocrisy has no bearing on the conclusion of the argument. You can be a hypocrite and be logically correct. OTOH, you can be far away from a hypocrite and be wrong.

Apple owns the OS. They've used private frameworks in their apps for years too, even though third party apps aren't allowed. Because they own the platform. It's hardly surprising.

There's a reason that there's App Review for 3rd party devs. Apple doesn't need app review for its own apps.

Adding to the other replies explaining why it's not hypocritical, I have an example. Consider section 5.2.4 of Apple's App Store Review Guidelines https://developer.apple.com/app-store/review/guidelines/#int...

    > 5.2.4 Apple Endorsements: Don’t suggest or infer that Apple is a source or
    > supplier of the App, or that Apple endorses any particular representation
    > regarding quality or functionality.

Apple definitely infers and suggests and even explicitly states that Apple is the supplier of their own apps. This clearly violates section 5.2.4. But Apple does not have to follow the rules that apply to third party developers because they're not third party developers. They're first party developers.

I think they mean "imply" rather than "infer", don't they? Inferring is drawing a conclusion: you imply something, and I infer your meaning.

Good catch. I didn't notice that when copying it.

Reply on news.ycombinator.com