How to build a plugin system on the web and also sleep well at night
figma.com
figma.com
In a nutshell, they use a same origin iframe to ensure the plugin gets its own copy of globals (so it can't mess up the globals your app uses), coupled with a proxy object which whitelists certain globals for the plugin to use along with certain vars from your app.
Really rather clever, although the guys who develop browsers should consider an API for something like this as it's becoming such a common use case.
We didn’t find anybody currently using this approach in production. Projects like Caja seemed unmaintained. So we’d have to reason about the security properties ourselves, but it’s hard because those approaches are more blacklist centric (remove unsafe JS features) than whitelist centric. They also did more than we needed, which increases the attack surface area.
There’s some papers on the topic that formalize Javascript in order to write proofs about it, but they’re quite old and some newer Javascript features like async/await definitely could invalidate some of assumptions behind these proofs.
While I don’t know the exact history, the Realms work does derive from Caja, it’s just the latest in this line of evolution.
The full new pattern, combining "with(proxy)" with a clever use of direct "eval", is from Caridy Patino of Salesforce.
The modern Realms Shim https://github.com/Agoric/realms-shim , and the SES Shim which builds on it https://github.com/Agoric/SES , is a collaboration of Agoric and Salesforce, especially JF Paradis of Salesforce.
We have come to refer to the fragment of code with the "with(proxy)" and the direct "eval" as the "Eight magic lines of code" and "The heart of the shim". It is explained at https://www.youtube.com/watch?v=9Snbss_tawI&list=PLKr-mvz8uv... and https://www.youtube.com/watch?v=mSNxsn0pK74&list=PLKr-mvz8uv...
edit: having now read the article, this is amazing, lots of get insights here. one question to the author if you are reading: it seems like it would be a worthy idea to open source the 500 line, security-sensitive interface Realm-shim. Selfishly, we would use it, but also, we (and surely others) would add eyeballs to it to ensure it's correct. Since it's a small slice of the system, and agonistic to the product itself, it seems unlikely to be part of any kind of technical competitive advantage. Any plans to do so?
Mozilla's very own Allen Wirfs-Brock started work on this sort of thing back in 2011 or so, called jsmirrors. The idea at the time was to inform the design of some reflection/sandboxing APIs that might make their way into the TC39 spec, but nothing really came of it.
Several years later I had a use case, only with security being the foremost concern, so I did a bunch of redesign and implementation work on that, including bugfixes and adding good handling of primitives. So there's already code available (around ~1100 here) for the kind of membrane described in the Figma post.
(At least it's halfway there; Allen's original work let you poke remote objects by way of serializing to JSON, and I neglected that area since it wasn't useful to my immediate use case and would have slowed me down--it sounds like you'd might need to stick something like that back in. For my own use as a consumer, I just ended up instead leaning on implementation-dependent details that I knew I could rely on from the way SpiderMonkey was hosted in Gecko.)
This was born in Mozilla, so it's already MPL2, and there wouldn't be any licensing issues.
Caveat: only ever targeted ES5, so changes for new JS features like proxies, symbols, and other ES6+ stuff almost definitely violates some invariants, but I couldn't tell you offhand whether that could manifest as security issues. The tests still pass.
Poke at it if you want:
cd /keybase/public/crussell/projects/jsmirrors && npm test
If I were you, I wouldn't assume good intentions especially considering you are just a start up and in a race for marketshare.
If it were up to me, I would set a date a year or two in the future and then open source it then, but only AFTER you have a large enough lead against your competitors in the market.
There is no reason that I can see to open source the secret sauce of your entire product.
We're kinda of all in this together. Security shouldn't be a feature or a competitive advantage, it should be a standard practice that all developers follow and participate in.
Everyone benefits when software is more secure.
Actually my demo goes as far as showing how one plugin can register a resource that is then used by another plugin. In this case I developed a plugin which registers a high level interface for access to an Ableton Push 2 device, and then another plugin uses that interface to draw to the Push's display using the canvas API. https://twitter.com/lwansbrough/status/1125842014128312320
[0] https://github.com/google/caja [1] https://github.com/lwansbrough/attack/blob/master/src/areas/...
Major, major kudos. This is how engineering should be done.
This is the first time I've heard of Realms API or QuickJS, will need to keep those in mind if I ever need to write a plugin system.
The introduction article [0] does a great job of explaining its motivation and purpose. I found myself nodding along to the points made, and the whole concept of a social publishing platform for interactive content.
here are some interactive examples:
https://epiphany.pub/post?refId=2684bc94f9fcb9ffe637ebfbeba2...
https://epiphany.pub/post?refId=3e123914a88818452d7f2c24fd8a...
I'm trying to make more.
I'm really amazed at the seamless reading and editing experience. This is very well-made, I can see it must have been quite an effort to make it feel so effortless.
The ability for a code block to provide a "mini UI" is perfect. A reader can tweak controls and see immediate visual feedback, in the form of graphs, algorithms with variables..
One control I missed was a stop button - for the second example, I wanted to pause the generation of the 3D city/terrains, to continue reading.
I liked the dotted blue vertical lines to indicate "block focus", to see what it contains when I click edit.
While dragging a block, I had some difficulty crossing over another block which was long - in fact, I couldn't swap them, since dragging down didn't trigger the scroller to start. From what I know, this relates to the "center of gravity" during drag, whether the dragged block goes under another one. As for the scroller, maybe it's using a range that needs to be bigger, so that when I drag a block "below" the bottom of the screen, it starts scrolling.
Anyway, lots of fun and impressive work! This looks very useful for educational purposes. You're really hitting a sweet spot with the concept - (just iterating on this phrase to get my head around it), a social publishing platform for interactive content.
Great execution, and I'm looking forward to seeing it grow.
I'm pretty early stage but I got a desktop app running last night. I've got some documentation work to do but I have a PoC of Devev as a webapp which I'd like to deploy soon. I haven't resolved all of the security issues yet so this article is a goldmine for me.
There is a way to avoid some of the pain discussed in the article with iframes as well, and it uses techniques from domain-driven design as applied to microservice architectures. So in this analogy the iframes are your "browser microservices" and the main app is also a sort of microservice and they all have to communicate with each other.
The basic idea there is that most microservice architectures are actually subtly monolithic because they have direct communication via The Database. Basically whenever you have a shared database you have a form of tight coupling which defeats the point of microservices. So in DDD, you deal with communications difficulties by creating these Bounded Contexts where a word means one given thing; applying that here with a notion that "meaning" is controlled by The Database, you want to transition to "miniliths" where a set of services has its own local database. Then your two miniliths talk to each other by passing messages back and forward; generally you want these messages to be Events ("this happened over here and I thought you should know about it but I don't want a response") rather than Requests (which invite responses and then you have to ask questions about what if the response is not what you expected or it was not received, etc., what happened in the middle?). You don't have to go the whole way to an Event Sourced Architecture (where your model is entirely determined by a sequence of received Events which have been stored in a database and can be used to "rehydrate" that model from scratch at any given time) to get about 80% of the value of the message-passing.
So translated to this context, basically what you have is a model of your system (possibly simplified) that you keep inside of the iframe and a model you keep in the app; the message bus communicates changes between the two but it seldom needs to serialize the whole structure at any given time. You allow plugins to interact with the in-iframe-model and it sends events "hey this happened" to the outside world, which has to respond to those events by updating its own data model. But fundamentally you have these two separate data models for the thing and they are being held together by a promise of eventual consistency through message-passing the diffs to the structure.
It makes sense to separate concerns by passing messages between isolated layers, and I can see how it could apply to other domains similarly. To summarize for my own understanding:
- "Minilith" (love the term!) with a model of your system (possibly simplified) that you keep inside of the iframe
- App with its own model
- The message bus communicates changes between the two but it seldom needs to serialize the whole structure at any given time.. Eventual consistency through message-passing the diffs
For some applications where I want to give users scripting capability, I've been thinking of a simplified DSL with its own "VM", passing some isolated objects from the host app to render results. The architecture you described, of "miniliths" with their own state (or in-memory database) passing messages to the host (or to each other) sounds like a sane and flexible approach.
With all the different interpretations of what is a microservice, a common one is that having a shared database makes it not a microservices architecture. But yeah, people call them that regardless.
The gist is to mirror a subset of DOM apis in workers and project changes back out to the main page.
As far as I know a few companies have tried similar methods, but most write proprietary APIs, rather than using the DOM.
Still in development but the examples are promising.
It really is delightfully fast. It's no surprise that the team behind it is producing this caliber of content.
const proxyHandler = {
get(target, name){
console.log(`get for target: `, target, name);
return 'tacos';
},
}
const p = new Proxy({}, proxyHandler);
with (p){
console.log(`document with proxy: `, document);
console.log(`access random property: `, a);
}
WebWorkers have the same issues as iframes where are basically like separate processes where data needs to be copied with message-passing. But if we had gone with the iframe approach, we’d consider sticking the plugin code inside a WebWorker inside the iframe.
Have they tried WebWorkers? This seems like exactly the type of thing WebWorkers were designed for, instead of iframes. A background thread that can load arbitrary code in a sandbox and postMessage to send messages to the main thread.
I remember looking into completely locking down the webworker to not even be able to make requests. It was possible! Just start a webworker from a sandboxed iframe.
https://stackoverflow.com/questions/10653809/making-webworke...
At one point they found a fucking legitimate reason to compile a javascript interpreter to javascript(wasm) to run javascript!
1: https://www.figma.com/plugin-docs/whats-supported/#trigger-p...
That said I'm totally fine if they isolate any event-based plugin as something you have to upload yourself or is bound strictly to your org using the app. As long as there's some way to do it designers will find a way, but without it I don't see it as being nearly as flexible as Sketch.
Compared to "native" approaches, where every application needs to implement their own solution, this seems far more durable (and easily fixed).
"PayPal Apps" Developer Guide (2010): https://www.paypalobjects.com/webstatic/en_US/developer/docs...
Here's an article from 4 days ago about it: https://news.ycombinator.com/item?id=20772326
Obviously, there are lots of inspiration to be drawn from apps we use everyday, such as GitHub, JIRA, etc, but these behind the scenes view is very informative.
There is material on the internet that are relevant to the topic, but it's quite hard to piece together. After all, there are only a handful of players right now who need to build an API that isn't a REST API.
Among big names, I can think of Zendesk (uses iframes), Coda (runs third-party code on their servers IIRC, isolated via server mechanisms), Salesforce (not sure exactly what they do, but I think they also use Realms as a component to their system).
https://medium.com/zendesk-engineering/sandboxing-javascript... https://trailhead.salesforce.com/en/content/learn/modules/le...
There's a couple of academic papers on JavaScript isolation, but you'll have to do a lot of work to figure out how relevant they are. Be sure to check the publication date.
The folks at Agoric are probably the leading experts actively working on untrusted code isolation in a browser environment right now. I would follow them if you want to hear about the latest new tech: https://github.com/Agoric/SES
https://userdashboard.github.io
I believe this approach was pioneered by Facebook some years ago in the earliest incarnation of their app platforms. There was no iframe sandboxing so they had an intermediate contrived language called "FBML" which compiled to a subset of HTML they allowed. That was the platform where Zynga made their fortune on Farmville originally, just before the iPhone.
Purchasing Sketch gives you one year of updates. If you decide not to renew the subscription, the app is still yours to keep and run - but you won't get any new updates. Figma follows the SaaS model of subscription, but they do have a free tier.
Sketch also has a lot more tutorials and plugins than Figma at the moment, in case that is important.
edit: I've used sketch from a design and a front end devs perspective and it was nice to use from both.
Used to work in an ad office making ads with Freehand and Illustrator and Photoshop. Figma is a wet dream compared to tools of that day. Could not have even dreamed of something like it back then.
Illustrator on the other hand is still vastly better at what it's named after - illustration. Figma has great basic vector functionality that will cover 99% of your design needs, but little around illustration needs.
I'm really excited for Figma and what comes out of it, but it still has a long way to go to catch up. I think it has a better foundation though.