FabricPaul

As promised: Security was an important part of the design of Fabric.  What security means depends on whether Fabric is running standalone from the command-line or as a browser plugin.

When Fabric is run from the command line, we are in general unconcerned with security.  This is because Fabric is run like any other program on a computer where the software is deliberately installed by the user.  Fabric runs as a module for Node.js or for Python, and runs with the same security credentials as Node.js and Python.  Like Node.js and Python, Fabric will only do what you explicitly tell it to.  Let us be clear: this is the context in which you might use Fabric for server-side work, much as one does with Node.js, Python or Ruby.

When Fabric is run as a browser plugin, security is a major concern because a Fabric application (or, more precisely, an in-broswer application that wants to use the computer the browser is running on to run code) specifies code that is compiled and executed on the machine running the browser.  To prevent the usual types of exploits, the language KL in which the Fabric operator code is written is both pointer-free (like Ruby, Javascript and Python) and provides bounds checks for all array accesses, throwing an exception for any out-of-bounds accesses (like Ruby and Python).  Access to third-party code, which does not adhere to the same pointer-free and bounds-checked rules, is done through our extension mechanism, and extensions, besides the default extensions provided with Fabric (that are just wrappers for common open-source libraries), must be explicitly installed by the user -- it is not possible to make the user's browser automatically "download" an extension.

Of course, you are wise to question whether you can "trust" our security model.  Fortunately, if you're really in doubt, you can simply look at our code, which is open-source; we believe this already places us ahead of common browser plugins, such as Flash, to which could be posed the same questions but for which one cannot audit the code.

we wrote this piece last year - it's focused more on why JS isn't enough, but there's some useful info about KL in there. http://fabricengine.com/2011/10/couldnt-you-just-use-javascr...

I've asked one of my colleagues to write a longer response to this (we may post it on the developer blog), but I'll give you a short answer in the meantime. We don't do sand-boxing - I think a look at the amount of money that's gone into NaCl shows that a startup would have no chance of pulling that off.

By being pointer-less, we block a lot of potential malicious code. If you don't have access to memory, it's hard to write anything dangerous. Our bigger concern with the plug-in is around our extension system - it allows us to include existing libraries, which of course means it's opening up to C/C++. Consequently, we force explicit install of extensions - if a developer builds a custom extensions, then the end user has to install it, the same as if you were choosing to install a local application.

OpenCL/CUDA are GPGPU programming languages - that's something we're getting to soon :) We previously exposed OpenCL as an extension, but took it out because it's a nightmare to support. We're looking at it again at the moment - it's pretty challenging to write GPU code, so it's hard to see how much benefit developers will get from it until we can target nicely from KL. One for the longer term :)

The rationale for creating KL was that we had some specific goals, and we couldn't find an existing language that did everything we wanted - the requirement of being high-performance _and_ easy to use was critical. We also had security concerns (we started out as a browser plug-in), so it also had to be pointerless. Given that we have a fairly narrow scope (writing high-performance operator code), we decided the best path was a DSL. If we didn't have the security concern, we would have stuck with C++ - but that wouldn't have had the lower bar to entry that KL has.

My co-founder who wrote KL still thinks we were crazy to do it - 'the world does not need another language' :)

we did a lot of documentation work for launch - the overview is a good place to start: http://documentation.fabric-engine.com/latest/FabricEngine-O...

as per my other comment: "please correct me if I'm wrong, but I don't see anything about Cython handling multi-threading and I don't see anything about dynamic compilation on target. I just had a flick through their documentation, so if this stuff is in there then I missed it..."

it's awesome. Investors are going crazy

got it - thanks for the clarification :)

You're right - it should be publicly visible. We're working on it - it will be updated and public in the next week. We'll be offering subscription pricing - so initially it will be a monthly that allows you to run x number of instances.

p.s. Fabric Engine 2.0 has a flux capacitor

lol :) Read this older HN thread - http://news.ycombinator.com/item?id=3227905 - a few people took our code and tried to make it faster in C/C++ and Java.

We made our benchmark code available: (https://github.com/fabric-engine/Benchmarks/tree/master/Serv...)

If you have a standalone module for the high-performance, then of course you can bind it to different languages. We're not interpreting the dynamic language, so therefore semantic differences between languages aren't a major factor.

If you think about what we're doing, it isn't surprising that we'd hit that kind of performance - after all, we're asking the developer for a concurrency-friendly description, then taking their operator code (which is strongly typed) and compiling it on target. It's more about the dynamic compilation that LLVM enables, and the ease of access for regular developers.

It runs on instances - how do you describe that other than to say 'in the cloud'?

The benchmarks are properly presented and explained: http://fabricengine.com/technology/benchmarks/

I understand the skepticism, but we have been open with our data and the code we used for our benchmarks. We're not claiming to go faster than light here ;)

Using pointers adds a certain amount of complexity, and they also introduce the possibility of security problems. This was particularly important for the work we were doing on the browser plug-in. For the type of work that developers use KL for, they aren't needed.

Replying to comment below (can't see a reply button) - please correct me if I'm wrong, but I don't see anything about Cython handling multi-threading and I don't see anything about dynamic compilation on target. I just had a flick through their documentation, so if this stuff is in there then I missed it...

p.s. I meant to say - yes, it is misleading. We are most definitely not an automagical interpreter :)

The 'new' part relates to using LLVM to do the compilation on target - this actually makes a big difference in workflow, and is also much more familiar to people used to developing with dynamic languages. It also opens up some cool possibilities around scaling of computation...

agreed - I was pulled up on this here last year when we published our benchmarks :) The other thing to remember is the low-level language we use sets a lower bar to entry than say C - the benefit of designing for a specific set of goals (ease of use, performance and security). That said, if you're familiar with C or JavaScript, then KL will be very familiar.

we offer commercial licensing that allows devs to bypass the AGPL requirements. I'll update the site info soon to show the subscription pricing options.

not really - Fabric is integrated with dynamic languages, we 're not interpreting JS or Python (we work with both languages). Fabric is basically a high-performance threading engine that you can call from your dynamic language - the key element is that the operator code (KL) enables the high-performance. This KL is only required for the operators, and is not as difficult or complex as C/C++ to use - it's designed purely for this task. A regular Python or JavaScript developer can pick it up.

for reference, we posted our beta benchmark stuff here and people had a good look at us: http://news.ycombinator.com/item?id=3227905

we dynamically compile on target (using LLVM), and the developer doesn't have to manage things like pointers.

Yes, this is correct - it's targeted at people who are comfortable using dynamic languages like JS/Python

Hi all - you might remember the Fabric benchmarks we did with node.js last year. We've now integrated Fabric with Python as well, so I thought I'd share the news.

An important change since last time - Fabric will be released as OSS (probably AGPL). We will take the usual commercial license/subscription approach (similar to 10gen with mongo).

Obviously there are plenty of ways to make the Python code faster - the purpose of the benchmark is to show the same performance as we achieved with node.js, within the same paradigm (dynamic compilation etc).

Thanks,

Paul

Hi again - we implemented asynchronous compute for Fabric and node.js, which means we can go after the much publicized Fibonacci criticism. We're looking for closed alpha participants, so if you're interested please submit the request using the form at the bottom of that article.

Thanks - Paul (I work for Fabric)

Cool - can you contribute the code to the repository so we can test and merge it in? Thanks

I agree. I have made a separate post on this thread that calls this out.

Thanks -Paul

I asked the engineer for a response: "For reference, I used -O6 because it's a historical convention (more of a joke, really) for "optimize the crap out of it". UNIX geeks have been using -O6 in this way for about 30 years."

I've had a few email/tweet exchanges regarding KL, so I thought it would be useful to clarify a few things:

- KL is a language with a syntax that is very close to Javascript. It borrows syntax from JavaScript, but not the rules of the language itself. Just like JavaScript borrows syntax from C, and OpenCL borrows from C.

- There are many things in Javascript we don't support.  Some things we don't currently support (eg. in-line initialization of arrays) but will probably support in the future; other things we probably won't support (eg. regular expressions as language objects) and finally there are things we will never support (ie. closures).

- There are nice features of KL that Javascript does not have, for instance arithmetic operator overloading.  These features are included because they are particularly useful for computational problems.

- KL is not JavaScript++ - It's designed for writing high-performance operator code, not to handle everything that JavaScript can do. We don't want to reinvent the wheel :)

We will work on a post to cover KL in more detail, including roadmap. You can email info at fabric-engine dot com if you have any questions you want to take offline.

Thanks, Paul

Hi - the parallelism is derived from the dependency graph, which provides task and data based (SIMD) parallelism. So, if you describe a bad graph, then you'll get limited concurrency. The performance you get is directly related to how well you can describe concurrency in your graph.

Dependency graphs aren't new - Intel TBB recently introduced a dependency graph model, and it's a well known approach. So if you're looking for a tool for making it easier to run concurrent C++, TBB is a great option.

Companies have tried the 'analyze code and work out how to multi-thread it' approach with limited success - it's pretty hard to do that in a dependable manner. Often you see horrible code snippets being inserted as flags etc - which very quickly gets messy and intrusive. It's a lot easier to describe the parallelism at a high level and let the engine handle it from there.

Our view is that bringing this kind of performance to dynamic languages is a big opportunity. Our reasoning: - Modern hardware requires developers to write concurrent applications - the days of one big core are long gone. This is hard. - Dynamic languages are flexible, easy to work with and fast to iterate. They're also very slow compared to native code, let alone multi-threaded code. - There are a lot of people that know dynamic languages that can't, won't or don't want to work with compiled languages.

That said - Fabric will be free for non-commercial use (students, researchers etc). We open-source everything we build on top of the core engine - all of the extensions, 3D scene graph, rendering etc.

As for patenting - we've filed around some of the client-side stuff (I can't disclose details just yet - sorry). Most of the ideas that went into Fabric are not original - it's really how we've combined them to offer something that's (hopefully) compelling. We were very lucky to come at this problem when there was a perfect convergence of technologies - particularly JavaScript, HTML5 and LLVM.

Apologies for the ramble - been a long (but gratifying) day :)

Right - they have to be 'trusted', much like a plugin. People will have to explicitly choose to install them. Right now we bundle them all in he plugin, but that's just until we write the manager

Sure - we designed Fabric to be extendable, so it can include existing c++ libraries. So we just integrated the Bullet SDK. http://fabric-engine.com/2011/11/building-extensions-on-wind... covers the extension model in detail.

Does that answer your question properly? I can get more info if needed.