Prepack helps make JavaScript code more efficient
prepack.io
prepack.io
Very impressive overall.
If you call a function on `window` during initialization you would need to use `__residual` to emit the code into the residual program (assuming the call has no side effects on the heap that need to be seen by other initialization code). This is because otherwise Prepack cannot know how the abstract function would modify the heap (if it is an arbitrary function it could modify the global object for example), so it is not safe to execute.
It's like optimizing handwriting for readability than typing it out. You're running a transform on the code that changes the format completely, how optimized the original form is doesn't matter.
I think prepack will also generally generate larger, bulkier code than you might get out of closure compiler. When I give prepack code from closure compiler, it seems to make the code much larger but change nothing. Your mileage may vary.
Tangentially related: what would happen if you did sue Facebook for patent infringement, and continued to use this software?
Remember - without that patent grant you have no rights to any of Facebook's patents anyhow. With it, you do.
So the worst case is that you'd be in the same situation if you didn't have the grant.
Explicit is much much safer for users of software, legally speaking, because implicit grants have to be settled in court.
If you really want to go to court over it, and can afford it, then you may be right. Want to test that against Facebook? They reasoned (quite rightly) that it would be silly to suggest that.
Has any court ever ruled that a permissive license like BSD does not include a patent grant when it says "use in source and binary forms, with or without modification, are permitted"? Because that seems flat-out silly.
There is almost no case law that supports implicit patent grants and what little there is requires the patents be ones...
"that dominate the product or any uses of the product to which the parties might reasonably contemplate the product will be put" -- HP vs O-Type Stencil (1997)
So, again, very limited. Very dangerous. Very very open to being sued and crushed economically.
This is exactly why FB's patent grant provides you more rights that you get without it. There is no ambiguity, no fussing over what patents may or may not apply and be covered by the case law. It's explicit and better for you.
Long story short: in many ways it's less safe to rely on implicit grants.
Yes, the grant is better than nothing, but also, the risk if the grant is removed seems minimal.
It's of course possible that they have (or will have) some that would be covered by an implicit grant - though you'd need to go to court to confirm that.
But it's quite possible that they could have many patents that cover non-substancial parts. Guess what? You're now in violation of those if you haven't already licensed them.
So, again, even in the best case the implicit grant still requires you to go to court to maybe get clear of some liability.
Explicit ones have you covered from the start. More rights for you, broader coverage for you.
"You have stored the software on electronic media and used that media as a doorstop, violating our access control patents by keeping the door open."
Hmm. Has that been filed yet?
I could totally see the point of a FOSS software project that implements a patented algorithm, where people work together to improve the thing but everyone who uses it still has to get a license from the patent-holder. (For a recently-top-of-mind example, Fraunhofer's MP3 decoding patent.)
Thus, it's not obvious that a FOSS license automatically implies patent release. Fraunhofer could have open-sourced some reference MP3 encoder themselves, without releasing the MP3 patents.
Software distributed under those terms would no longer adhere to OSI's definition of open source (I think it violates points 1, 3, and 6) nor the FSF's Free Software Definition (points 2 and 3).
It might not stand up in court, but I do think it's obviously unethical to release patent-encumbered "FOSS" without mentioning your patents and then demand users obtain a license after the fact.
I would be amazed if something like this does not stand up in court. Read the BSD license — it's really relatively simple english.
Adding a huge pile of details about what exactly that means makes me rather nervous. I am not a lawyer, and I worry that somewhere in that huge pile of explicitness that there are consequences that I don't anticipate, and that don't match up with the broad simple language those explicit details replaced.
(This is why companies have legal teams.)
The justice system is like a computer. It has no common sense; it has no "Do What I Mean" button. You have to write things out in full explicitness, for it to do anything predictable/sensible at all.
And just like with explicitness in programming, explicitness in a contract doesn't translate to "more things that could go wrong"; it's instead fewer degrees of freedom. More things pinned down; fewer left to interpretation.
It ain't hard fact until a judge says it is, and even then it can be moved with a big enough lever. That that specificity reduces axes of freedom on which to have what you think is obvious be not obvious is why I upvoted you, because that's the super important part everybody ignores.
A good lawyer tries to phrase their contracts et al to expose the connotation of the relevant laws in the text, so that you don't need to read the laws, only the text. Sort of like explicitly including default parameter assignments to a function.
If you ask a lawyer what is best, they will likely recommend a professionally drafted license like the Apache License v2. Just as you may not understand every aspect of a doctor's diagnosis or prescription, it may not be feasible to understand all the relevant statutes and case law. But I think it makes more sense to defer to expertise advice than to bury one's head in the sand.
In all likelihood you don't fully understand contracts regardless of length because you're unaware of the entire legal environment they exist in. The reason contracts often have weird stilted phrases is that they have specific meanings established over decades if not centuries of legal battles over those semantics.
A good example for doing this right are the Creative Commons licenses: they come with a summary in plain English for humans and a full license text for the legal system.
These are questions that you should be asking because the name of the game is minimizing risk.
The sooner the software developers of the world realize that there are reasons that we hire lawyers for legal tasks just like we hire programmers for programming tasks, the better. The profound arrogance you are choosing to exhibit regarding complex professions you don't fully understand is one of the absolute worst things in tech.
However, even the people who DO think an implicit grant exists would mostly agree that the implicit grant is not sublicensable, which makes it a horrible mess and probably unusable.
An explicit grant is strongly preferable, IF people can agree on the terms. Facebook's terms are on the harsh side, but there's clear advantages to it existing.
They sort of have. A patent has to be a major and "dominant" part of the implicitly licensed tech for it to be granted.
Basically - all existing case law says that you get some rights from implicit grants but it's also far less than explicit ones like FB's
I personally think that patents have no place in contemporary society, but that's just, like, my opinion, man.
http://en.swpat.org/wiki/Patent_clauses_in_software_licences...
Amend that to Software Patents and... I'm basically with you. Generally I think they are useful as a concept but I think they've become a bloated mess.
That said, I think I broadly agree with you about software patents; one reason why is that they're the glitter of intellectual property.
If you buy an Apollo diamond or one of those magnets, the use, possession, modification, etc of those items are not covered by the patents on the diamond (AFAIK, IANAL). With software patents, it seems to be the case that the "final product" IS covered by the patents.
This seems like it makes the presence of two separate categories pretty damn clear.
If you know better, you might want to give an answer here:
https://law.stackexchange.com/questions/14337/q-about-conseq...
I never imagined, then (being late 90s) that a brochureware dot com might require liability boxing legally, but I never imagined whole libraries like React being adopted so wholesale, so seemingly blindly (if not blindly, why has nobody posted the outcome of due diligence? A blog topic I'd like the attention from.) and so trivially as, yes, our brochureware site just could get used against us. Our early clients expressed concern, first: we had to pass their IP hygiene checks. We adopted those, right away. Such stringencies are why many entities I've worked with, have no or empty websites, but old old registration dates on their dot coms...
* edit, missing dependant clause I guess was strongly implied, but it's about when we first saw frivolous, vexatious patent suits aiming at the front doors of random web presences condition there might be money and weak legal... I mistakenly imagined that to be spurious, not it would develop into a psedo- legitimate"industry".
A moral thought: Maybe if we sold fewer reinvented wheels, there might be less temptation for parasitic behaviour to organize itself, like patent trolling has done? I can certainly plead the fifth to wheel reinvention, most days I write code ... (edit last) my meaning behind this is that I frequently have found highly objectionable behaviour being justified by the calling out of perceived comparable poor behaviour. Now I don't go sp far to condemn e.g. the js library crowd or any of anything for that matter, it's too young to blame yet. But with the first generation who grew up exposed to computing comparable to modernity, now maturing, its natural the industry will mature also. I see the brake on maturation more as lots of great new tools, than moral or human lacking. But self appointed grown ups trying to bully tax us, might be the natural parasitic compliment to this rich novel ecosystem. We don't need no random adults to appoint to combat this, we just need to question and talk about what is sane to accept. There's too much over reaching paternalism propping up big business assumptions, right now.
You do not lose the right to use Prepack. You lose the right to use whatever patents Facebook may or may not have on the technologies underpinning Prepack, if any.
If you believe Facebook has a lot of strong patents relating to Prepack, the value of the patent grant is high, and the cost of losing it is high. If you do not believe Prepack is encumbered by Facebook patents, then the value of the grant is nil, and the cost of losing it is nil.
> Tangentially related: what would happen if you did sue Facebook for patent infringement, and continued to use this software?
In my view, almost certainly nothing, because I don't think they have any patents on the underlying tech. But if they did, then they'd be added into the ongoing patent fight, and would give Facebook marginally more leverage when negotiating the final settlement.
I rather suspect that the patents Facebook has on other non-Prepack things would be much more decisive.
I could easily see a company 1) using something Facebook has created, like React, for a VR based UI; 2) patenting something related to their VR technology; 3) Oculus making the same kind of technology and not paying the company royalties/licensing its usage; and, 4) suing Oculus for patent infringement.
Of course it is within Facebook's right to make the patent clause as broad as it is, but I don't feel like it is fair that the company above would not be able to use React because Facebook infringed on a patent unrelated to React. It would be a lot nicer if Facebook either used an already existing license like Apache 2, or updated the patent clause to be more specific.
Again, keep in mind that in your example, the company still has a license to use React. What you're losing is your explicit grant of a license to the patents Facebook MAY have on the technology. But nobody has ever identified such a patent, and one of the core React devs is on record as saying he isn't aware of any either.
- Facebook says, "hey you can use this software, no copyright strings attached."
- Facebook says, "also, any patents we have to that software, here's a license. One stipulation, if you sue us for patent infringement, we revoke that license."
- Your hypothetical company, let's call it Acme, seeing the value of getting to use great software, for free, with no copyright or patent royalties, takes Facebook up on their charitable offer.
- Acme then, sues Facebook for patent infringement.
- As per the license, Facebook revokes their free patent license they gave to Acme.
And somehow the victim in this story is Acme? That's pretty rich.
Here's an idea: if part of your strategy as a company is using your patents to sue people, maybe don't expect those other companies to give you their patents for free?
You assume that I am making Acme out to be a victim, which isn't true. I'm arguing that Facebook should give a patent license that's more specific to the software they are licensing, e.g. React, Prepack, rather than a blanket license for any patents they ever get. For example, read the Apache 2 license, which is my preference for licenses that need a patent grant, as it's much more specific. You also assume that the strategy of getting patents is to use them to sue, which isn't true. You can get patents without suing, and the likely only reason you'd sue is if a company is willfully infringing upon a patent (e.g. won't license the patent).
(function() {
function fib(x) {
y = Date.now(); // the useless line I added
return x <= 1 ? x : fib(x - 1) + fib(x - 2);
}
let x = Date.now();
if (x * 2 > 42) x = fib(10);
global.result = x;
})();
I understand Date might not be acceptable for inner loops but a lot of my code that deals with scheduling would benefit significantly if I could precompute some of the core values/arrays using a tool like prepack.If you replaced the line with say: `var y = 4;` you'll notice that it has been optimized out.
You're right about why it's included, but this is a bug, not a feature. If this is a hot function it'll be optimized. The recursive call is a known, non-dynamic invocation so the function location will be inlined leaving just the fairly low function call overhead itself. There's a reason nobody does arbitrary-length loop unrolls.
I would definitely bet that this is a bug to be fixed :) Right now you can see this if you call fib(20) in that example...the compiler times out while trying to unroll that far. Clearly that behavior won't stick around.
It makes no sense to use an optimization tool that does the exact opposite in cases it cannot handle. In no world is 1400 lines better than 10.
If you know the possible range of accepted inputs, 1400 loc would be better than 10.
One normally think much more loc is worst than a one line solution, but that's, like most answers, always depends.
And that's a simple case. There are lot of similar cases one dont realize.
(function() {
for(var i = 0; i < 10000; ++i){
Date.now();
}
})();For instance, a common pattern being:
(function(w) { console.log(w.location); })(window)
This results in issues because it's unaware that window would contain other methods. Is it possible to exclude certain objects in this case?
V8 == user impacted.
Compile time V8 == compiler impacted, users happy.
It's like interpreting what you can during compile time, with caching capabilities.
To speed up boot time and init time, not runtime per say.
Information Theory, and the Pigeon Hole Principle in specific, says there is no algorithm that can compress all data. That doesn't mean that compression is a fruitless endeavor and we should never have written a compression library.
It means that you have to figure out if your input falls into a subset of data where the outcome of the effort is both tractable and useful. You compress text and structured data, you don't compress noisy data or already compressed data, because it's usually worse than doing nothing.
Similar thing with code analysis. If there is back branching or asynchronous code, you are probably going to hit the Halting Problem, so don't even try. But if the code is linear, then precomputing the output is tractable and useful.
You could also simply apply a budget. If an attempt to unroll a block of code exceeds a certain number of clock cycles or branch operations, you should give up and look at the next most likely scenario. The analysis you're doing might reliably halt in an hour, but who wants to wait that long? Especially when it's one of many such evaluations you'll have to do per build or per day? Just give up and keep moving.
Just evaluated this:
(function() {
var a = 'a';
var b = [a, 'b', 'c']
.filter((i, index) => i.charAt( 1 - index) !== 'b')
.map(i => i + '!')
.join(',')
console.log(b)
})()
BTW how you avoid going to infinite loops?Edit: Looks like this is targeting compiler output, so Flow types would typically be gone by that stage. Integration with Flow could come via a babel plugin that emits `__assumeDataProperty()` when it encounters a convertible Flow type.
I made a plugin to do this here - https://github.com/codemix/flow-runtime/tree/master/packages...
Having said that, there appears to be a sourceMaps flag in the Prepack source: https://github.com/facebook/prepack/blob/57cc59c07d164e4827c...
I will be attempting to use this as a simple black box, relying on Webpack for features that are only tangentially related to this new optimization stage in my build stack.
I'm a developer currently learning more backend and JS. I hope this question doesn't come out as arrogant. I heard once that "a good compiler should be able to compile itself". Does prepack prepack itself to run faster or can it just for the fun of it? :-)
https://github.com/facebook/prepack/pull/397
Currently it is blocked on Map/Set support in the output which is an outstanding issue. We could also try it with a Map/Set polyfill.
We're very close to being able to though!
In fact, this isn't just Prepack itself but it is Prepacking the JS part of the entire Node.js runtime as well!
In terms of the development status, does the current codebase generally work with React?
I'm happy to throw this at my codebase and report bugs, but whether React itself (which is ~95% of my bundle size) is expected to work (unlike some other optimising products) would say a lot about what the expectation might be.
I love it and can't wait to use it on some projects!
Prepack looks like the same kind of optimizer – it could be a fun task to write an interpreter and see if this can turn it into a compiler/transpiler.
Prepack reminds me more of a "supercompiler," because it focuses on partial evaluation rather than optimization.
I'm not sure you could take Prepack and do this. But it sure seems interesting. A kind of partial evaluation coding notebook... not so unlike a symbolic spreadsheet I suppose.
https://nolanlawson.com/2016/08/15/the-cost-of-small-modules...
Which is to say, one of its most effective use cases will be making up for deficiencies in Webpack, Browserify and RequireJS. Which I'm a little ambivalent about - I wish we could have seen improvements to those tools (it's possible, as shown by Rollup and Closure Compiler) rather than adding another stage to filter our JavaScript through. But progress is progess.
function define() {...}
function require() {...}
define("one", function() { return 1; });
define("two", function() { return require("one") + require("one"); });
define("three", function() { return require("two") + require("one"); });
three = require("three");
---> three = 3;
There is a certain irony that now it's possible to do optimisations like that in javascript - a dynamically typed language with almost no compile time guarantees.Meanwhile java used to have easy static analysis as a design goal (and I think a lot of boilerplate is due to that goal) but the community relies so much on reflection, unsafe access, dynamic bytecode generation, bytecode parsing etc that such an optimisation would be almost impossible to get right.
Arguably java has a larger subset even today.
Or roughly, if it compiles without errors, is it safe to assume it won't introduce new bugs?
Having said that, it's quite safe, but won't be undetectable. Code using eval could detect injected identifiers, we don't currently aim at preserving function names, and the method bodies you get with toString() are altered. That should be roughly it.
Prepack doesn't seem to optimize for V8 in particular; rather, it just does some calculations ahead of time. V8 cannot possibly do these optimizations. How would it know that e.g. my code tries to calculate PI to 1000 decimals? That's where prepack steps in and calculates PI to 1000 decimals for you during your build-step, so that the JS you ship doesn't have to do that calculation.
> experiment with JavaScript features by tweaking a JavaScript engine written in JavaScript, all hosted just in a browser; think of it as a "Babel VM", realizing new JavaScript features that cannot just be compiled away
I've been playing with making toy languages inside of Javascript, and I believe there's lots of untapped power there. The paradigm battles don't have to be: we can run all sorts of paradigms in the same VM, with data executing as code. This means that you can decompose expressions to see where they came from (e.g. the steps in a state machine that yielded an evaluation result). If you believe (as I do) that invisibility-by-default is one of the greatest pain points in the history of computing, then these sorts of approaches are essential.
The problem with doing that is that some things will be slower than they would in "native" JS. I've been proceeding anyway and thinking that could be dealt with later. So I'm bookmarking this, because it attacks that exact problem: runtime-compilation of generated AST's.
The name is a little unfortunate, in that respect, though, especially since it will make people think it's a build tool like Webpack. Webpack is for dead fish. This is incomparably more powerful.
edit so to answer your original question, this would piggyback on the optimizations of the VM (V8 or SpiderMonkey, or whatever), taking for granted that JS which is not needlessly verbose (as generated code must sometimes be), can be run nearly optimally.
I haven't used webpack but I have used many similar tools, and I've heard a lot of praise for webpack. Can you elaborate on why you dislike webpack so much?
Babel and all sorts of other compilers can run in the browser. Whereas webpack is mainly concerned with the text that you transmit and could not "by nature" be made to do its job in the browser (because then it would be too late). That's the difference I had in mind.
No for client javascript. Compilation is a slow and intensive process. You don't have the time to optimize the mountains of code loading on every new page. It makes the page much slower to load. The returns are negative if all the code executed turned out to be a single function to flash a menu, before the user left.
Yes for server javascript. It's long running processes, web servers should take time on startup to perform JIT compilation. It's beneficial in the long run.
React / javascript programming, is the most complex environment I've ever dug into, and it's only getting more complex.
create-react-app is great for hiding that complexity until you need to do something it doesn't support and then it's like gasping for air in a giant sea of javascript compilers.
vs.
run through JIT compiler -> download js -> execute
Whatever overhead the JIT compiler adds will be latency for the user.
Just another day at Facebook's office...
I wonder if this is the same project[0] Sebastian McKenzie previewed at React Europe 2016?
Can you convert a code base to React from Angular? Maybe, but the effort to write this converter is higher than rewriting the code base.
This tool looks interesting, particularly the future direction of it, but I'm weary about efficiency claims without a single runtime metric posted. The claims may be true, initializing is costly, but so is parsing huge unrolled loops. For an optimization tool, I'd hope to see pretty strong metrics to go along with the strong claims, but maybe that is coming?
Interesting work, nonetheless!
fib(2);
and more common to do: getInputOrHttpOrSomethingAsync().then(function(a){fib(a)});It's sad that there are developers and projects who write the type of code that causes these sorts of performance trade offs. I stopped writing this kind of fancy code a long time ago when I realized it wasn't worth it. You're just shooting yourself in the foot in the long run.
I think static analysis performance optimization tools are great but a certain part of me thinks it just raises the waterline for more shitty code and awful heavy frameworks that sacrifice the user experience for the developer experience.
"Just run it through the optimizer" so we don't actually have to think about what a good design looks like...
There's a LOT of parts that do different things. They may be aliased under one command (with tons of flags) but a modern system does a lot of stuff.
For modern JS we appear to have:
- Linters (ESLint, etc)
- Transcompilation to Object Code (Babel, etc, transpile to JS)
- AoT Compilation (this & Closure Compiler, etc, do optimizations on the code ahead of running it)
- Recompilation (AST based compression - like Uglify)
- Compiling (the actual JS VM, V8, etc)
- JIT (in the actual browser)
None of these steps are alien to other build chains.
In general, I would expect from a compiler to JS not to emit performance killers like that. Any decent compiler to JS should be able to remove its own crap on its own ;)
Optimizers are there to help you write maintainable code instead of unmaintainable micro-optimized spaghetti. Use them.
I respectfully disagree :)
I stopped writing excessively terse, 'overly clever', borderline 'minified' code a long time ago when I realised it wasn't worth it - it isn't readable, and it certainly isn't maintainable by my future self, let alone others. Maintainability is generally more important than saving a few cycles. Abstraction does not necessarily === "shitty code" - abstractions help us to build a mental model and reason about what the code is doing.
Having said that, I certainly see where you're coming from - abstraction can be overused, taken to the nth degree such that it's almost impossible to work out how anything actually works, and I do rather loathe such excessively heavy frameworks. But Prepack can help optimise the good frameworks that use just enough abstraction.
On the face of it, Prepack gives us the best of both worlds - developers can write maintainable code, and Prepack can optimise it a bit for us. Others here have menioned it too, but I see it as akin to the same sort of optimisations that compilers generally do.
I'm not sure what you mean by "fancy code." I feel like it's the opposite. We often try to write code that is readable and maintainable, and has as few as possible "fancy" magical constants littered throughout it, but I'm happy if a compiler wants to optimize it.
If I have to specify some variable, e.g. `frameRate = 60` and then I have a whole bunch of other variables throughout the code that depend on it, then I'm going to reference framerate everywhere: `framePeriod = 1/frameRate; frameBufferSize = frameRate * 100` or whatever. This way if we change our specs, I have a single number to change, instead of dozens of numbers that all depends on each other.
Similarly, if I'm calculating the volume of a sphere, you bet I'm going to put `v = 4/3 * Math.PI * Math.pow(r, 3)`, because it's a meaningful statement that everyone will understand, without trying to get "fancy" by optimizing it.
But should a compiler optimize it? Heck yeah.
It's the same problem TypeScript have/had that for external libs you need definition files for it to work. Now if we had TypeScript-to-assumeDataProperty generator that would be VERY interesting!
And the current state doesn't even know how to constant fold this loop.
function foo() { const bar = 42; for (let i = 0; i <= bar; i++) { if (i === bar) { return bar; } } };
(function() {
function foo() { const bar = 42; for (let i = 0; i <= bar; i++) { if (i === bar) { return bar; } } };
console.log(foo());
})();https://github.com/facebook/prepack/issues/543
Are you sure?
http://www.haskellforall.com/2014/09/morte-intermediate-lang...
The result was pretty good.
null or undefined TypeError at repl:537:23 at repl:5:16 at repl:2:2
* https://raw.githubusercontent.com/prettydiff/biddle/master/b...
Edit: @jagthebeetle: have you tried "advanced mode"? (One should read the documentation before using it, it's really a game changer but requires one to read the docu first)
Closure is really a great compiler, it's just a shame it doesn't interact well (that is, at all) with the modern JS ecosystem.
Within Google, CC is heading towards being a an optimizing backend for other less painful languages such as Typescript (tsickle) and the yet-to-be-released J2CL compiler.
CC does pretty well with these examples (our debugger is not quite as flashy): https://closure-compiler-debugger.appspot.com/#input0%3D%252...
I actually considered musing about something like this in my post. Typescript support would be very interesting.
Fwiw, the tooling support around the CC has been a problem historically. We relied on Plovr for a long time, but eventually it fell unmaintained, and there wasn't an alternative for a lot of the relevant parts (e.g. gathering source files you care about). Some important dev features also just didn't quite work as intended (sourcemaps) for a long time.
Not the best idea, imho.