The reason to scrape them is simple: the total API surface of XUL-based extensions is pretty much all of the internal APIs of Firefox, which means that any change anywhere in the code of Firefox breaks some extension accidentally. That's a compatibility burden that Mozilla could afford when the only competitor was IE, but it makes the development of Firefox much less nimble than that of Chrome & co. So a new, less invasive and more future-proof API is sorely needed.
But that's also what makes them more powerful than any other extension API.
If your box is airtight then there is no out-of-the-box thinking.
An almost trivial example: Internal APIs can inspect the current (as-enforced) sandboxing flags of an iframe. Which is not the same as the sandboxing HTML attributes on the frame, because those can be changed without effect after loading and also due to inheritance of sandboxing effects.
This tiny tiny tidbit is useful when making content-filtering decisions.
And it is just one among countless others.
I don't remember specific numbers, but I seem to remember that pretty much all the extension points that had been requested by add-on developers were at least somewhere on the TODO list of WebExtensions. I hope that by the end of 2017, they will all be available, with a much cleaner API than their XUL/XPCOM counterpart, better tests and much better survival chances.
Of course, some of the changes will require pretty drastic reimplementations by the developers. For instance, going from js-ctypes to the native bridge will require full reimplementation in a different programming language. On the upside, js-ctypes was really painful to use and the native bridge will let you use a much better language for the task. Say Rust :)
js-ctypes allowed an addon to access already-present native libs on the the host machine or bundled with the addon itself. Messaging only works if you can get users to install some external application in addition to your addon, which is a too-tall hurdle for small extensions.
From my perspective those barely comparable.
> That's one of the reasons the WebExtensions team has been very actively discussing with add-on developers to try and prioritize which APIs should be created/ported to WebExtensions.
You are taking requests to keep existing uses supported, mostly for big, already established addons. My concern extends beyond that. Simply by having access to internals anyone can currently develop novel uses, even if they initially do not have a userbase. Those can only be prototyped by access to internals. Someone might not even have an idea if they don't know an API exists that could be leveraged to do it.
In other words, "I can't let you do that, Dave" API design can stifle creativity.
And it also introduces a bias in favor of already entrenched extensions, which is less open than the current approach.
I haven't checked, but I seem to remember that you can provide a platform-specific binary as part of an add-on, spawn it from WebExtensions and then communicate with it through the native bridge. Building/shipping one binary per platform is annoying, but the rest is infinitely better than using js-ctypes. I should know, I've been one of the first users of js-ctypes and I still shudder at the thought :)
> In other words, "I can't let you do that, Dave" API design can stifle creativity.
True. On the other hand, this is the kind of paradigm shift that was implemented by operating systems ~20 years ago (more in the case of Unix) and so far, improving the safety and security of operating systems has generally played in favor of users and creativity, not against them.
This doesn't mean that there is no space for lower level, MS-DOS/Apple II-style hacking – I'm doing exactly that these days. But still, separating both sounds like a good idea to me. YMMV.
> And it also introduces a bias in favor of already entrenched extensions, which is less open than the current approach.
Good point. I seem to remember discussions about how to have an "unsafe" extension mechanism that would make it easier to experiment with non-standard APIs. I haven't checked myself, but I believe that this mechanism has been published, as an extension called "the WebExtension Extension" or something like that.
No. Kernel modules can be installed at runtime. The user just has to grant the application the necessary privileges.
This means OS developers do not try to attempt to "strike a balance" between power and safety. They allow both, the unsafe stuff merely requires a fairly simple opt-in.
This is the whole crux of the issue. I do not mind security measures, as long as there is a bypass. Mozilla has taken the - in my opinion extremist - position that even bypasses cannot be allowed because they know what's best for users.
> I haven't checked, but I seem to remember that you can provide a platform-specific binary as part of an add-on, spawn it from WebExtensions and then communicate with it through the native bridge.
I am not aware of any such feature. This used to be true for bootstrapped extensions, but not for webextensions.
> I haven't checked myself, but I believe that this mechanism has been published, as an extension called "the WebExtension Extension" or something like that.
yes, only works on dev edition though.
What? No. Every OS out there allows the loading of user (administrator) supplied code to increase it's functionality.
In other words, every OS out there allows extensions that touch their internal APIs (or a very permissive subset of them in the case of Windows, that is known by how restrictive this is).
So far, Firefox did not even manage to reach chrome parity, let alone bug parity.
If you look at the Advisory Group meeting notes (in charge of new APIs AFAIK), there is almost nothing still: https://docs.google.com/document/d/1qCIUX0LavYixkzwan8IcSYeu...
Sorry David, but you're kidding yourself here.
Developers and users are also concerned about add-ons being prevented from exploring radically new concepts which would require those "super powers" apparently taken away by the WebExtensions API.
I'd like to reassure them: Mozilla is investing a lot of resources to ensure that complex and innovative extensions can prosper also in the new Web-centric ecosystem. In fact, as mentioned by Bill McCloskey, at this moment I'm working within Mozilla's Electrolysis team and with other add-on authors, involved in the design of mechanisms and processes helping developers experiment in directions not supported yet by the "official" the WebExtensions API, which is going to be augmented and shaped around their needs and with their contributions.
https://hackademix.net/2015/08/22/webextensions-api-noscript...
That comment by Giorgio (nice guy btw, shared a room with him on at a couple of mozilla events) is over a year old by now and rather optimistic. So far, nothing of that happened, nor will it ever happened at a scale that actually accommodates most add-on developers.
He wrote that it was already happening a year ago: at this moment I'm working within Mozilla's Electrolysis team and with other add-on authors, involved in the design of mechanisms and processes helping developers experiment in directions not supported yet by the "official" the WebExtensions API
I'm not an add-on developer but I've read about it happening in other places too, and I know for a fact that Giorgio is working on Firefox WebExtensions issues in Bugzilla.
I don't know how you come to this conclusion. They've been able to maintain XUL up until now and whatever they end up with in this new API, it'll be cheaper to maintain than the monstrosity that is XUL extensions.
The specification and implementation are done in cooperation with the add-on developers. They can draw from the community here.
And while they haven't reached Chrome parity yet, they do have already implemented some additional APIs. You're writing as if one thing would block the other.
PS: As to maintaining XUL vs WebExtensions API, they always maintained XUL/XPCOM themselves because that's what Firefox itself uses, meaning the entirety of Firefox developers "maintains" that "API". They regularly broke stuff for add-on developers, which was sometimes annoying, sometimes avoidable, and other times just necessary.
Some add-on developers learned to adept to that, other add-on developers switched to the add-on SDK (which like WebExtensions is a limited API, just not chrome compatible) if it was feasible, and a lot of developers will switch to the WebExtensions API if feasible in the future or even now.
So for the foreseeable future, servo is not an alternative to replace XUL/XBL anyway. mozilla said that themselves.
Using servo to render the actual web pages inside the browser, that's another matter (once servo is considered stable enough, of course). To the add-on developer, this should not really make any difference anyway, XUL add-on, SDK add-on or WebExtension alike, as most add-ons for the most part will use the regular DOM APIs which servo has to provide anyway to interact with web stuff.
It's definitely the case that the Firefox UI very much makes it impossible to move away from the current layout implementation at this point, and moving away would be a massive job. At the same time, the Firefox UI cannot move away from XUL/XBL until extensions no longer have free access to it, so this is definitely a first step that's relevant here.
As for using Servo for web content, it's worthwhile pointing out that not only is it the direction Mozilla is taking (with Project Quantum, and progressively moving components from Servo into Gecko), but it would also be a technical challenge because web extensions have access to extra APIs and have security checks disabled in others.
But those developers that cannot or will not use WebExtensions (e.g. I will not/can not port most/any of my add-ons incl. DownThemAll!), but probably would manage breaking changes (I did that for more than a decade now), are left in the cold, rainy dark now.
1/ either try and locate all the add-ons that will be broken, get in touch with their developers, be ignored by most of them, start several weeks of negotiation with those who do answer, then eventually, several months after my code is ready, land the change, and notice that Chrome has landed that same change a few months ago;
2/ ignore the add-on developers, improve Firefox immediately, but certainly break some add-ons, hence breaking the user experience of millions of users for no understandable reason.
As you can imagine, neither solution is good and everybody suffers from either.
WebExtensions condense all the instances of 1/ into a single point of time, hoping that we never again need to go through this painful dance.
For reasons listed in other comments above, the "we will expand WebExtensions API coverage based on current add-on needs" scares the shit out of me.
I'm having visions of why I quit using Chrome because there was some download UX that could not be altered, and the default Google "one true way" of doing it irked the hell out of me.
Note that in this case, we're not discussing (only) "unsafe" APIs but also "unstable" APIs that can change from one day to the next without warning.
(emphasis mine)
And responding to that emphasis: so what?! People who use Firefox instead of Chrome do so for a reason! You're failing in two ways: 1) "breaking the user experience of millions of users for no understandable reason," and 2) turning Firefox into Chrome.
If Firefox users wanted Chrome, they would already be using Chrome. They don't want Chrome, they just want Firefox to continue being Firefox, to keep working without breaking their add-ons that make their browser theirs!
Why is this so hard for Mozilla to understand?
The day Firefox stops supporting XUL extensions is the day I stop using Firefox. I've been using Firefox since it was Phoenix. I already have Chrome installed--I'm not using it, and there are reasons for that. You should meditate on this.
Mozilla is slowly committing suicide. It's painful to watch, because it doesn't have to be. One of the great forces in the Internet's history is not going to be around in a few years. Firefox will live on, because of the MPL. But what a shame to destroy the organization that brought it into being. Such a waste.
Oh well. If the Internet has taught us anything, it's taught us that, when it comes to software, developers are fungible. The phoenix will rise again.
Presumably he's talking about web features when he says "Chrome landed that change a few months ago". This isn't copying Chrome, this is web compatability. You need that for websites to work.
In any case, you certainly don't speak for all Firefox users.
So to fix this they're going to break every extension.
I think it's a fair assumption given how long the current extension API has been around that there will be more extensions that will die to disinterest or inability to move to WebExtensions than have been broken by all firefox releases from 4.0 to date.
As the market share of Firefox is not that high, maybe extension developers eventually won't be bothered with supporting that extra version, which means Firefox extensions will be less up-to-date and there will be less of them.
That's probably a trade off they have to make, though I imagine they're aware that some add-ons won't work and that users will be annoyed by that.
Software exists to be useful. The APIs in question make it useful. The developers have been maintaining it for nearly 20 years. Their employer receives millions of dollars a year to do so.
They don't want to maintain this "burden" anymore because it's not fun to maintain old code. It's not glamorous. No one becomes a rock star by unloading the bus. But if you remove the baggage compartments, the bus ceases to be useful, and the show doesn't go on.
Chromium is such a cooler project. It was started from scratch (except for the WebKit part), and it's made by Google (which at least used to be cool), and it's got all these modern APIs. They're so great that it only took them 7 years to support resumable HTTP requests[1].
Firefox used to be about the users. Now it's about the developers. (This is not to say that individual developers are selfish, but that the organization as a whole is behaving in a way that disregards the needs of users and prioritizes the desires of the developers.)
1: https://bugs.chromium.org/p/chromium/issues/detail?id=7648. Just look at this cleverness: "In the absence of a crypto::SecureHash object, DownloadFile reads the partial file and calculates the partial hash state in a new crypto::SecureHash object. If a prefix hash value is available, then the hash of the partial file is matched against this prefix hash. A mismatch causes a FILE_HASH_MISMATCH error which in turn causes the download to abandon its partial state and restart." Any other software in the world would just restart the download. I mean, they already give it a special ".crdownload" extension, so it's not like any other program is going to mess with it. But no, they can't just resume the download, they have to make 15 hash checks and pass around 7 different objects and find every possible reason to start the download all over again. What could have been a 5-line patch turned into 7 years of waiting, a dozen revisions across 50 files...
1. The remote file has changed since the previous download, so "resuming" the download will end up combining two different files.
2. The user's computer crashed during the first download, and some of the data in the partial download was not written to disk properly. (A particularly common case: the last few blocks of the file are zeroed out.)
2. Rollback a few megabytes.
These problems were solved ages ago by most download managers.
I'm with you. We should start moving to Pale Moon before the end comes.
So it was less of providing a poor api like Chrome and more like creating a newer and better API that also happens to be a superset of Chromes API.