Multiprocess Firefox
billmccloskey.wordpress.com
billmccloskey.wordpress.com
[1] https://github.com/mozilla/servo/
[2] I wasn't exactly thrilled about this myself, but as long as Servo has to interact with C++ code (notably, SpiderMonkey) this was judged critical for security. Fortunately, pcwalton seems to believe that Servo's tab processes will occupy less memory than Chrome's.
That being said, I'd also be ok with process-per-window, as that would give me the same basic ability.
- Tabs need to communicate with each other, or with a "host" application. In-process communication might be simpler than inter-process.
- Some people have WMs that don't provide an acceptable tabbing interface, and you want your application to have an acceptable interface everywhere. In this case I guess I'd suggest distributing a separate tabbing program along with your app, but that kind of separation of responsibility has definitely gone out of fashion.
Have you imagined an alternate scheme to tabs, where there are 100s of thousands of potential "tabs" which can be organized, called up in groups, moved in clumps together, and shared?
Imagine each group of tabs like soldiers in a Command and Conquer game...
[1] https://support.mozilla.org/en-US/kb/tab-groups-organize-tab...
I'd love to have something indexing and closing old tabs for me in the background instead of manually managing them. Heuristics like 'this tab or any pages on this domain haven't been visited in a week, index the page and close the tab'. At times I use tabs as reminders, perhaps adding some intelligence like "This Kickstarter is expiring in 3 days and hasn't been viewed in a week, you still interested?" might be nice. Any tab with future dates approaching might be worth alerting people to. A summary page might be nice with the last closed tabs, the upcoming dates tabs mentioned above. Pinboard or the like integration would be pretty awesome, that is primarily how I'm managing tabs now.
The more I think about this the more excited I get. Of course the tough part is satisfying each of us who have their own reasons and their own process for managing multiple tabs.
I have been thinking of something that combined Bookmark, History and Open Tabs.
* [A task based web browser - Conquering information overload](http://www.slideshare.net/mohanrajrm/a-taskfocused-approach-...)
* [A task-focused approach to support sharing and interruption recovery in web browsers](http://vimeo.com/9088447)
I am still waiting for a plugin which allows me to impose a efficient low-cost organisation layer over tabs (have made multiple feeble attempts to do a plugin which helps organize tabs using mind-maps, a org structure that is my personal favorite). I have tried almost all plugins over the years which could potentially solve this for me - but have not been successful in finding one that fits my needs.
It would kewl if this vein of technology grows. Perhaps cached tabs could store a snapshot for offline-browsing and recall after the page is down, too...
...or if multiple web pages could be combined into a single browser page.
Yes, some kind of meta-layer would be neat.
Maybe even combine a "multipath" "narrative" "storyline" into a navigable structure for communicating ideas from multiple perspectives, like news stories or technical manuals or software/hardware/testing/docs of the same product..
sorry for all the quotes and sky-high speculation - i'm buzzing on caffeine, and very excited to chat with other people about the possibilities of extra-tabular information navigation!
Firefox also does a neat thing with tabs that have been opened and not used in a long time, effectively unloading the website and reloading it once you visit the tab (for some reason it doesn't do so with tabs that I keep open in the background on purpose, like Gmail). They also had memory fragmentation problems, which they gradually fixed and now I'm very, very pleased with Firefox. Chrome is really draining my resources at 30 tabs opened, whereas I've had Firefox open with 100 tabs without issues.
The one process per tab model does have an advantage for misbehaving long-running apps. For example I had problems with Asana at some point, as I think it was leaking memory or something and Firefox couldn't handle it. In Chrome, you can find out what misbehaving tabs you have and simply close it.
If Firefox can switch to the one process per tab model, but somehow keep its current efficiency in managing memory, that would be so awesome.
Just bring up the menu bar (or hit Alt), then go to File -> Exit, which should close all Firefox windows but save the current tab history. Then, simply restart Firefox, again go back to the menu, then do History -> Restore Previous Session.
Besides the few tabs that get auto-loaded (i.e. the ones you were viewing in each window), the remaining tabs should remain un-loaded until you manually re-visit that tab (at which point the page will be redownloaded or loaded from cache into RAM).
(The standard tab UI also does not facilitate massive numbers of tabs, but tree style tabs solves that problem beautifully.)
The ability to queue up a reading list, to tag elements, to expire portions, to group stuff for later, to annotate it.
A mix of tabs, bookmarks, a tool like Readability, and Calibre.
So yea, that behavior is quite possible today.
They run self-contained in their own directory so you can quickly extract them to your Desktop or portable device. The installer downloads the latest build as you install it and configures it for standalone use. When you're done testing, you can just delete the FirefoxPortableNightly directory.
Bonus: The Nightly branch also has the new Australis UI redesign that they've been working on and is worth checking out.
Now with the security exploits many plugins have exposed and the way a misbehaved thread can bring the whole application down, we are moving back to the multiprocess model as a better sandbox model.
Old becomes new as they say.
Shared memory (be it shm/heap in same process) with associated mutexes, semaphores, locks and the like is a right pain to get right without introducing race conditions, deadlocks etc.
Go is interesting as it uses "micro threads" (goroutines) and message passing CSP style but I haven't found a use for it yet.
But is also has shared heaps by default. So you still have to rely on everyone being an "adult". I wish shared heaps would have been a specially enabled feature not the default. But I guess the language was position to compete with C++ and Java and isolated heaps would have provided a performance decrease in benchmarks -- and thus people's willingness to adopt it.
For large concurrent systems, safety and fault tolerance often leads to their failure but it is kind of hard to encode that in a quick benchmark to impress people.
Here are a few languages/systems with default isolated heap runtimes between concurrency units: Dart's isolates, Erlang's processes, Nimrod's threads, Web Workers in modern browsers. Anyone know of more?
http://dlang.org/migrate-to-shared.html
http://www.informit.com/articles/printerfriendly.aspx?p=1609...
> and no threads in favour of "select".
Not sure why you mentioned that. Callback chains can create concurrency contexts (callback chains) that can interfere with each other, much like multiple threads would. It would have a much higher granularity but you are not out of the woods.
That is true but you only have one thread to synchronise so mutexes and locks are vastly simplified.
Rust's "tasks" feature isolated memory, and, thanks to the magic of linear types, passing data from one to another is both statically-guaranteed to be safe and is never more expensive than the cost of copying a pointer.
Well, if you're passing something that is pointer size, yes. But say, `chan.send([0u8, .. 1_000_000])` will do at least one 1 MB memcpy to load that into the stream, and a 1 MB memcpy to load it out of it when `.recv()` is called.
I hope to someday see the language at least do the opposite, have a specially-declarable "isolated" goroutine. In theory, the compiler ought to be able to analyze a goroutine, determine that it shares no data at startup with another process (i.e., nothing in a closure or something), determine that it only communicates via value-passing channels (which courtesy of my previous restriction, can be analyzed by simply looking at what channels are passed in at startup time, and some analysis of the types of the channels), and thus guarantee that at least this goroutine is fully isolated. Pervasive usage of the new keyword I'm hypothesizing would allow a diligent programmer to recover most of the isolation advantages without having to rewrite Go entirely. It also ought to enable some other optimizations against these guaranteed-isolated goroutines, the biggest of which is that they no longer need to participate in a global stop-the-world GC, both in that they can continue running while that is occurring and that they also relieve the global GC from the task of scanning over them.
(In fact all the analysis ought to itself be fully automateable, and the user shouldn't have to declare it; I'd want them to still have the option to make a declaration so the compiler can tell them if they screwed up, though. I don't like such critical functionality being behind an opaque optimizer.)
But this certainly won't happen soon; there's a large enough list of stuff that comes before that.
That would preclude sending interfaces over channels (along with any other existential or mutable reference type), because the type system doesn't know whether the interface is closing over shared state. Not being able to send interfaces over channels would mean that channels would be restricted to only one kind of type, because Go doesn't have discriminated unions so interfaces are the only way to perform type-switch. Those goroutines would be so restricted as to be almost useless.
You cannot just bolt isolation on after the fact. You must design your language for it from the start.
That said, Go's race detector is very good and it's awesome that they focused on getting first-class support for runtime race detection so early.
Yes, I agree, and I'm sure I'm going to have many years of wishing they had. At the moment I don't have a better entrant in this field that is palatable to my coworkers, though. They've rather disliked Erlang (and not for lack of trying, and not for lack of good reasons, for that matter), Haskell's right out, and I'm running low on production-quality true isolation-based languages here. Several additional up-and-coming contenders; I'm sure if I could have used Rust-from-2018 I'd take that in a heartbeat, but, alas, it's 2013.
Go is tolerable, at least the way we're using it.
may you please elaborate on reasons for not using Erlang ?
It's a brilliant language that was well ahead of its time, and I don't mean that merely as a "I want to be nice" parting comment; it is a brilliant language that was ahead of its time and every serious language designer should study it until they deeply understand it. Indeed, I will absolutely attribute a significant portion of my success in programming Go to the wisdom (no sarcasm) I learned from Erlang, and Go would be a better language today had the designers spent more time learning about it first. (It still wouldn't be an Erlang clone, but it would be a better language.) But it's just become increasingly clear that it has been a drag on my project, for a whole host of little reasons that add up. It was the right decision at the time, because virtually nothing else could do what it did when I started, but that's not true anymore.
Someone will be tempted to post a point-by-point rebuttal. My pre-rebuttal is, I've been programming in it for six years (so, for instance, if there's some "magic solution" to clustering that has somehow escaped my six years of Googling, well, I think I did my part), yes, I know all other languages will also have "little things" (and big things), and Erlang may be perfect for your project, absolutely no sarcasm.
Tcl runs an interpreter per thread and communicate each other using message passing.
When you have multiple processes talking to another then, unless all your RPCs are completely stateless, you still need to orchestrate synchronisation in some form. There's also the problem that in 2013 we still don't really know how to do IPC/RPCs really well, and doing so portably is hard (Notice Mozilla are writing their own from scratch)
It's all very well new languages coming along and solving the trivial stuff (setting up message queues, isolating memory) but very few of them do anything to solve the real problems.
The "let it crash" model from Erlang and micro-kernels.
Still you are right, we are far from having an acceptable solution that we could apply everywhere, so to say.
I think that Rust makes a lot of effort to move things in the right direction. In Rust you can share memory, but unless you are in the unsafe sublanguage (which is clearly marked) the type system restricts you to one of: (1) copying messages; (2) transferring ownership of a message so that the original thread cannot race on it; (3) sharing immutable memory only; (4) taking a lock before accessing mutable data. We've seen huge engineering benefits from this: making parts of Servo thread-safe has been simply a matter of running the compiler repeatedly and letting the error messages tell us where to insert the locks; when it compiles we know the data races are gone. (You can still deadlock though.)
Just imagine the performance impact of something like Eclipse if each plugin was a separate process using message passing.
Maybe it is doable, not sure. Anyway I rather favor safety over performance.
Interesting that they chose to share code with Chrome. Since the two are competitors, I would have thought that they'd use completely separate implementations. It's interesting that open source makes this sharing possible.
- pridefully write your own
- use theirs
I would take for granted that the Firefox developers aren't stupid, and that they have enough interesting work to do that they aren't going to spend time on it as a learning exercise.
"We have to reinvent the wheel every once in a while, not because we need a lot of wheels; but because we need a lot of inventors." - Bruce Joyce
(I am aware that a fully sandboxed JIT is more complex to integrate than an IPC library)
[1] http://robert.ocallahan.org/2010/01/activex-all-over-again_2...
That's what open source is about: collaboration instead competition. Why compete when you can pool your resources together.
They have used SQLite to store bookmarks since Firefox 3: http://kb.mozillazine.org/Places.sqlite
But that doesn't mean they haven't rejected it elsewhere. Was just curious.
Edit: I see now. The WebSQL de-facto standard, which Mozilla rejected, essentially said: "use SQLite." Here's a discussion: https://plus.google.com/+KevinDangoor/posts/PHqKjkcNbLU
With my list of extensions[1] this doesn't seem to be particularly stable. It fails to bring up my tabs from last time. That would be OK for experimentation, had it not been for the fact that it also crashes regularly.
These two combined really is test-stopper for me.
Note: I'm not complaining. I'm very pleased this is being worked on. I'm just commenting first-hand experience about the state of things, so that others can make up their minds if they want to give it a go as well.
[1] Installed extensions: Adblock Edge, Duckduckgo search, Firebug, Flashblock, Norwegian dictionary.
Firebug might be a problem because it is tightly coupled to Firefox's internal debugging APIs.
1) Check if new commit arrived on 'head' 2) Auto backport it to the multiprocess branch 3) Try a build + run tests. Everything looks good ? Keep it 4) Not goot ? Send an email to the multiprocess maintainer so that he has a look ?
Is there another way to do that ?
But when Mozilla does branch off separate trees, VCS merges and lots of automated tests are largely sufficient.
Edit: you will lose your current session
I'm not sure what's causing the CPU utilisation in Firefox. It's common to blame extensions in the FF community because there's no easy way to determine where the problem is.
Granted this is due to AB+ and Reddit Enhancement Suite. Although imo.im leaking memory over time doesn't help! (I am somewhat annoyed that an IM client takes up 500MB of memory, I miss Meebo!)
Right now FF is at a fairly svelte 1.8GB. Heh.
The other problem is that performance degrades dramatically as the number of open tabs increases. Once I hit 50 or so tabs scrolling becomes horribly jerky. From the sounds of it, this change may very well fix that as well.
(For reference I am on an insanely fast home built machine!)
That could be a GPU driver issue. GPU scheduling is generally horrible outside the "run a fullscreen game as fast as you can and screw everything else" usercase.
I can also pop over to IE and it scrolls a-ok! (To be fair, IE11 has beautiful scrolling, everything else looks jerky in comparison, it really is quite a lovely effect!)
But all my plugins are in FF, so.... with an SSD it is not like FF takes too long to come back up anyway! Still annoying though!
It is weird that this issue is seldomly mentioned, but I think that it is much more important then the other performance benchmarks such as javascript performance.
Spending developer time on things like start up and installers which are just 1% of what software does is not very productive. Yes its good to look at start up every now and again, however it shouldn't be the main focus of any project.
... that said, try closing Chromium with 20 odd tabs open and then reopening it. It'll take bloody ages to reload all those tabs. Firefox lazy tab loading saves a metric buttload of time in this scenario.
http://support.mozilla.org/en-US/kb/reset-firefox-easily-fix...
I seem to remember that there are things you can do if you deviate too much from the average of others.. but I don't remember where to find that info.
http://www.tomshardware.com/reviews/chrome-27-firefox-21-ope...
I just hope Mozilla won't go extreme, and won't use a separate process for each tab like Chrome does. It produces memory bloat if you have many tabs open. While they say they'll mitigate memory issues, this should be balanced.
Given how cheap memory is, and the fact that almost all new devices are multicore I see this as a big positive.
I guess everything is stable, if you just avoid all the features everyone uses.
HTML <audio>? Web Audio API?
Most sites run flash in windowed mode, and for good reason - flash's performance sucks in windowless mode, and it cannot make use of hardware acceleration (IIRC). Since Adobe's NPAPI flash seems to be essentially in stability mode, it's unlikely this will be improved :(
Now, in current multiprocess mode, we actually force flash to use windowless mode -- because support for windowed mode isn't finished yet (bug 923746). But the aforementioned performance issues mean that we'll probably remove that restriction once we support windowed mode in multiprocess.
It has taken far too long for e10s.
Lets look at the reasons given as to why they want to do this.
>Performance. Most performance work at Mozilla over the last two years has focused on responsiveness of the browser. The goal is to reduce "jank"—those times when the browser seems to briefly freeze when loading a big page, typing in a form, or scrolling.
You can do all of this with proper threading and task delegation. Putting things in separate processes will not magically make things better. The answer to "jank" is proper coding, not over engineering. Last time I checked there was the same "jank" in IE and Chrome even though they use MPs.
>Security. Technically, sandboxing doesn’t require multiple processes. However, a sandbox that covered the current (single) Firefox process wouldn’t be very useful. Sandboxes are only able to prevent processes from performing actions that a well-behaved process would never do. Unfortunately, a well-behaved Firefox process (especially one with add-ons installed) needs access to much of the network and file system.
This is BS. You could have three processes and have FireFox sandboxed completely. Main process runs in a low integrity mode which limits it's resource access to a single directory. Second process is a download delegation process (takes a file after it is downloaded and moves it to the requested location while also promoting it's integrity) running in normal integrity mode. Third process is a network communication delegate/proxy running in normal or possibly even low integrity. These two delegate processes I mentioned will still be needed for the MP Firefox so it is no more work to create them.
>Stability
This is the only true benefit, but it is of very little value. Firefox almost never crashes and when it does, the session restore brings you back to were you left off in seconds.
Cons? More complexity means more bugs. This is a workaround for really fixing FireFox. I am going to have 150+ extra processes in my task manager now. More memory use. More context switches in the operating system eating up resources and causing more system latency and overall slowdown (context switches at the kernel level which will affect the whole OS).