Heavy SSD Writes from Firefox
servethehome.com
servethehome.com
There is a Linux utility that takes care of all browsers' abuse of your ssd called profile sync daemon, PSD. It's available in the debian repo or [1] for Ubuntu or [2] for source. It uses `overlay` filesystem to direct all writes to ram and only syncs back to disc the deltas every n minutes using rsync. Been using this for years. You can also manually alleviate some of this by setting up a tmpfs and symlink .cache to it.
[1] https://launchpad.net/~graysky/+archive/ubuntu/utils [2] https://github.com/graysky2/profile-sync-daemon
EDIT: Add link, grammar
EDIT2: Add link to source
overlaid 2.0G 128M 1.9G 7% /run/user/1000/codemac-firefox-3ckuo4v7.default
Searching on Github, this seems to be it. Turns out, there's releases for Arch, Debian, etc. and it's even in the repositories. No need to add a ppa. https://github.com/graysky2/profile-sync-daemon
For Debian and co:
$ apt-cache show profile-sync-daemon
$ sudo apt-get install profile-sync-daemonSee eg https://launchpad.net/openstack, https://launchpad.net/inkscape, https://launchpad.net/shutter etc
bird 11,38 GB 119,8 MB
kernel_task 8,82 GB 23,0 MB
launchd 8,46 GB 77,7 MB
revisiond 8,35 GB 105,8 MB
Tweetbot 3,01 GB 50,2 MB
Safari 879,6 MB 10,0 MB
mds_stores 726,2 MB 570,1 MB
systemstatsd 695,0 MB 775,0 MB
nsurlstorag 685,3 MB 9,5 MB
Google Chrom 681,4 MB 303,5 MB
iTerm2 653,7 MB 1,6 GB
cfprefsd 509,1 MB 45,8 MB
Papers 3.4.7 465,9 MB 169,8 MB
cloudd 441,3 MB 9,7 MB
ruby 224,6 MB 1,7 MB
coreduetd 217,5 MB
Reeder 207,2 MB 28,5 MB
apsd 65,7 MB 11 MB
Safari 55,3 MB 12,6 MB
rsync [...] /dev/shm/.browser ~/.browser
rsync [...] /dev/shm/.browser ~/.browser.new && ln -sf ~/.browser.new ~/browser
PREVIOUS_BACKUP=$(cat ./prev_backup)
NEW_BACKUP=$(date +%F_%T)
rsync --link-dest=$PREVIOUS_BACKUP /path/source /path/dest/$NEW_BACKUP
echo $NEW_BACKUP > ./prev_backup
And that's it.
Since it's the name is a time stamp. When you need to restore, just read ./prev_backup or just list the directory content, sort it and read the last entry.
The reality is that even with fairly heavy use, most of them will far outlive the computers they're in. And their owners for that matter!
I purchased my current SSD in March 2015 - it's a Samsung 850 EVO 512GB. It's done 7332 Power_On_Hours, and 36486573259 Total_LBAs_Written, which works out as just under 17 TB written, or about 31.6 GB/day.
That kind of sounds like a lot. But let's put it in context. Even the previous generation of TLC NAND SSDs were recorded in endurance tests doing around 1 PETABYTE of writes before failing. The 850 EVO, with it's 3D V-NAND, should be capable of at least double that.
For arguments sake, lets assume it will last for 1 Petabyte. I've written 17TB in 1.5 years, or 11.3TB per year. At that rate, this drive is still going to last at least another 90 YEARS.
So is this really something to obsess over?
I'm not sure what the lifetime of a computer is. In my family, we are still occasionally running machines from circa 1995. They work fine. When should I stop using my MacBook? The only reason I'd stop is if it died in a way that can't be cheaply fixed.
Pretty sure it's not 21 years, c'mon...
If you keep with 17 TB per one and half year your SSD should be OK for your for at least next 11 years.
Yes, but endurance tests almost invariably show that the warranty ratings are extremely conservative. The 1TB 850 PRO (which uses exactly the same NAND chips as the 850 EVO, just a different controller) has been endurance tested to more than 7 Petabytes. See: http://packet.company/blog/
The older generations typically had better endurance, the newer generations are more dense (cheaper to produce) but less durable.
That was true when comparing MLC flash to TLC. But the 850 EVO and PRO use 3D V-NAND, which has significantly more endurance than previous-generation TLC. This seems to be confirmed by endurance tests.
your SSD should be OK for your for at least next 11 years
The warranty runs out after 5 years, anyway. But that does not mean it will suddenly stop working on that date!
It's not "much" higher, actually, 3D MLC as implemented by Samsung is just "up to twice" better than the planar MLC given the same die area and capacity:
http://www.samsung.com/semiconductor/minisite/ssd/v-nand/tec...
"Samsung V-NAND provides up to twice the endurance of planar NAND."
But if the size of the cell drops, the number of P/E cycles drops. Samsung's endurance declarations are real and to be believed, they initially used bigger cells than some of their other chips (or competitors), but Samsung engineers know what they do, that "some tests" achieved "much more" can be either an accident or due to the errors in the test methodology (and I very much suspect the later, because it also allows the accidents to be taken as the "success"). At the time these SSDs appeared, Samsung declared twice less TBW than they do now, so now the declared endurance is surely not too pessimistic, but based on the real knowledge of what's inside.
In all likelihood the drive will never get anywhere near any of these values - it'll be replaced with newer, better tech at some point. It may then get used for a backups or secondary storage, but in those scenarios the daily writes will drop enormously.
Most of the SSD failed before a petabyte... and started encountering lots of errors before that (forget about the Samsung Pro series which is the only one that survived till its multi-petabyte end).
If a 250GB 840 EVO can get to 800TB without errors, then I'd expect a 500GB 850 EVO to reach 1 PB, easily.
A 500GB 850 EVO has newer-generation, more durable NAND endurance and twice the capacity to work with.
However I also only close my browser once every couple of weeks as well. System updates etc.
But for 99% of the users, they want it.
Heck, anecdote for anecdote: the very first thing i do on new machines is to open up firefox, go to settings, set the browser to startup with my previous session's tabs.
I guess doing something like this on Windows is too much to hope for, but does anyone know how to do this sort of thing on the Mac?
services.psd.enable = true;
services.psd.users = ["your-user-name"];The OP is complaining about excessive write traffic, which wastes power, steals disk-time from other applications, and may wear out SSDs prematurely.
A large profile file does not imply excessive write traffic, so it's not clear from your report that Chrome is hitting the same problem at all. Definitely worth watching, but big-file != excessive-writes.
There's an edit at the end of the article:
> Currently in the middle of a Chrome Version 52.0.2743.116 m test. We have been able to see a pace of over 24GB/ day of writes on this machine...
24GB is higher than the 10-12 he was complaining about from Firefox, even.
I hope we can get around to doing it someday. Of course, as usual in an open-source project, contributors welcome :)
On the back-end, I believe that Session Restore should be backed by a database, with each tab updated independently, rather than a big bunch of JSON data. The rationale being that:
- it's performance-critical;
- it's safety-critical;
- we have users with 300+ Mb of data in their Session Restore and JSON isn't meant for this scale of data;
- we wouldn't need to rewrite x Mb of data every 15 seconds, just a per-tab update;
- if we're using a relational database, it would be easier to trust the code to not screw up with the data;
- we wouldn't need to load the entire Session Restore upon startup.
(On the minus side, this might make backups a bit more complicated.)
On the front-end, we would need a high-level API for Session Restore, which would let us do things such as accessing per-tab data, (de)hibernating tabs, etc. Oh, and it would need to be accessible by WebExtensions.
In the middle, we would need to re-engineer Session Restore to make sure that we don't need to maintain this huge object representing the entire state of the session. We would also need improvements e.g. to cookie management, to avoid having to re-collect cookies so often.
Also, people who keep Session Restore tabs open (you know, the tab that lets you restore your session, when you have crashed) and continue browsing – Firefox needs to store several nested Session Restore JSON files.
Why not just use the filesystem? Imagine a layout like this:
+ profilename.default
|--+ sessionrestore
|--+ <timestamp>
|--+ window1
| |--+ tab1
| | |--- cookies
| | |--- formdata
| | |--- title
| | |--- url
| ---+ tab2
| |--- cookies
| |--- formdata
| |--- title
| |--- url
---+ window2
|--+ tab1
| |--- cookies
| |--- formdata
| |--- title
| |--- url
---+ tab2
|--- cookies
|--- formdata
|--- title
|--- url
No serializing and writing entire sessions to disk at once. Only small writes to small files. All plain-text, easy to read outside of the browser. Easy to copy, backup, modify, troubleshoot. No complex JSON, serializing, or database code. Use the write-to-temp-file-then-rename-over-existing-files paradigm to get atomic updates.
Need to know if part of a session on disk is stale? Check the mtime for that file, see if it's older than the last time that data was changed in memory.
Simple stuff. Straightforward. No overengineering. No bloat.
Why not do this?
It's a non-issue anyways, I have yet to see anyone show any actual evidence of an ssd dying prematurely or suffering degraded performance from this behaviour.
Maybe you could give users the option? (until it can be fixed fully)
about:config shows a big warning message "This may void your warranty".
(I'm guessing there must already be functionality to diff a bunch of JSON somewhere in the millions of lines of code).
Though I'm sure this doesn't make usually make a dent in a SSD's lifetime. But there are still people running Firefox on low end Android phones with meager flash, and Raspberry Pis with SD cards.
It's something that sounds easy until you actually try to get it coded up and shipping.
I explained below that this thing isn't a factor for Android because the program gets suspended.
It doesn't talk about the difficulties in getting data safely to disk. There's just a worry that taking a hash of the entire session state is expensive. I'm skeptical that a fast hash would take long compared to the time spent serializing to JSON in the first place, let alone time spent diffing.
That is actually a good idea we haven't considered yet. A bit too brute force for my tastes, but relatively easy to implement. We would need to determine how much CPU is needed for a diff between two 300Mb JSON files, though (yes, some users have these).
Of course, we're back to the issue of manpower, but definitely worth trying out.
> Though I'm sure this doesn't make usually make a dent in a SSD's lifetime. But there are still people running Firefox on low end Android phones with meager flash, and Raspberry Pis with SD cards.
The implementation of Session Restore for Android is largely independent, so I'm not sure how it works these days.
It'll compress to about 30% of the size, it's easy to do, and it shouldn't add more than a tiny CPU overhead over formatting the JSON itself.
It solves half the problem with like 15 minutes of work.
Also, what do these add-ons do? The only use case I can think of is figuring out whether the user has a tab open to a given site, and that's going around the browser's security model, so breaking that would be a good thing.
Then an hour of writing good tests.
Then lots of manual and automated testing on four or five platforms, and fixing the weird issues you get on Windows XP SP2, or only on 64-bit Linux running in a VM, or whatever.
Then making sure you don't regress startup performance (which you probably will unless you have a really, really slow disk).
Then implementing rock solid migration so you can land this in Nightly through to Beta.
Then a rollback/backout plan, because you might find a bug, and you don't want users to lose their session between Beta 2 and Beta 3.
Large-scale software engineering is mostly not about writing code.
Add the code that's able to load compressed session backups and leave it in for a couple versions.
Once enough versions have passed enable the code that writes compressed session backups.
It's really not that hard to do unless you want to enable it now.
No, for example, LZ4 is unbelievably fast:
https://github.com/Cyan4973/lz4
almost 2 GB per second in decompression!
I've just tried compressing some backupXX.session file (the biggest I've managed to find, just around 2 MB) and it compressed to 70% of the original, probably not enough to implement the compression -- and I suspect the reason is that the file contains too much base64 encode image files which can't be much compressed?
So the answer to having sane session files can be first to stop storing the favicons (and other images(?)) there? I still believe somebody should analyze the big files carefully to get the most relevant facts. For the start, somebody should make a tool that extracts all the pictures from the .session file (should be easy with python or Perl, for example), just that we know what's inside.
For a while now I have been running a cronjob to commit my profile's sessionstore-backups directory to a git repo every 5 minutes.
This is because, occasionally, when Firefox starts, it will load tabs in some kind of limbo state where the page is blank and the title is displayed in the tab--but if I refresh the page, it immediately goes to about:blank, and the original tab is completely lost.
When this happens, I can dig the tabs out of the recovery.js or previous.js files with a Python script that dumps the tabs out of the JSON, but if Firefox overwrites those files first, I can't. So the git repo lets me recover them no matter what.
What I have noticed is that the git repo grows huge very quickly. Right now my recovery.js file is 2.6 MB (which seems ridiculous for storing a list of URLs and title strings), but the git repo is 4.3 GB. If I run "git gc --aggressive" and wait a long time, it compresses down very well, to a few hundred MB.
But it's simply absurd how much data is being stored, and storing useless stuff like images in data URIs explains a lot.
Like you, I also observed that exactly the people who depend on the tabs to "remain" after the restart are those who are hit by the bugs in the "restoration" and as I've said, I believe the users would more prefer to have "stable" tabs and URLs than the "fully restored sessions in the tabs" when all the tabs fully randomly (for them) disappear. Maybe saving just the tabs and URLs separately from "everything session" would be a good step to the robustness (since it would be much less data and much less chance to get corrupted) and then maybe, pretty please, an option "don't save session data" can be in the about:config too)?
Once there's decision to store just the URLs of the tabs as the separate file, the file can even be organized in a way that just the URL that is changed gets rewritten, therefore making the "complete corruption" of the file impossible and also removing the necessity for Firefox to keep N older versions of the big files (which then eventually still don't help the user like you).
Yes, that would be very, very useful. I can get by if the tab's scroll position and favicon and DOM-embedded images--and even formdata--are lost. But if the tab itself is lost, and it was a page I needed to do something with, I may never even realize it is gone...
Or, maybe to reformulate, which wild scenarios does Firefox want to support now? I can imagine that the user's experience wouldn't match the wishes. Some people that use session restore claimed they "lost everything" from time to time, and I had to fish "just the urls" from their session store files which looked strange ("full of everything"), but automatically restored to nothing.
The goal of session restore is to restore your session -- your open tabs should come back, the same pages should load, scrolled to the same place, and with the right content.
Someday the pain may motivate me to try to learn enough about Firefox's internals to do it. :)
It would be interesting if somebody would actually analyze what takes the most of the mentioned 300 MB. I see a lot of base64 encoded stuff, if they are "favicons," come-on. There are so many caches in Firefox already, JSON files certainly aren't the place for these images.
But yeah, storing favicon in Session Restore would be pretty bad. I didn't remember that it was the case, though.
https://news.ycombinator.com/item?id=12569999
700 KB of binary images in a 1.7 MB session file, which can be compressed only to the 70% of its size.
I also see a lot of things like \\u0440 which spends eight characters for one unicode character (in another file, not from me). But that file was reduced to 37% of initial size with LZ4. It seems LZ4 is still worth doing, if the content remains easily accessible with the external tools, e.g. lz4cli.
For example, let's say I have 10 tabs open but I have been using only the latest tab during the last 5 minutes. In this scenario I don't care about the cookies (eg due to ongoing ajax) nor the state of the other 9 tabs. If the browser crashes I'm OK with those 9 tabs being restored with a 5 minutes old snapshot.
So for instance FF should be smart to save the state of new/recent tabs, and should slow down progressively on old/inactive tabs.
1. Thanks for your work.
2. How do I really contribute? Could you link to what I need to do to start working on this now? I'm affected by this problem ame want to figure out if I can fix it.
That advice was a lot simpler than I thought it would be.
To mitigate this I have created a daemon to copy the file to a separate directory every few hours. I then delete those old versions manually every year or so.
https://gist.github.com/alphapapa/bc7f3b25025fb99dad56
The git repo grows in size rapidly, but can be compressed way down. (Today it was 4.3 GB, but "git gc" compressed it to 230 MB). Every now and then you can blow away the repo or filter-branch to get rid of outdated sessions.
I feel it's little antisocial for regular desktop apps to assume it's for them to do this.
Chrome is also a culprit, a similar sync'ing caused us problems at my employer's, inflated pressure on an NFS server where /home directories are network mounts. Even where we already put the cache to a local disk.
At the bottom of these sorts of cases I have on more than one occasion found an SQLite database. I can see its benefit as a file format, but I don't think we need full database-like synchronisation on things like cookie updates; I would prefer to lose a few seconds (or minutes) of cookie updates on power loss than over-inflate the I/O requirements.
For completeness, this is probably the eatmydata command.
In my case it was not SSD wear I wanted to reduce, but the short pauses during interactive use.
Edit Double-checked in the code: I'm right, it doesn't.
Edit If you're interested in how Session Restore improves the safety of its writes without using sync()/fsync(), the details are on my blog: https://dutherenverseauborddelatable.wordpress.com/2014/06/2...
Checksums/signatures would be more precise, indeed.
In the case of a Chrome (many versions ago; may have changed), we manually tweaked that pragma in all our users profiles. I recall something like an order of magnitude reduction of pressure on the /home server.
So basically, no, you don't have to worry about this. Essentially, people are complaining that the disks they bought are being used for their intended purpose: making sure your data doesn't disappear on a power loss.
There is also a case of the silly SSHD's or w/e which combine a tiny SSD and a mechanical component which can die considerably faster.
Overall the amount of writes an SSD can handle is tied directly to it's capacity since it determines the amount of average writes per a given time unit that each cell will experience as long as the firmware actually does it's job (in Samsung's case it may very well not do it as they had more than a few firmware screw ups in the past).
This can also more severely affect you if you have a medium sized drive (say 128-256GB) but it's pretty full.
A good SSD's will move things around somewhat to even out the wear but it still increases the amount of writes you have.
If your SSD does have an "even wear" feature when you write 12GB to drive which is say 50% or more full it usually triggers more than 12GB of effective writes in the drive itself as the firmware would move already written data around to ensure that every cell has been written to approximately the same amount (if you say only have 20GB of free space and you write 12GB daily and delete it, your effective writes would be closer to the full capacity of the drive per day than to 12GB).
You also have a lot of embedded/mobile devices which use considerably shittier storage than modern SSD's, you can easily chew through a SD card or a similarly rated flash storage with anywhere between a few 100's of gigs to only a handful of TB's (or a only few times the capacity of the card if buy bargain bin memory cards).
Since these machines are more commonly used these days it's not a feature that should be so quickly obfuscated from the user since it doesn't seem like there is an "intelligence" behind how conservative or not the browser is with writes to local storage.
If you install firefox on an SBC/netbook/microcomputer or even a mobile phone (if this affects the mobile version also) with eMMC/MMC only rated storage 12 gigs a day can and will kill it rather quickly.
Other things like add ons, amount of tabs open and etc. could also affect it.
Would be interesting to see if some one has SBC's with either SD or onboard eMMC rated storage to run this test and see how much is being written too and calculate just how much of the lifespan of the storage is being reduced due to this.
IIRC even the iPhone 6s still uses TLC flash for it's storage, TLC is only good for about 500-1000 P/E cycles, so it needs pretty good capacity overhead and well "TLC" to live long.
In comparison SLC gets you over 100K P/E cycles these days, MLC can get to over 30K.
Also if anyone knows if mobile phones do wear leveling for their eMMC/MMC storage I would love to know that ;)
You're acting like the browser is a major local storage application (like excel for example), almost everything I'm going to be working on is an online application, and it takes care of storing my data on the server infrastructure. So yea, 15 seconds or 60 seconds isn't going to be a major 'data loss' issue either way.
Do you manually hit the save button every 15 seconds?
Some webapps do this for you. Some do not. Firefox has got your ass covered even for those that do not.
It might be nice if this was tunable via a setting though.
Because you hear the data volume, the function, and you think there has GOT to be a better way
Firstly it shouldn't be backing up every "N" seconds. It should be backing up after every change, and for performance reasons coalescing changes in time interval bins (eg 1-5 second blocks).
Secondly, it should only be writing the information that changes. Sure websites contain a lot of data, but most of it doesn't change very fast. So it's a big win to only back up the deltas (and reconstructing it is fast on SSDs).
If a page does have lots of transient data... write lots of data! That's partly how the time coalescing parameter is tuned.
Ideally it would only write out the entire page of data when it loads, or when the size of the deltas exceeds some threshold (possibly different for HDD vs SSD).
If there is also an associated 12 GB/day of reads then half that estimate (unlikely to be equal due to the nature of the cache). Either way -- not a significant degradation.
Edit: Degradation or not -- i'd personally rather have my drive resources available. In the context switch from firefox the cache process could interfere with loading other data.
Drives that reach >1PB or write endurance are no longer on the consumer market. You can buy them (SLC), but you wont after seeing the price.
You can do nice tests that will last for hundreds of TBs .. until you pause for 24 hours and realize all the data is gone.
[0]http://techreport.com/review/27909/the-ssd-endurance-experim...
840 is especially broken and dramatically slows down when you dont touch the data for >month. The "fix" is firmware silently rewriting everything in the background burning thru the rewrite cycles.
You still lose all your data though :(
Real lifetimes are often a multitude of the endurance rating, BTW.
No one is arguing against that point. However, most people do not need their browser state to be needlessly backed-up that aggressively (every 15 seconds - which is about the same time it took me to type everything before this sentence). I'm OK with losing up to 20 minutes of my state in a browser. Most work done in a browser is usually synced to an upstream browser every few minutes - auto saved emails or multi-step web-apps.
If your job includes having to use a web application for significant portions of said job, it's even more invaluable. I've worked on several applications that account for more than 40% of some employees' time per day... This was back in the IE6 days, they bemoaned having to close/reopen once a day because of memory issues (IE6 had horrible GC across COM boundaries).
[1]: http://www.intel.com/content/dam/www/public/us/en/documents/... section 2.6
As I understand this feature is there so if the browser crashes it can restore your windows and tabs - I don't remember having a browser crash on me since the demise of Flash.
And yes, there are browser crashes without Flash
I stopped using Edge because it only saves your tab session if you turn off the PC, not if you quit the browser.
Mind you, for a while I ran Firefox using a RAMdisc for the profile simply because all that flushing makes a big negative impact on PC performance.
Chrome Extension: https://chrome.google.com/webstore/detail/tabist/hdjegjggiog...
Firefox Extension: https://addons.mozilla.org/en-US/firefox/addon/tabist/
source code: https://github.com/fiveNinePlusR/tabist
Let me know if you find it useful or have any suggestions.
Anyway I used an extension like this for awhile, called session buddy. But now I just use control+D and bookmark all my tabs. Then you get bookmark syncing, and you can save your tabs when you export bookmarks, or wget them to have a local archive, etc. It's just more convenient, and one less extension I have to worry about stealing my personal information or bugging and corrupting my data.
FWIW, Mozilla looks at all extensions before fully approving them and the app is fully open source and fairly easily vetted if people are worried about that sort of thing.
Thanks for the feedback!
On suggestion - please give a button to save all of this as a "session" or something. I'm a tab hoarder and sometimes I just want to dump it somewhere and start afresh.
Please help me !
thanks!
this session restore feature, lets just save the urls in a list. that data structure cant be GB of GB right! ;) I think thats enough for the, lets say, typical FF user.
Now, that i know about this issue, i will bring down my tab usage to no more than 10 tabs.
The internet allows it to do that.
Just try browsing reddit with the RES extension or streaming heavy data for a few seconds (like a server side PDF processor that takes more than 30 seconds to finish, it's the same old behavior of blocking JS dialogs from years ago, white screen of death). When I'm using a iGPU (512MB) and I open some heavy images in sequence the browser crashes after a certain amount of time but this time it freezes the entire computer for a few seconds, so I suspect it's OOM (iGPU). I never bothered looking for a solution, nowadays I don't even care when my browser ocasionally crashes. My overall desktop (software) experience gets worse every year so I have gone used to it.
But Firefox still is my favorite browser and I wouldn't trade it for anything else!
Yeah that's the difference - I only use few well known extensions like uBlock Origin, Self Destructing Cookies, Lastpass and Decentraleyes. No trouble with any PDFs either. Anyway this is getting off track with the crashes or not - my point was even if it crashes it is no big deal if you can't restore - just reopen will work in most cases.
RES is more well known (in terms of number of downloads/users) than 50% of your listed extensions, for the record.
I don't like the anxiety of opening Firefox in front of someone, and wondering if my previous tabs will pop up.
At work, I routinely have something like 30-40 tabs open, if my browser can restore the session after a crash, that is very valuable to me.
I'll see if it has any affect on my laptop's battery life and fan usage.
I was thinking the same and will do so myself, if no-one else reports doing so.
It also is about single CD speed (yes, you could almost record uncompressed stereo CD quality audio all day round for that amount of data)
All to give you back your session if your web browser crashes or is crashed.
Moore's law at its best.
This is a far superior solution to fiddling with configuration options in each individual product to avoid wearing down your SSD with constant writes. Murphy's law has it such hacks will only be frustrated by next version upgrade.
And no, using Chrome does not help. All browsers that use disk caching or complex state on disk are fundamentally heavy on writes to an SSD. The amount of traffic itself is not even a particularly good measure of SSD wear, since writing a single kilobyte of data on an SSD can not be achieved on HW level without rewriting the whole page, which is generally several megabytes in size. So changing a single byte in a file is no less taxing than a huge 4 MB write.
Because FF may die but the OS will save it later. That's fine
Not every write to a file means a write to disk
This is very much the feature working as intended: Firefox captures webpage state every 15 seconds, so a crash, power outage, accidentally closing the browser etc will not result in data loss for the user. For storing the data persistently, it uses the persistent storage, i.e. your harddrive. For the HN crowd, that's going to be an SSD. The SSD can easily deal with the resulting write traffic with negligible effect on its lifetime - I have no idea why people even think this is an issue.
Seems like it should only write after a user action or if the open page writes to storage
Webpages can change their contents or state without user interaction.
Webpages are also big and bloated. Storing a little data, all the time, 24/7, adds up over time to a surprisingly large number.
I understand that it's probably not an issue for most users/SSDs. However, if I open just a static HTML file and leave it idle, I don't think the browser should be continually writing to disk. If it is, then to me that is a bug.
EDIT: Also, it seems like when I remember restoring a browser session after a crash, all the tabs get reloaded, not restoring the page state, just the tabs/URLs (ie. browser makes new request(s) for each restored tab). I usually use Chrome though, not sure if this has happened to me on Firefox.
Yoric pointed out here that not being able to save tabs individually is a weak point of the current implementation.
The problem is that it's happening continuously. Any page with analytics or ad engines (I'd guess easily 95% of the top visited web) is continually logging how long you've been on the page, scroll position, idle time, etc. And then setting a cookie to remember the session info/last time it was updated.
It would be interesting to see what installing an ad blocker would do to these numbers.
There is some element of slippery-slope. People say, "So what if your web browser uses 8GB of RAM for six tabs, you've got 16GB!". But I'd argue if every piece of software squandered RAM with such abandon, your machine would quickly choke.
It's also got some good shock value, like the one-paragraph web article that overloads your phone & generates 75MB of traffic.
I still think the worry about it wearing out an SSD is overblown. The 20GB per day of writes is extremely conservative and mostly there to avoid more pathological use cases. Like taking a consumer SSD and using it as the drive for some write heavy database load with 10x+ write amplification and when you wear it out demand a new one on warranty.
Backing up the session is still sequential writes so write amplification is minimal. After discovering the issue I did nothing and just left Firefox there wearing on my SSD. I'll still die of old age before Firefox can wear it out.
Once I noticed that excessive writes were occurring, it was easy for me to identify FF as the culprit in Process Hacker but it took much longer to figure out why FF was doing it.
I actually use a tmpfs for a few things:
$ grep tmpfs /etc/fstab
tmpfs /tmp tmpfs nodev,nosuid,mode=1777,noatime 0 0
tmpfs /var/tmp/portage tmpfs noatime 0 0
tmpfs /home/zx2c4/.cache tmpfs noatime,nosuid,nodev,uid=1000,gid=1000,mode=0755 0 0If I want to save something, I'll download it. If I want to come back, I'll bookmark it. Other than those two cases and settings changes, all of which are triggered by my explicit choice & action, it really shouldn't be writing/saving/storing anything. Would be nice if there were a lightweight/portable/'clean' option or version.
When I tried Spotify, it was pretty bad about that too - created many gigabytes of junk in the background and never cleaned up after itself. I made a scheduled task to delete it all daily, but eventually just stopped using spotify.
If it's genuinely receiving new data at this rate, that's kind of concerning for those of us on capped/metered mobile connections. The original article mentions that cookies accounted for the bulk of the writes, which is distressing.
If it's not, using incremental deltas is surely a no-brainer here?
Might be surprisingly hard or inefficient given that you're trying to capture "webpage state". Think about React sites etc.
Just another firefox ssd optimization.
Edit: And see bernaerts.dyndns.org/linux/74-ubuntu/212-ubuntu-firefox-tweaks-ssd
It talks about sessionstore.
Even if some data is being written, it could still be orders of magnitude lower than the writes executed by the program.
There are legitimate pros/cons of using sync() or not. Missing it out could mean that the file data is lost if your computer crashes. But if firefox crashes by itself, the data will be safe.
Basically chattr +i on a whole bunch of its files and databases, and everything's fine again...
Chrome was also pretty crappy in 2003. /s
It kicks in about 10-15 seconds after initial launch of the browser, and basically churns it to a near halt for at least 30 seconds before it becomes usable.
Could be that this tablet of mine provokes the behavior by having a slow emmc that is nearly full, but it has resulted in me minimizing my use of a browser i have been using for close to a decade across various computers and operating systems.
And no, it has nothing to do with extensions. I still see it with no extensions active. Heck, i even see it with guest mode active.
If you can grab an adb log, come post it in #mobile on irc.mozilla.org.
Grabbing a adb log may be a bit out of my league though.
Disclaimer: I dual boot (camp) windows 7 on my mac.
Windows file compression cookies.sqlite => 1MB => 472KB sessionstore-backups => 421 KB => 204 KB
Move TMP cach folder to ram drive ie ImDisk
The Rust part of your comment is irrelevant. There is Rust code shipping in Firefox, but nowhere near the amount needed for observable changes in behavior. Work is being done to put larger components in but nothing is shipping yet. Also, IME most crashes are asserts (in all browsers) and that's not a problem Rust fixes. If you get a segfault-based crash please report it, it could be exploitable. (report normal crashes too, really)
Note that this means that tabs will share content processes, so a crash in one tab may crash a few others. You can reduce this by ramping up the number of processes, though that can lead to bloat.
No idea how stable this is right now. For me, on Dev Edition with regular multiprocess mode enabled, things work fine and I don't get crashes.
Anyone got ideas on that?
I just built a new PC with SSDs, and switched back to Firefox. Even with 16GB of RAM on an i3-2120, Firefox still hiccups and lags when I open new tabs or try to scroll.
This new issue of it prematurely wearing out my SSDs will just push me to Chrome. Hopefully it doesn't have the same issues.
I need to find time to experiment with Chrome while blocking all the Google chatter to see how functional it remains without being able to tattle to the mothership (otherwise I won't use it).
I'd imagine someone like the above poster would be concerned about that. At least I know I am
Maybe moving this folder to a HDD should suffice.
So I think the session restore is almost always used in a case where firefox crashed, but the system is fine.
> failed to read the very first recommendation on every single guide for ssd life: use ram disk cache for browser temp files.
yeah, let's upvote this
Firefox really started to annoy me with its constant and needless updates a few months back; the tipping point being breaking almost all legacy extensions (in 46, I believe). This totally broke the Zend Debugger extension, the only way forward would be to totally change my development environment. I'm 38 and now, and apparently well beyond the days when the "new and shiny" hold value. These days I just want stability and reliability.
Firefox keeps charging forward and, as far as I can tell, has brought nothing to the table except new security issues and breaking that which once worked.
I haven't updated since 41 and you know what, it's nearly perfect. It's fast, does what I need it to do, and just plain old works.
Firefox appears to have become a perfect example of developing for the sake of.
source: http://www.ghacks.net/2008/07/09/change-the-session-store-in...
I'm holding out hope that whoever's responsible for Firefox abandoning "good, light, cross-platform web browser" as its differentiator stays away from Servo, at least for a while. That's a niche that's unserved currently, AFAIK.
If you're intentionally running very old versions which by now have published exploits, then I can see that these are not things you would consider important.
I'm now running the latest version of FF, but still able to use my plugin.
It's exactly the opposite of this. Have you read through the FF update notes? The web standards are a moving goalpost - constantly evolving.
On top of that they've been slowly landing incremental improvements to Electrolysis that have allowed them to turn it on for more and more users without big risks to stability. You can't do that for a browser with "big bang" releases every year.
In the last years I think it's been shown that the only model that works for browsers is "evergreen".