Tmpfs considered harmful
rwmj.wordpress.com
rwmj.wordpress.com
I believe there was never such expectation of /tmp. On many systems it's being wiped periodically or even when the user's last session ends. The example app(s) that use /tmp for drafts need(s) to be fixed.
The memory size issue is basically a non-issue at this point (pair of desktop 16 GB DIMMS are ~$80 and laptop isn't that much more).
Secondly, no I don't think it's broken. But it doesn't in any way affect the fact that as a last resort I can reboot in single user mode and grab the precious mutt file before it gets deleted.
/tmp is simply RAM that has the API of a fileFirst time I've heard about /var/tmp.
It's not just systems that delete on reboot, but also systems set up with cron-jobs to regularly wipe /tmp, or sysadmins that will (rightfully) see /tmp as something that can be wiped at pretty much time.
Although I agree with your points, I find that the design of one application alone cannot be a justifiable point.
I've never had a problem with this using mutt, but if I cared, I'd switch it to use /var/tmp.
Either way, if there's a problem, it's with the program, not the filesystem - no need to use a cannon to kill a fly.
If you're going to trash my /tmp with gigantic files please don't. Really
"it’s better to fix the filesystem to make it faster"
Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place
So basically, he contradicts himself.
Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks.
Better to add complexity once, in the filesystem, than in every program that writes to it.
I mount /tmp as a tmpfs for almost exactly this reason. I have an always-on home server/HTPC that runs the OS off of a small (40 GB) SSD, and I don't want to wear down the SSD with writes to /tmp, /var/tmp, /var/run, /var/log, etc. In theory, I don't want any writes at all, unless the thing is actually doing something useful. So I simply threw in 2GB extra RAM and mounted all these temporary/logging directories as tmpfs. Periodically (and on reboot) I zip the logs up and copy them to a backup directory just in case something fails.
Of course this runs the risk of losing temporary data and logs whenever the server goes down, but that's a trade-off I'm willing to make. It's been running just fine for over 2 years now. I'd do the same thing straight away if I ever need to set up some similar system.
Nothing stops you from changing fstab to make any mount tmpfs or disk.
I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session and my disks kept filling up, so I decided to use /tmp and let the OS manage the space. Is there a better way to manage large tmp files than this that I'm missing?
In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs)
But if having them on /tmp as it is working for you, great, no need to complicate things.
There are also speed considerations, if you're using these files as actual intermediate storage (as opposed to a log/debug dump) it may be interesting to have them written to a different disk (like an SSD)
As you say, /tmp is working well for me, my example was to perhaps illustrate that there are (what I think are) legitimate uses for large files in /tmp.
Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is).
There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)
/var/run "This directory contains system information data describing the system since it was booted" which doesn't seem to fit my use case that well. Not to say there isn't another directory that does fit, but I haven't found it.
Where else are temporary files supposed to go, exactly? It's no abuse to put in /tmp temporary "scratch" data which have no reason to survive a process restart let alone a machine reboot, that's the mount point's explicit purpose.
I guess it only takes one silly person to connect "tmp" in the name to "/tmp", and the rest is history.
debugfs is mounted on /sys/kernel/debug
securityfs is mounted on /sys/kernel/security
So tmpfs is mounted on /tmp? Can see that...
I get that the guy want's attention for this life-and-death issue, but linkbaiting is just mean :(
Having it sit in RAM is great, because I get the speed without subjecting my disk to unnecessary writes - for certain operations, this can be a huge bonus.
Is there some better place than swap to accommodate /tmp?
One simple benefit is that files there do not get caught in backups - they are temporary detritus so I don't want backup space wasted by them.
The major benefit is that fsync is really slow on Linux. Typically it turns into a sync, and a lot of programs like to ensure you don't lose data by being fsync happy. For temporary stuff that is even more painful - you don't care about data loss - that is why it is in tmp in the first place.
TLDR: I prefer filesystem operations on transient/temporary files and data to run at the speed of RAM, and not end up in backups or waiting for syncs.
The big trouble for me would be "Everyone must now be careful never to store a file in /tmp that might grow large". I often use /tmp for all kinds of temporary storage. Be it a small text file or a GB big tar file.
I remember CD burning softwares and torrent clients asking for special temp directories to be used for that.
Allowing every day applications to create GBs of tmp files would mean you would need to worry once your partition has less than 100 GB free.
However, nobody stops you from mounting /tmp anywhere else or even disable tmpfs.
Not quite. There are many much more difficult problems in a competitive optimizing compiler before hitting that step.
You can get reasonable results from even a linear scan allocator, which is simple.
Current x86 implementations do a good job of being fast even when registers spill.
I've seen dual-issue Power cores (32ish registers) that only have 16k of 4-way dcache, which means register spills can easily kill performance.
Apps are already in the habit of writing a lot of expendable garbage to /tmp that doesn't require the fast I/O of tmpfs. Clogging RAM with this junk is bad idea.
/tmp This directory contains temporary files which may be deleted with no notice, such as by a regular job or at system boot up.
Be warned that when he says this, he doesn't mean it will eat all your memory, it will only grow upto a maximum size (by default half of your ram). When this is reached, it acts like any other filesystem that is full: you get write errors. There is actually another implementation called ramfs, which doesn't have such a size restriction. However, tmpfs is what is actually being implemented here. The ramfs itself is more of a 'toy' filesystem, also being quite limited, including not being able to use swap space (unlike tmpfs).
The Linux FHS explicitly states "Programs must not assume that any files or directories in /tmp are preserved between invocations of the program." That's not only reboots. It's not meant to be used to store drafts or anything that you might want to reopen at a later time. That's what we've got /var/tmp for. The FHS specifies as follows: "The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots. Therefore, data stored in /var/tmp is more persistent than data in /tmp".
Tmpfs isn't harmful because of this, it's the default mutt configuration which is broken. If developers would have adhered to the standard the problem wouldn't exist. Since programs should adhere to the standard, because decisions like the one in question are made upon it, they should be reviewed and their default configurations should be changed in order to fix this issue.
User browsing the web for a month without rebooting. Flash stores temporary files in /tmp/. For a month straight, Pandora is pumping out songs to /tmp/ as a cache. 43829 minutes in a month, imagine an average of 3 minutes per song that's 14609 songs, at an average 1.5 megabytes per song that's 7304 megabytes that's 7.13 gigabytes. Probably shouldn't be stored in RAM.
On the other hand, you have ganglia clusters of about 2,000 hosts. You have multiple gmetad's collecting data from different data sources and pooling it into your web interface box. It's updating ~10,000 tiny files every 30 seconds. Your i/o wait and system time is through the roof. You move the place gmetad writes to a tmpfs mount, and rsync to the disk every minute. Suddenly the i/o and system times are at 0.01%.
Tmpfs for /tmp/ is considered harmful, but extremely helpful in other cases.
As long as that is true, it's not a good idea to use tmpfs for /tmp.
It's perfectly reasonable, however, to use tmpfs for, say, /var/local/tmp and let individual users point $TMP at that. Want to live fast and dangerously? Set $TMP.
As an amusing example, the SHM (shared memory) system in linux is actually implemented on top of tmpfs (/dev/shm is a tmpfs mount).
Other uses include CoW shadow mounting (when unioned to an immutable filesystem).
That said, the point that /tmp should not be a tmpfs mount is actually a decent one, just because we're used to the idea that /tmp is neither space-constrained (it's scratch space) nor extra performant.
IO bound becomes CPU bound so you're getting as good speed as you can get.