Hacker News Reader

Comment by raverbashing | Hacker News Reader

"Everyone must now be careful never to store a file in /tmp that might grow large"

If you're going to trash my /tmp with gigantic files please don't. Really

"it’s better to fix the filesystem to make it faster"

Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place

So basically, he contradicts himself.

Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks.

View on HN

>If you're going to trash my /tmp with gigantic files please don't. Really

I write statistical programs that often generate temporary intermediate files of multiple GB size, which need to persist at the end of runs for trouble-shooting, partial re-runs etc. i.e. the software shouldn't delete them when it's finished, but they don't need to be kept long term. I kept forgetting to delete them when I finished a session and my disks kept filling up, so I decided to use /tmp and let the OS manage the space. Is there a better way to manage large tmp files than this that I'm missing?

Given that they should persist, the better way would be /var/tmp I would think, at least if persistence means "should survive reboot".

Even before shifting to tmpfs I'm pretty sure Fedora was erasing the contents of /tmp at startup (and I know Gentoo/OpenRC is).

There might be an even more appropriate FHS-compliant directory (/var/run perhaps?)

Apologies, by persist I meant just beyond the end of that run of the software (i.e. the software itself couldn't manage/delete the files), to see what went "wrong". Deleting at reboot is perfectly fine.

/var/run "This directory contains system information data describing the system since it was booted" which doesn't seem to fit my use case that well. Not to say there isn't another directory that does fit, but I haven't found it.

Depends on your use case and how much of that complexity you wish to manage

In your case, what could work is having the new run use the same file names as the old run, or coming up with a rotation schedule (so you would keep the latest N runs)

But if having them on /tmp as it is working for you, great, no need to complicate things.

There are also speed considerations, if you're using these files as actual intermediate storage (as opposed to a log/debug dump) it may be interesting to have them written to a different disk (like an SSD)

In my case, the runs are often run in parallel and in batches of varying sizes, so "same name" or rotation schedules wouldn't work well. They are a kind of hybrid log/storage file (they contain intermediate data sectioned/logged into the actions that created them, like an extremely verbose log). Dedicated SSDs would be very nice, but out of the price budget (i.e. its a zero cost project, run on existing scavanged hardware!).

As you say, /tmp is working well for me, my example was to perhaps illustrate that there are (what I think are) legitimate uses for large files in /tmp.

Sounds like a job for /var/cache.

> Mounting tmp as a memory backed fs is great (to which there may be better options than tmpfs), avoids (potentially) spinning up disks covered in rust for minor tasks.

I mount /tmp as a tmpfs for almost exactly this reason. I have an always-on home server/HTPC that runs the OS off of a small (40 GB) SSD, and I don't want to wear down the SSD with writes to /tmp, /var/tmp, /var/run, /var/log, etc. In theory, I don't want any writes at all, unless the thing is actually doing something useful. So I simply threw in 2GB extra RAM and mounted all these temporary/logging directories as tmpfs. Periodically (and on reboot) I zip the logs up and copy them to a backup directory just in case something fails.

Of course this runs the risk of losing temporary data and logs whenever the server goes down, but that's a trade-off I'm willing to make. It's been running just fine for over 2 years now. I'd do the same thing straight away if I ever need to set up some similar system.

Yep - make your box do what you want.

Nothing stops you from changing fstab to make any mount tmpfs or disk.

The /tmp on your system is not 'your' /tmp. The home directory is yours. /tmp exists pretty much solely as a place for programs to fill up with a mess of files, with the understanding that it will get automatically cleaned out and they aren't messing up the system proper. If a program needs to trash a place with large files, /tmp is where it should do it.

> If you're going to trash my /tmp with gigantic files please don't.

Where else are temporary files supposed to go, exactly? It's no abuse to put in /tmp temporary "scratch" data which have no reason to survive a process restart let alone a machine reboot, that's the mount point's explicit purpose.

>Yes, let's spend adding more complexity to the file system to work out every /tmp abuser out there. But his original point was avoiding adding complexity in the first place

Better to add complexity once, in the filesystem, than in every program that writes to it.

Reply on news.ycombinator.com