Hacker News Reader

Comment by satoshiiii | Hacker News Reader

Even in paid shared web hosting services, there is an inode limit. And it was more than a decade ago since I stumbled with this issue. Since then I monitor my file count and delete those with last TOUCHed of at least 30 days even if I have moved already to a dedicated server long time ago. I was caching complex database queries to the disk and did not bother to delete the files before.

I wonder if there is still a limit with modern Linux OSes and that is why Google is limiting some of their users?

View on HN

I think Google Drive is abstracted enough that they are not running into something basic like an inode limit...

But this does remind me of having to set up a cron job on an old PHP application because the sessions were saved as files and it ran into the inode limit several years in :P

And, of course, 5MB isn't some power of 2 (as most limits are). Its hard to imagine where Google got the number unless its just some hardcoded constant.

They most likely hit some sort of limit in the Zanzibar implementation.

Every file has a set of permissions attached to it, because you can grant fine-grained permissions to every single file in Drive, Photos, YouTube, ...

I imagine that they hit some limit in that part, and tried to find a way to limit the issue...

>I wonder if there is still a limit with modern Linux OSes and that is why Google is limiting some of their users?

Ext4 has 32-bit/~4billion max limit per filesystem, newer filesystems like btrfs have a 64bit limit. Neither allocates the whole max size by default. I don't imagine Google drive uses an out of the box filesystem underneath though.

Btrfs is particularly special as the inode limit is per subvolume, so it's quite easy to have an impractically huge number of them with some cleverness.

> I wonder if there is still a limit with modern Linux OSes

Since filesystems introduced dynamic inode count it hasn't really been a problem. I remember ages ago you had to format filesystem in certain way if you wanted a lots of small files (say news or email server), but the need of that went away with better filesystems and growing storage sizes.

> and that is why Google is limiting some of their users?

On their side it's most likely on the number of entries in database, which usually is more than just storage cost.

It seems that there is no hard limit with ext4. It depends by inode size, disk size, etc. I searched a little and I didn't get any deeper than that. Anyway, df -i reports this about my 2 TB SSD

     Inodes   IUsed     IFree IUse% Mounted on
  122068992 4724107 117344885    4% /

I guess that 122 M inodes in the first column is the limit of that file system. Much more than 5 M but Google Drive is probably giving a quota to users. No idea of which file system it actually uses on the disks.

Reply on news.ycombinator.com