Hacker News Reader

Comment by snewman | Hacker News Reader

I worked on the original (long since superseded) implementation of the metadata store for Google Drive, i.e. the system which was responsible for tracking file / folder relationships. The requirement to allow an item to appear in multiple locations was a huge complication, in part because of the way it interacted with permissions being inherited from a folder to the items in that folder. I imagine this change may be motivated by a desire to move away from that complex model, and whatever team owns that system now may be very happy to see it going away.

(IIRC, the requirement stemmed from the need to support the various applications that were being folded into / integrated with Google Drive, such as Photos which of course allows a photo to appear in multiple albums.)

View on HN

Googler, opinions are my own.

This was my understanding as well. The original Drive was built effectively as a directed graph (with cycles allowed). Any file or folder could be stored in multiple locations. And permissions were at a per-file basis, so 2 people viewing the same folder may see different sets of files.

And permissions were definitely a hard part of it, as if you applied new permissions to a folder and all children, it had to walk the entire graph to update the permissions.

This is the advantage of the Team Drive style structure that the Drive team put out. It follows the classic filesystem design of a tree, which allows for easier permissions modeling, among other things. It's also why all "hard links" are now becoming shortcuts / Soft-links.

Forgive my ignorance (and there's a lot of it), but navigating a graph and setting permissions doesn't seem like a terribly difficult problem, especially for Google - the king of graphs, in a word. I would think maybe the issue has to do with how permissions of a file/folder apply to different users, but then isn't that just exactly the raison d'etre of permission systems?

This isn't an area I'm qualified to have a technical opinion on but if you'd care to elaborate I'd be interested to learn more.

Googler opinions are my own. I've never worked on Drive.

It's not hard, but expensive.

If you have a file system where it is a proper tree in any given node within that tree only has One parent, and walking up all the parents you always only have one parent up to the root node, you gain some useful properties from that. You can build a permission model where you just have to check the permissions of all of your parents to see if someone has access to a given file deep within the system. This means that if you Grant Reed access on any node within that line of parents, all children implicitly get access.

Now imagine a directed graph, and you set permission on a node and you want to say all connected nodes also should get this permission. You could definitely walk all the nodes and apply that permission, but now you have to write said permission to every node, rather than being able to set it on a parent and let all children inherit it. Even if you know all of your parents, then all of those nodes have multiple parents. And you can have cycles. Checking parents for permissions becomes way more complicated for users to understand, which is why Drive likely didn't do this.

Can you explain why team drives (now called shared drives) have a limit of 400.000 files while there are no limits and unlimited sharing for files in "my drive"?

I worked on another sync client's representation of filesystem structure, and came to the same conclusion. Hard links enable some cool behavior, but in retrospect added more complexity than anyone expected. Migrating to shortcuts / soft links seems very reasonable - I wish I had started there.

What kinds of cool behavior do hard links enable?

Every file is essentially the master pointing at the same inode. So the file remains as long as any of the hard links exist. With soft links if you delete the master file all the soft links stop working.

Apple's Time Machine uses hard-linked directories to replicate your home directory again and again, but with only the changed directories/files actually taking up space. The unchanged directories are just hard-linked to the previous versions, so any one version looks like your entire drive, but it takes much less space than full copies.

Used to, the new APFS implementation uses snapshots.

> various applications that were being folded into / integrated with Google Drive

The Photos/Drive integration was removed a long time ago. What other integrations were behind the original requirement? I'm curious to know if the extra complication was worth it in the long run and how long the integrations that needed this feature hung around for.

I don't specifically remember whether there were other motivations for that requirement. Possibly a desire to support "tag" style interfaces, where an item can have multiple tags applied. But it may have mostly been the Photos integration.

And no, definitely the complication was not worth it in the long run. :-) That project got way more complicated than anyone wanted (not that there's anything unusual about that).

I actually remember early Google Drive to have only tags for files, and no folders. Apparently, the idea was to distance from the file tree paradigm, but eventually the traditional approach proved to be more user friendly.

So is this basically a switch from hardlinks to softlinks?

It's a switch from inference to direct collection of intent from the user.

How so? Are hard links still allowed?

Yes.

Reply on news.ycombinator.com