Dot Dot Considered Harmful | Hacker News Reader

Dot Dot Considered Harmful

fuchsia.googlesource.com

345 pointsrohan1024173 comments

>If a handle is provided to a directory, it should imply access to resources within that directory (and additionally, their subdirectories).

Maybe within Fuschia, but not in my UNIXy systems. All users on a system need r-x on /Users, but that doesn't give them r-x on every user's home directory located as a subdirectoy of /Users - a handle acquired by bob to /Users does not imply access to /Users/delinka.

Further, if your process is treating '..' as a subdirectory, you're doing it wrong. Paths must be normalized (e.g. ~ expanded, . and .. resolved) before requesting a handle via absolute path.

Lastly, this document reads as if knowing a full path grants access to that path and its subdirectories. If that's the case ... oy.

NTFS has a toggle bit for access inheritance: http://www.ntfs.com/ntfs-permissions-explicit.htm

Not just NTFS; it’s a common feature in systems that support NFSv4 ACLs, like Solaris or FreeBSD.

The answer to that for a capability-based system would be to not grant a process access to /Users, but instead give it an opaque handle that grants access to /Users/delinka. It's definitely not how unix systems work (where you need read access to all of the parents to access a child directory), but in a capability-based system it makes sense IMO.

You don't need read access to all parents (on Linux and BSDs at least) -- a privileged can pass a dirfd to a less privileged process and that process can access paths under that dirfd without any permission checks being done for parent directories of the dirfd.

This is correct. But this is not traversing a path, or even knowing it.

The comment was refuting the claim, "where you need read access to all of the parents to access a child directory."

You don't need read access, only execute access.

> Further, if your process is treating '..' as a subdirectory, you're doing it wrong. Paths must be normalized (e.g. ~ expanded, . and .. resolved) before requesting a handle via absolute path.

".." is required for atomic traversal (similar to symlinks), which is important in some situations, such as making sure a file tree you've descended isn't removed out from under you in a way that breaks your state. (The directory itself can be moved while you hold the handle, but the important thing is that being able to rely on ".." permits certain algorithms that are safer and more consistent.) Canonicalizing paths introduces race conditions between canonicalization and actual access, which is why this is performed in the kernel.

Canonicalizing paths makes sense if you're accepting paths from untrusted sources and you cannot make use of POSIX openat() + extensions like O_BENEATH. HTTP GET requests are supposed to be idempotent, anyhow, so there shouldn't exist any sort of race condition as a conceptual matter.

But in regular software, it's better to just pass paths as-is. The shell performs "~" expansion, but it doesn't resolve "." or "..". And the shell performs other expansions, like file globbing, for which there's rarely any reason to implement in regular software. Supporting "~" expansion but not file globbing is inconsistent; if you're not the shell, don't implement shell-like features as it just creates confusion and unnecessary complexity.

Calling ".." a holdout from POSIX is misleading. The semantics exist for legitimate and even security-relevant reasons, albeit not necessarily reasons that an embedded smartphone OS may care about. What POSIX lacks is O_BENEATH (from Capsicum) or a similar flag for tagging a directory descriptor to prevent relative paths or otherwise ascending the tree. Capsicum extensions make POSIX perfectly capable of implementing strict capability semantics, and they do so by extending semantics in a manner consistent with POSIX. POSIX isn't inherently broken in this regard, it's just perhaps too feature rich for some use cases and not feature rich enough for others.

There's no end of complaints about POSIX--either it's too complex or too simple. The fact is, no single specification will ever please anybody, so griping about how POSIX is broken is not only pointless but belies a failure to appreciate the underling issues. The alternative to POSIX is basically nothing. Standardizing on Linux is, at best, a lateral move. Adding optional components to specifications has proven the worse of all worlds, and like most other standards POSIX has been slowly removing some optional components altogether while making others mandatory.

As a complete aside, I'm currently working on a patchset to add O_BENEATH (and some other path resolution restriction flags) to Linux.

https://lore.kernel.org/patchwork/cover/1011459/

Thanks for working to incorporate it in Linux.

FreeBSD has moved to be compatible with the proposed Linux O_BENEATH as well[1]. We're hoping this helps write portable, capability-restricted code between FreeBSD and Linux.

[1]: https://svnweb.freebsd.org/base?view=revision&revision=33974...

(We already had very similar functionality for Capsicum-restricted directory fds to piggy-back off of.)

Yes, though FreeBSD's O_BENEATH would probably have to be done using (O_BENEATH|O_NOMAGICLINKS|O_XDEV). Linus, Andy and Al were very insistent that various parts should be split into separate flags.

> Further, if your process is treating '..' as a subdirectory, you're doing it wrong.

That's the point they make. And in Unix, it's really easy to do it the "wrong" way (like, "type 3 characters" easy), so lots of people do.

Back before protected memory was common on PCs, some people said "If you're chasing pointers without being sure that it points to a valid address, you're doing it wrong". Well, perhaps so, but in practice lots of programs were doing it wrong, and it was the users who suffered.

Capabilities sound to me a bit like protected memory for persistent storage. It'll be a little inconvenient for a little while, and eventually we'll wonder how we ever lived without it.

> "If you're chasing pointers without being sure that it points to a valid address, you're doing it wrong". Well, perhaps so, but in practice lots of programs were doing it wrong, and it was the users who suffered.

Except ".." also is a solution to permitting you to traverse a directory tree without accidentally chasing an invalid pointer; no bug in your program nor any external application can ever make ".." invalid. ".." is equivalent to a back pointer in a linked list or tree structure. Imagine you have a handle to /foo/bar/baz and want to ascend to baz's parent. Between acquiring the handle to /foo/bar/baz and attempting to ascend to the parent, "baz" could have been moved and "/foo/bar" may not exist. Without ".." all of a sudden you're orphaned and your application is stuck in an inconsistent state. Maybe the best solution is to just panic, but that's like saying that all applications should be prepared for any pointer access to segfault at at any moment. That's one solution, and it's actually how some smartphone application environments work. Another solution is guaranteeing the condition can't happen, period. Which is preferable depends on your use cases and which side of an interface you'd prefer to place the burden. For pointers it's fairly obvious which provides the most preferable semantics for maximum safety (or it was until the smartphone and cloud paradigms), but for file systems the answer is less obvious. In Unix when ascending directory trees the scenario of a valid pointer becoming invalid is impossible such that there's always a valid path to the root of the tree (even if the depth can change; you just stop when opening ".." simply reopens the same directory), but for descent there remains the race between readdir + open (as opposed to readdir atomically returning file handles).

These particular semantics were clearly purposeful, not an accident. The former was thought useful, the latter an acceptable simplification. It's why in Unix you can't delete a populated directory in or when a process holds an open reference (unlike other file types), and why you can't hardlink directories.

> no bug in your program nor any external application can ever make ".." invalid

Sure it can, if you consider lack of permissions to be invalid. We already do, in other similar situations.

> Without ".." all of a sudden you're orphaned and your application is stuck in an inconsistent state.

No, it just means you need an out-of-band method to accomplish this.

> that's like saying that all applications should be prepared for any pointer access to segfault at at any moment

No, nobody's talking about crashing. It's more like saying you can't assume you can do raw pointer arithmetic to jump around in an array. Languages like Java and Python feel restrictive to C programmers at first, too.

> Without ".." all of a sudden you're orphaned and your application is stuck in an inconsistent state.

I don't understand these claims of races and segfaults. Doesn't Fuchsia avoid race conditions like this with VFS cookies?

> > Further, if your process is treating '..' as a subdirectory, you're doing it wrong.

> That's the point they make. And in Unix, it's really easy to do it the "wrong" way (like, "type 3 characters" easy), so lots of people do.

The OP is writing about iterating subdirectories and accidentally treating ".." as such, if you use certain low level functions - which is kind of bad practice in most cases anyway.

> That's the point they make. And in Unix, it's really easy to do it the "wrong" way (like, "type 3 characters" easy), so lots of people do.

Yeah, pointing to a wrong directory can lead to unexpected results. On the other hand, if this becomes a security issue, then maybe the process should have properly restricted rights as pointed out elsewhere already.

Talking about "..", one could BTW extend the discussion to mixing Filename characters with path separator characters in the same string. ;)

Sure, you can do it that way; but now that process being run by bob has full access to everything owned by bob (including, for example, `/Users/bob/.ssh/id_rsa`).

Fuchsia considers that level of access to be unacceptably broad for most applications, which is why it uses a capability-based permissions model instead of a user-based one.

It does on Ubuntu. /home/* is readable and executable to everyone.

Ubuntu, with default configuration, does create new user directories as globally-readable, but that's just a poor security decision on their part. There's nothing special about /home itself; it's just the default behavior of adduser. And the permissions on /home have no effect on what the permissions are on new subdirectories created when a user is added.

Indeed, this ship has sailed. And if .. did not exist then chdir(2) would be the same as chroot(2) unless knowing an absolute path was enough to allow you to access it (assuming --x permissions on the path's dirname's directory components) then, yeah, you wouldn't gain that much as many paths can be guessed.

There just isn't a short-cut for making sandboxes trivial to setup.

I really wish that Solaris/Illumos Zones were standard on Linux. You could have really light-weight containers as anonymous/ephemeral zones whose "init" is the program you want to sandbox, and more heavy-duty guest-like containers as Zones already is.

The difference between Zones (or BSD jails) and Linux containers is that with Zones (jails) you have to explicitly decide what to share to the zone, while with clone(2) you have to be explicit about all the things you DON'T want to share with the container. I.e., Zones requires white-listing while containers requires black-listing, and we all know that black-listing doesn't work as a security device. Granted, the kernel developers could have forgotten to virtualize something important, but when they fix that you don't have to modify and rebuild the zone/jail launcher.

> unless knowing an absolute path was enough to allow you to access it

If understand correctly in fuchsia "absolute path" is always relative to a filesystem handle so knowing it and being able to use it are pretty similar

Ok, that works, though you pay a price: you have to keep track of a fair number of such handles. You'll need one for /usr/bin, and for /bin, and all the lib and libexec and share and varstate directories, and /etc. You do get to not let processes see $HOME if you don't want to, and that's very nice.

In a shell one would have to expose a path->handle dictionary for scripts.

You mean key->handle dict? Most environment variables on my windows seem to be what you are describing, except currently being path strings instead of file system handles.

I think you're confusing namespace and unix file permissions.

You can think of capability-restricted directory descriptors as (sort of) individual-fd chroots. File permissions still apply inside a chroot. But the namespace of anything outside the chroot is totally inaccessible.

> Lastly, this document reads as if knowing a full path grants access to that path and its subdirectories. If that's the case ... oy.

Well, they speak of the path as a "resource provided to a subprocess". In that context, it sounds more like a handle/file descriptor that the child process can pass to some "read", "write" or "get handles of children" syscalls - and that happens to correspond to the file object at /home/bob/foo.

If so, it wouldn't imply that knowing (or guessing) the string "/home/bob/foo" would automatically give you access to the handle.

That's just my reading of our though, no idea of that is what they actually do.

* our -> it

Lastly, this document reads as if knowing a full path grants access to that path and its subdirectories. If that's the case ... oy.

Indeed. When I read...

As a consequence, this implies that a handle to a directory can be upgraded arbitrarily to access the entire filesystem.

...I was wondering whether the author even knows what filesystem permissions are and how they work. I say let the filesystem handle resolving relative paths; and let the permissions system handle the check on whether one is allowed to access the referenced object.

User centric permissions are too broad. Classic example: you might not want your browser to ever access your own ssh private keys

The topic here is to let a user start a process and pass a restricted view of the file system to that process which in turn can spawn child processes to which it could restrict access even further. In order to make it possible to do useful work it's sometimes necessary to also pass around handles/filedescriptors between processes (possibly within different sandboxes) and it's a good idea that the rules governing the view narrowing are not broken.

Rob Pike apparently regrets all the "." file naming.

https://plus.google.com/u/0/+RobPikeTheHuman/posts/R58WgWwN9...

This comment made me realize that there are a lot of really good posts on Google Plus by people like Rob Pike, Greg Kroah-Hartman, Linus Torvalds and others. For whatever reason, systems programmers liked Google Plus.

Anyway, I hope someone is archiving them somewhere, because there's a lot of knowledge there that will otherwise be lost in a few months.

I always wondered about the "dot files" convention. That's a really good story! If I knew there where posts like this one I would join in a heartbeat. Where can I find communities like this today !? I always seem to be too late ...

My immediate thought when they announced they were shutting down was "What about Linus's posts?!"

Google Plus played less with people's emotions, that's why it sucked as a social media but was quite okay to follow content makers and to manage/participate in groups.

Isn't he talking about something different, though? Specifically the convention that file names starting with "." are hidden?

Yep, nowhere in the post does he suggest regretting chosing "." to denote the current working directory.

Reading that post indicates that he regrets the existence of hidden files, not the existence of '.' and '..'

Yes, that's right. He does have issues with how ".." works in Unix though. There's a pretty detailed explanation from him here: http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames (scroll down to about the third appearance of "..")

Yes, though Plan9's implementation of ".." Is incredibly different to Unix. It's treated as a lexical token rather than an actual directory entry, and so therefore certain strange cases with symlinks are handled differently (though Plan9 doesn't have symlinks either). From memory Rob Pike had a blog post about this problem and why Plan9 does lexical path cleaning on all paths before resolution.

I had no idea .. wasn’t treated as a lexical token in POSIX. Why the hell is it not and what would break if it suddenly was? Also, why do you need a new syscall to do it when it seems like string manipulation is all that’s necessary?

The most obvious example of something that would break is any code that uses symlinks. Here's an example:

  % ln -s foo/bar/baz link
  % ls ./link/..

This resolves in POSIX to "./foo/bar", not "." (which is how it would resolve in Plan9 if Plan9 had symlinks). If you treated ".." as a lexical token you'd have to sanitise it before doing anything else and thus you'd end up with Plan9-like semantics. You could also make the argument that Plan9's ".." semantics actually cause inconsistencies between "cd $path; ls .." and "ls $path/.." -- and this is especially true for Linux's "magic links" in procfs. So if you prefer a consistent VFS then making ".." a lexical token is not the best idea.

> Also, why do you need a new syscall to do it when it seems like string manipulation is all that’s necessary?

I'm not sure what you mean -- Plan9 is a separate operating system which does the above path sanitisation with each path-related syscall.

I think you'd have to treat `.` as a lexical token as well. So parsing `./link/..` would first resolve to `/home/me/link/..` then '/home/me/`, no?

> I'm not sure what you mean -- Plan9 is a separate operating system which does the above path sanitisation with each path-related syscall.

From http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames

> A new kernel call, fd2path, returns the file name associated with an open file, permitting the use of reliable names to improve system services ranging from pwd to debugging. Although this work was done in Plan 9, Unix systems could also benefit from the addition of a method to recover the accurate name of an open file or the current directory.

> I think you'd have to treat `.` as a lexical token as well. So parsing `./link/..` would first resolve to `/home/me/link/..` then '/home/me/`, no?

You could, but that doesn't really change the point of my comment -- "." is very trivial to handle either lexically or as a directory entry and generally Linux's handling of it is basically a no-op (though it should be noted that it's a no-op compared to $path/ not $path -- which is an important distinction with symlinks-to-directories or mountpoints).

The key difference is whether the symlink is actually resolved (and thus ".." applies to the partially-resolved prefix of the path resolution) or if it applies to the symlink component itself (and thus it never gets resolved).

> A new kernel call, fd2path, returns the file name associated with an open file [...]

fd2path goes from fd -> path, which is the inverse operation to path resolution. All path resolution in Plan9 goes through cleanpath() (as far as I know). fd2path is similar to readlink(/proc/$pid/fd/$fd) on Linux.

The two things are separate concepts.

I am starting to see why symlinks screw everything up. I guess I come at it from the URI standpoint: `https://example.com/foo/../bar/` should be resolvable client-side to `https://example.com/bar/` without having to worry about what's on the server, right? So I'd say the path should be processed the same way: without looking at the filesystem first. I'd advocate that the `/../` should be applied before the symlink is resolved because when I am in `/home/me/bar/` and I say `cd ..` I mean "go to `/home/me`" not to some arbitrary part of the filesystem.

Actually, in the case of "cd .." shells will often try to fake how symlink resolution actually happens. You can see this if you "cd" into a symlink, you shell's $PWD and built-in pwd command will tell you that you're inside the symlink but not the actual directory you got thrown into. This results in "cd .." actually doing what you describe -- even though it's a complete fantasy invented by your shell. Try it out.

Your comment is a bit misleading. He is regretting introducing a bug which resulted in creation of hidden files. He isn’t regretting using . and ..

> this is an essential idea encompassing microkernels, and other “capability-based” systems

Tiny nit, but microkernels don't imply a capability based security model. For instance Mach, QNX, Redox, etc. aren't capability based.

It's a very good idea for your microkernel to be capability based because it cuts a lot of validation out of the critical path for IPC, but it's by no means a requirement.

Mach is capability based. (Mach ports are capabilities.)

You're totally right of course.

I was falsely under the impression that the Mach port table only had a single global namespace.

This may be off topic, and is hard to ask without doomsaying: but can we trust Fuchsia? Google is a machine for turning user's personal data into their dollars, and has been getting more and more crafty at achieving this goal.

I want to believe that this is a good-natured effort at improving the state of modern operating systems; but I feel like I've been burned by my trust in Google too many times.

> I feel like I've been burned by my trust in Google too many times

I'm genuinely curious about this. In what way do you feel burned?

I think I get what you're saying. I use gmail and Google uses an algorithm on gmail to decide what ads to show me when I do a search. The ads make money for Google and I don't get any of that money. All I get is free search results. I think what you're saying is that because Google is using your information to show you ads and make money that somehow you feel cheated?

But I don't feel particularly burned by this arrangement. If Google gave an option to start charging me money to avoid ads if I'm honest with myself I'm pretty sure I'll choose to continue seeing ads instead. I guess I'm curious about exactly how you feel burned.

Probably the most obvious example is when Google almost destroyed the entire RSS ecosystem with Google Reader.

I'd argue that competitors flourished when Google Reader went down. I still use RSS today.

Besides, it's certainly not their responsibility to maintain a service indefinitely.

It was a close call. For a number of months there it looked like RSS was going to be well and truly dead.

The fact that my comment is being downvoted tells me that people here have a shockingly poor memory for this sort of thing.

And heck, it's not even the death of Google Reader that was bad for RSS. The life of Google Reader was bad for RSS too. It was a "good enough" product released for free, as in good enough that it wasn't worth trying to compete with a free product so nobody did, but it wasn't actively maintained and just served to cause the entire RSS ecosystem to stagnate for years.

Bit of a no-win situation if they're the bad guy for both maintaining and not maintaining the service. :)

Not at all. They could have actually put effort into it. The problem was that they didn't maintain it. They let it languish, staying exactly the same for years, good enough that nobody wanted to compete with a free product but not actually doing anything to improve the RSS ecosystem, so pretty much nothing happened with RSS for years besides it just existing. And then they decided to just shut down Google Reader with a relatively short time frame, because they never figured out how to monetize it.

They had 3 perfectly good options:

1. Don't build Google Reader in the first place if they weren't interested in actually maintaining the damn thing.

2. Put some effort into it, keep improving Google Reader, make the whole RSS ecosystem better rather than causing it to stagnate.

3. Sunset Google Reader over a much longer period of time, like a year instead of the 3.5 months they gave. Those 3.5 months were just barely enough time for people to build replacement services.

>The problem was that they didn't maintain it. They let it languish, staying exactly the same for years ...

That's what maintenance is though. Keeping something functional, but not developing it further.

>... because they never figured out how to monetize it.

That's not true. They shut down Reader because the codebase was dated, and there were few engineers left on that team. It was a reallocation of resources.

>Sunset Google Reader over a much longer period of time, like a year instead of the 3.5 months they gave.

That I agree with.

Maintain has multiple accepted usages. But I'm not interested in playing word games.

> They shut down Reader because the codebase was dated, and there were few engineers left on that team. It was a reallocation of resources.

And why did they reallocate resources? Because they never figured out how to monetize it.

RSS is essentially on life support. every website that supports it hides the functionality a couple of clicks deep(where newbs won't think to go) whereas pre-google reader it was front and center

RSS feeds are linked in meta tags. You simply point it at the frontpage (or blog) of the site and it'll find it. There doesn't need to be a visible RSS button on the page.

What about when it turned out Google was tracking users second-by-second location data, even after users had removed consent and turned off locations?

I dislike Google's data slurping practices, and moved away from Gmail and other of their services years ago due to mistrust, but I think that was just an honest documentation bug. The button was to disable only the Location History feature, but the description made it sound like it was a global setting for the whole device. As a developer, I could very well see myself making that mistake; being embedded in the context, it's easy to miss how it will be read by someone else.

> In what way do you feel burned?

By having too much data on me, my friends, my kids, politicians, crunched by AI. It is just too much power, if you think few moves ahead.

Isn't that the _potential_ to be burned, as opposed to being burned?

To be fair, there are many threats in the world we can all be worried about, but largely ignore in our daily lives, to no meaningful detriment in most cases.

Yes and no. Thing is, we may be burned already few times, but cannot prove or check it.

Did the bank set your mortgage rate based on risks, derived from your data?

Did you lost job application because your private emails leaked conflict on previous job?

It is dark ways your data can propagate, Goofle is a business, and it makes money on your data. Is ad targeting the only service they sell?

I just decided to ungoogle myself completely, and hide from all tracking on the web. No data from me anymore, I ceased to exist :)

There is no chance that you do massive surveillance in a microkernel. There might be running an app on the top of the kernel that does that sure, and yes we cannot trust google with mobile OS like Android but I think developing a microkernel is fine.

Just want to point out - Zircon is their microkernel, while Fuchsia is the operating system on top of Zircon. Both are open source though.

I stand corrected. Thanks for the clarification.

Well, it's open source. Are there mysterious binary blobs that I'm not aware of? If not, then when this becomes Google's new mobile OS then it's going to be the same as the situation today with AOSP running a bunch of untrustworthy closed-source Google services and apps.

Fuchsia has different Open Source license. OEM don't have to make their modification opensource. So if you oem you can run any binary blob. Also you don't have to make drivers available.

I'm very surprised that I agree after reading. As I read initially, I was thinking "what about..." but it's really about separation of concerns.

This paper, depending on how you look at it, either justifies the existence of dot dot, or makes it seem like a wart. IMO dot dot is too well-established and essentially sensible (in a minimal UNIX fashion) for it to be removed on a whim. Consider the approach taken by https://man.openbsd.org/unveil.2: if you try to access a file you aren't explicitly allowed to, you get EACCESS or ENOENT. I'm not an expert, and the possibility of errors obviously requires applications to tread more carefully than they might have previously, but it seems like a clean solution.

> removed on a whim

They changed the whole concept of having one single root. As I understand the plan is to use many fragmented and independent filesystem handles that can optionally mounted together.

So the restriction is more about non being able to access a folder if you don't have access to an appropriate handle.

Paths like folder1/folder2/../folder2/file are still perfectly fine.

To clarify: that's the Plan 9 situation. I'm talking about the merits of the Fuchsia situation (".. is no longer available").

References to a forbidden parent directory from a chroot can just return ENOENT, because it doesn't exist in that universe. I may not be understanding this fully, but to condemn ".." based on some accusation that it's incompatible with chroot semantics (or "a holdout from POSIX") seems tendentious.

Didn't MS-DOS already do this with letters? (e.g., a:, b:, c:)

Maybe I don't really understand Fuchsia's approach. But I do really like OpenBSD's approach in unveil.

It is ironic this is coming out of a company with an internal site dedicated to depreciating "Considered Harmful" because it is the epitome of grandstanding.

Here is some further reading on why we should downvote these types of things into oblivion: https://meyerweb.com/eric/comment/chech.html

That's a silly article. The essay says that we should disregard essays if they have a title like the essay's own, so... pradox.

When "considered harmful" is considered flaimbait rhetoric, perhaps the problem is with the readers who refuse to engage with any mildly-worded criticism.

It’s surely in how the reader interprets it, but I always read “considered harmful” as quite mild but to the point. Something like “isn’t great”.

A refreshing opposite of clickbait as the actual proposition is in the title.

For irony sake one may link this file as

    https://github.com/fuchsia-mirror/docs/blob/master/the-book/../../master/the-book/dotdot.md

Not as ironic as you'd think considering the article specifically talks about support for .. in userland tooling and uses CWD as an example.

well actually the article complains about stuff like

https://github.com/fuchsia-mirror/../internal-page/secret-re...

which quite more serious

Nope, that actually 404's. It may appear to work, because most browsers will apply some equivalent to the `__fdio_cleanpath()` function mentioned in the article, resolving the traversal locally.

You can verify this as follows:

    curl -I --path-as-is 'https://github.com/fuchsia-mirror/docs/blob/master/the-book/../../master/the-book/dotdot.md'

if most browsers apply it... doesn't that means it works? Maybe not in the resolved-by-the-server-returning-a-304 sense but in at least some sense.

> at the time of writing, symbolic links do not exist on Fuchsia

Is this for technical reasons, or similar philosophical ones because symbolic links also allow for escaping from “jails”?

I think "at the time of writing" implies they're still under consideration, rather than outright rejected due to philosophical differences.

Their whole "scheme", where ".." can be dealt with as a canonicalizing step BEFORE submitting the path to the OS for actual opening, seems to require refinement when symlinks are involved.

Take a perfectly spherical unix:

    $ mkdir /tmp/hn
    $ cd /tmp/hn
    $ ln -s . foo; mkdir bar; touch baz
    $ ls -l bar/../baz foo/baz foo/../baz
    ls: cannot access 'foo/../baz': No such file or directory
    -rw-r--r-- 1 jepler jepler 0 Nov 28 18:21 bar/../baz
    -rw-r--r-- 1 jepler jepler 0 Nov 28 18:21 foo/baz

Fuscsia-with-symlinks would have some extra hoops to jump through to make rewriting act like POSIX in the case that /tmp/hn is not escaped, which surely you'd want if you went ahead and introduced symlinks.

> which surely you'd want

Nope. I don't want that at all. There's a reason that by default cd goes out of its way to make .. ignore the parent of a symlink.

It should be noted that this is not without precedent. Plan9 doesn't have symlinks for example, everything is done using binds. You would be surprised how many complications come up when you add symlinks to path resolution code.

> What about shell traversal? > [..] For example, if the CWD is “/foo/bar”, and a user calls “cd ..”, then the underlying call may be transformed into “chdir /foo/bar/..”, which can be canonicalized to “/foo”.

So fuchsia also won't have CWD, then? Because if it has CWD, then the process can always chdir / rendering this lack-of-.. exercise pointless.

I don't see what made you think it won't have a CWD.

fuchsia is not posix, so they can (and I suspect have) change the semantics of chdir to be more restrictive.

I think the idea in Fuchsia is that directories can be passed as resources to children processes, which can then move freely around in that directory structure. If the parent passes a child "/foo/bar/", the child sees "bar/" and it's contents, which might be "bar/baz/bingo/". The child can then move to "/bar/baz/bingo/" do work there, and then recurse up the hierarchy. It simply cannot ".." out of "foo/".

Hope that makes sense.

Given that they’re still resolving “..”, they’re just resolving it in code client-side before requesting the content, they really haven’t gotten rid of “..”: they’ve implemented a path sanitization library, similarly to how many other frameworks (notably those designed to serve files over the web) behave.

That is getting rid of ".." - the client-side version is fundamentally different in behaviour, since it can't be resolved without prior knowledge of the filesystem. ".." isn't being sanitised from user paths, it's being translated, because there is no equivalent concept in the file server.

No, it's deeper than that. They are describing the capabilities of a process handed a file descriptor, which means it does not have a path at all. There's no way to resolve '..' relative to no path.

Compare that to a POSIX system where a directory has an actual child which is a reference to the directory's parent whose name is always '..'.

The talk about resolving '..' is merely a demonstration that the behavior of "cd .." can be supported/emulated in a context where you have both an open directory and corresponding path, without requiring that '..' literally exist.

To add to that, as the document writes in the very first section, it can sort of be thought of as a chroot. You can't .. out of a chroot; there is no "..". (Yeah, it's not exactly a chroot; that's well covered in the document.)

It’s a thing born in google. It shows. The reasoning behind problems and solutions is so abstract that makes for a beautiful paper or a nightmarish reality. In the ‘80s there was a little known machine, called the Commodore Amiga. Paths were addressed by a volume:folder/file schema. Apps had logical volumes too (progdir:) and the os injected others (env: temp: fonts: ..) guess what? Just use that schema and control what an app can access or not. If you don’t give me a volume for a disk, I cannot make my way to it if it’s not collated into a mountpoint thing

This particular idea wasn't born in Google; it traces back through previous secure OSes like EROS and KeyKOS maybe with a dash of Plan 9 thrown in.

NLTSS as well

I wouldn't say the Amiga was little known, though it did lose the war over the desktop.

I like that syntax by the way, kind of like drive letters but much more descriptive and not limited to A-Z. Netware also used it for full path specs.

What you describe is called "namespaces". Plan 9 and Linux have them already.

In Plan 9, special measures were taken to get dot dot right, not so much because of private namespaces but because of bind.

http://doc.cat-v.org/plan_9/4th_edition/papers/lexnames

For those uninitiated to Plan9, binds are basically the replacement for symlinks (they're kind of like a mix of mount namespaces and bind mounts but more fundamentally baked into Plan9).

Docker "Volumes" are headed that way... but aren't quite there yet. There's a lot of finer detail to hash out first, though.

I believe it comes from DIGITAL systems, eg VMS.

That’s why tools like gosec complain when user input is used to access files - specifically because ‚..‘ allows one to „escape“.

Gosec: https://github.com/securego/gosec

So can a process create a symlink a la

    ln -s ../../.. root

And gain access? Or are symlinks now fucked with too? This seems like a poorly thought out plan, and unnecessary at that.

> This seems like a poorly thought out plan, and unnecessary at that.

Welcome to Google Future (c). Enjoy your stay.

Any good papers to read on fuchsia?

This is ironic

cd..

Here you go https://fuchsia.googlesource.com/docs/+/HEAD/the-book/

i really wish the "considered harmful" meme/trope/whatever would take a nap.

This should probably point at the upstream page by the fuchsia project (https://fuchsia.googlesource.com/docs/+/HEAD/the-book/dotdot...) rather than a mirror source repo

> Upstream

Ironic

Rusty... trying to picturesque it...

Sure thing, we've updated the link from https://github.com/fuchsia-mirror/docs/blob/master/the-book/....

I was wondering who updated it. Thanks.

Lost me at 'story' 5 times in a paragraph. Get rid of the story and get on with the real relation.

"considered harmful" considered harmful

The origin of the meme was an editor's labeling of Dijkstra's commentary. Titling your own article this makes it seem like you wrote the article, forgot you wrote it, found it again, and are presenting it to others as an interesting perspective.

That makes it impossible to properly implement POSIX on top of it and seems like a poor design.

The correct design is to have both a "root" and a "current object" associated with every "file descriptor object" and allow ".." to work up to the "root" (and thoughtfully handle cases where the "current object" is moved outside the "root").

You can't do it with paths because that doesn't track directories being renamed, that would cause the descriptor to suddenly become inaccessible in the middle of an operation.

> That makes it impossible to properly implement POSIX on top of it

so don't implement posix. I don't know why people get so hung up on new oses being POSIX compliant.

Mostly because being POSIX compliant makes it far easier to port a lot of command-line and server-side software quickly. Most new operating system projects don’t have the manpower to rewrite the world - I think the last major one was probably Android.

Well. They’re google.

Also the redox team knocked out rust versions of most of the standard utilities in a few months.

You'd be surprised how generic most code is or how easy it is to port if it's POSIX-like. Out of the major consumer OSes, only OSX is POSIX certified (in 10.5 which is only 11 years ago). Hell, the majority of Android code is written in Java which has no POSIX roots at all so avoiding POSIX isn't a death knell you seem to imply. iOS, while it supports POSIX, most of the developers use iOS-specific APIs unless they're in shared code with another OS & not using something like Boost. Have you actually tried using POSIX networking APIs? They're god awful. POSIX threading primitives are terrible. etc etc.

POSIX is sometimes useful for writing cross-platform C/C++ code. It's extremely limiting though, out-of-date, & doesn't actually offer the write-once run anywhere you'd like with POSIX.

You can write POSIX code that will fail to build, fail at runtime, or even behave incorrectly when you run it on another "POSIX" system (at least as far as Windows/Linux/OSX/Android are concerned). Certainly a far cry from how a standard is supposed to behave.

POSIX also, for the most part, targets the lowest common denominator of platform features which means the POSIX API isn't as rich as makes sense for the majority of applications, doesn't have the same performance/security, and/or isn't as easy-to-use.

POSIX also leaves many subtle decisions to implementations' discretion which means that even if everything works in the happy path, it'll break in subtle corner cases. For example, PATH_MAX is defined as 256, _XOPEN_PATH_MAX is 1024, but Linux & OSX both have unbounded limits so a POSIX program can easily fail to be able to open all files on an OS; making this a build-time constant was the stupidest decision in the world & endemic of how POSIX is designed.

Most modern language runtimes these days (Rust, Go, Java, Swift) comes with a far richer, less bug-prone & more feature-full set of features in the standard library out-the-gate on all platforms (so you just need to port the 1 standard library) & most libraries build on that standard library so you usually get them for free.

Most platform vendors also provide custom APIs to interact more richly with their specific features for performance, battery, usability, security, etc. To take full advantage of a platform, which you're pushed to usually by market forces, POSIX doesn't help you.

* EDIT: Also, POSIX is gigantic. The majority of useful existing tools probably use 20% of the entire standard. Porting that smaller API surface isn't challenging.

Basically POSIX is the C standard library they didn't want to make part of ISO C.

As you well point out hardly significant when using other programming languages, even C++ standard library improvements are making it less relevant for C++ devs.

That is a recipe for a mono-culture stuck in the past.

Of course it is. It’s not a good thing - it’s essentially killed any possibility of a capability-oriented desktop OS for a number of years, for a start. But it’s the state of things - if someone can’t run at least most of their software on a new system, it’s a massive barrier to adoption.

The reason Android and iOS got away with it was that nobody was really running important software on their phones in the first place, so it was a brand new capability that didn’t need to be compatible with anything.

Symbian and J2ME phones were quite popular in Europe and Asia, lacking POSIX compatibility wasn't an issue. In fact, the PIPS library for Symbian was only introduced after the decision to buy Qt.

You also forgot about ChromeOS, which I bet hardly anyone on the US school system cares about POSIX.

Or the webOS running on LG televisions nowadays.

FreeRTOS, mbed and IncludeOS aren't POSIX as well.

Regarding Fuchsia, there is already support for ISO C, ISO C++, Rust, Go, Dart/Flutter, with Java/Android on the horizon.

So I doubt Fuchsia will miss POSIX that much.

My experience of J2ME phones (as someone living in India pre-iPhone) was that the Java functionality went unused, except as far as it was used to implement vendor-written apps.

ChromeOS benefits from the fact that it is just a web browser - if you want just a web browser rather than a generic operating system, Fuschia is of course unnecessary.

Nobody targets webOS, and I suspect people’s choice of TV has more to do with how it looks on the in-store display or how cheap it is than what apps you can run on it.

FreeRTOS, mbed and IncludeOS are not generic operating systems in the first place.

There are essentially no “pure” ISO C or ISO C++ programs - everybody uses system-specific libraries at some level.

Basically, Fuschia suffers from the fact that a large amount of userland will need to be rewritten for it, depending on what market segment it targets. (“Just” being an alternate Android runtime for phones, as an example, wouldn’t require this.) This is an obstacle - not an insurmountable one, but being non-POSIX is an obstacle for any OS, that the people behind it need a strategy to tackle. Whether or not people will code apps for Fuschia will depend on how excited developers get about it at first, and then how many people use it.

As I understand it, Fuschia doesn't aim to comply with POSIX.

Your suggestion matches the behavior of "O_BENEATH" in Linux and FreeBSD, or e.g., Capsicum directory descriptors with the CAP_LOOKUP capability.

https://lwn.net/Articles/604015/

https://lwn.net/Articles/482858/

you can define a root filesystem and mount everything you want to access there. the restriction is that your program (or a compatibility layer) should always keep an handle on the root filesystem.

Coming from Ruby, NodeJS annoys the crap out of me with its lack of decent autoload functionality. Seeing relative paths in modules really annoys me. JS world can't figure out what should be global and what should be local it seems.

What do you mean? Global packages are global, local packages are local.

Wrong thread?

It's interesting to think that the goal of '..' was for navigation around the command line shell. Which is a tool that 95% of people developing software never really use, or use as a necessity (type one command and leave). It's really an interesting thought piece to consider that the idea of the '..' directory is as legacy as the headphone jack.

Developers who don't use a command-line shell, at least some of the time, aren't much of a developer.

I think he means that people who use IDEs don't use shell file navigation as often.

Yes, that is exactly what I mean. The majority of people developing software are doing it inside of Visual Studio Code or similar tools, where their only exposure to the command line is when they do "create-app"

You keep using the unqualified term "majority" with no way to actually back that up. This would probably be less critized if you stated it as your opinion (which it is) and not as a definitive statement.

Even a qualified appeal to the majority isn't a very good argument. There are many scenarios where the majority of events is caused by a minority of individuals.

For instance, 20% of developers could be responsible for 80% of all software, or 80% of the most used software, or the software that generates 80% of GDP, etc.

I think there could be a generational difference or "how you learned" angle at play here. For me learning the command line and early computer use went hand in hand. And I simply could not possibly imagine using git or other source control through an IDE and not a CLI. (I know they offer it, but it just doesn't seem natural.)

There may also be a factor in whether you learned to program as a child or at a university where Unix culture was dominant.

I learned to program (in the sense of writing programs in a compiled language) around the time the Macintosh and Amiga came out, before free Unix-like OSes on your PC were much of a thing. So while the Amiga did have a command line, and so did Macintosh Programmer's Workshop, I mostly saw a command line as the obsolete interface associated with MS-DOS. Obviously source code control was not a thing for a kid programming in the 80s.

Even though I have more recently used git than any other source control, I don't think the horrible interface has anything to do with its utility. You may like it, you may hate it, but it doesn't have to be like it is, it's just the personality it has. Kind of like Linus and his grouchiness.

Anecdotal, but I learned to program as a kid prior to college using free Ubuntu CDs[0], and became familiar with the command line sort of by necessity.

[0]: Ubuntu used to mail you an install CD, for free, anywhere in the world. In the early 2000s. It was pretty cool.

I've been using Linux for roughly 20 years, and I prefer the command line. But that's only because it's Linux. There's no inherent need to do a lot of things with a command line except in a context where it's the simplest, most stable, predictable, and documented way purely because of historical and cultural reasons.

Because of the long running Linux/Windows rivalry, hardly anybody can imagine something better than a Unix-clone any more. But the very name Unix was chosen because the original OS was not intended to be the be-all/end-all of OSes.

In the early 80's I started on a Commodore with a BASIC interpreter "command line," spent a few years later learning DOS, and did not encounter UNIX until about '93 when a teacher mentioned Minix as a way to learn it. At work I installed Slackware from floppies, and was exposed to Sun/SGI/and VAX, good times. Took to each like a duck to water. Liked GUIs and GUI programming as well.

Never understood the folks that wanted to only use one or the other, not both, when they are complimentary. Avoiding either is doing yourself a disservice.

This!

My start was on an Apple, but the rest tracks well enough. did Slack, then RH 5.2.

SGI handled the command line / gui matter particularly well. Most things had a GUI, and the GUI would issue the "--gui" or "-verbose" option to get the additional feedback needed for the GUI to behave more like one would expect, despite it basically being a wrapper for an otherwise CLI program.

The time I spent on IRIX really solidified when and where the two paradigms make sense. And they both do. There is no one size fits all winner here.

A BASIC interpreter is not what I was calling a command line. I guess I was thinking of it as implying an OS shell. Originally, the party line was that you did not need one on the Macintosh, but besides MPW, AppleScript came out by 1993.

For my part, I never understood why someone would want to go back to not using a GUI, once they had been invented. I mean, using a graphical interface doesn't prevent you from typing commands within windows. It's just a question of whether you limit yourself to the ancient teletype paradigm or not. So I never saw (post-1984) GUI and command line as equally valid and valuable worlds, because a GUI can encompass everything, while the command line doesn't.

This is how we loaded a program from cassette or drive:

    LOAD "A*",8
    
    from: https://www.c64-wiki.com/wiki/LOAD

It was an interpreter and command-line environment.

There's many reasons you'd still want to use a terminal interface, CLI tools, and a scripting language, easily searchable. They have staying power because they excel at certain repeatable tasks, whereas a GUI is often better for exploring.

No one has invented a lasting, portable GUI CLI, so that question is moot for now. Maybe they could, but the work involved is probably not worth it for the gain in functionality.

Next, that they are "ancient" is immaterial. They have enough of the features needed to be effective. Paper is ancient for example, and still useful in various situations.

Something like AppleScript may work acceptably for scripting, but is not available on 90% of the world's computers.

Also recommend giving a newer shell like Fish a try, it is quite helpful.

Command lines and scripting are useful for many things; I don't think they are alternatives to GUIs though.

People still use paper to a fair degree in many offices, as you say, but it's not an alternative to electronic records in the way Coke is an alternative to Pepsi.

For the record, I use VS Code, but the fastest way to _open_ VS Code is

    cd project-name
    code .

It also lets me type Git commands directly into the terminal, which is way faster than doing it through VS Code. And there are still plenty of things, like rebasing, which VS Code doesn't have GUI support for.

This is what I do. I use the command line fairly often.

> The majority of people developing software are doing it inside of Visual Studio Code or similar tools

I'm assuming this is an anecdote rather than data? Looking around my office, 19 out of 20 people are using the "xterm and chrome are the only two apps I run on my laptop" style of development; only one has a graphical IDE + chrome

That’s pretty harsh, it really depends on the platform/project. If you’re doing a lot of development on windows for example you’re probably not doing a ton on the command line (not everyone is on a Mac doing web or mobile). I know doing Unity3d game development in windows 10 the only time I really use the command line is when I need to do something in git that sourcetree can’t do. And I know the command line plenty well! There’s just no huge reason for this current project.

Plus some languages sort of have their own customs. Like if you were doing Smalltalk a lot of things you’d use a command line for in web development or mobile dev you’d use a workspace in the IDE instead. And then you have Lisp, where people used to joke that Emacs had become its own OS essentially.

Sure you can find focused exceptions, but the command-line+scripting is the cross-platform interface for developers, power users, and admins.

Even MS finally succumbed to reality when Ballmer finally got the boot.

> the idea of the '..' directory is as legacy as the headphone jack

so... not at all?

I like this approach from Fucshia, Makes me remember dumb sploits in php code. You can take the commandline from my cold, dead hands.

Yeah, that analogy was bizarre.

I think you vastly under estimate the usage of command lines by developers. Not to mention opts and admins.

I don't think there is a week that has passed in the last 10 years that I haven't used the command line for something.

To the point of the article, it sounds like the client will handle the work the server once did in parsing the navigation and path commands.

Though the lack of symlinks sounds like it would be the a more painful loss than ".."

Both of these would likely require many *nix utils to be changed to be compatible.

I'm not so sure. I can go a week and not use one. And lately its not been a win - the shell didn't get me anything.

A week! If I go an hour without using the command line, then it's because I'm not actually coding.

I actually code in the command line, usually. Vim.

Terminal, not command line.

Terminal emulator, not Terminal.

What does it matter if it is emulated or real?

Other than the web browser, I can go weeks without accessing any other GUI on my desktop.

Do you not use local media viewers for films, images, or PDFs? Or local image processing tools?

Maybe it is just that I overuse the internet, but if you have simple needs web interfaces are actually really good. most of them are broken, but to navigate a media list (photos, movies, albums, pdfs etc. (maybe not text file)) the most comfortable interfaces are often web.

I view all of these in my web browser. I don't store these locally.

More like 15 minutes, but I get your point.

It's possible parent is some super-junior developer doing compartmentalized tasks and never leaving the IDE, but for anything past that avoiding the CLI and remaining productive is basically impossible.

After developing on unix variants for over 20 years, I can say that most people developing software do not use the command line. I see this every day... Even skilled developers run from command line git and into a GUI tool, instead of really understanding the tooling they're using.

`history` tells me I used 17 commands this evening, I'm a plain old user in relative terms here [my day job only requires computers for accessing social media].

Uh what? That’s maybe true with extremely junior devs, or in Windows shops, but most devs I know use the command line very heavily.

I cant imagine development without my shell at this point.

Even if I didn't use vim+tmux, so much other "stuff" needs the terminal. Compiling, profiling, testing, searching, moving files around, ssh/rsync and so on.

Sure, IDEs have most of this stuff, but they're not always as user friendly as cli tools, at least once you're acquainted with the later.

I've only worked at 2 companies, but every developer I've worked with at both used the shell at least weekly.

Maybe if your experience using a shell is only Window's CMD. I live in the shell. My Linux machines are configured to automatically open a terminal upon login and the first thing I do when I get a new Windows installation is getting bash working on it even if it's not for development.

Reply on news.ycombinator.com