Support for better symlink handling in Windows 10 | Hacker News Reader

Support for better symlink handling in Windows 10

blogs.windows.com

133 pointshashhar82 comments

I worked at Microsoft during the development of this - and I'm so very happy it finally shipped.

We started working on this about a year ago after chatting with npm, git, Ember, and various other command line tools that were substantially slower (or straight up broken) on Windows. I casually mentioned how big of a performance impact we could have to Satya's Assistant and found myself in a meeting with all required groups (filesystems, security, etc), making my case for non-elevated symlink support, within a few days.

Windows still is a massive tanker, but I'm so excited for the Microsoft that is able to make sensible changes quickly.

Fantastic work :) It's always worth bringing up stuff like this. In the open source world, I find just asking for features has about a 1 in 10 success rate so always worth it.

Hopefully there will be a way of editing these in explorer too at some point.

Can you (or anyone else) explain the rationale behind having to specify a new flag SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE? What's wrong with simply just not removing SeCreateSymboliclinkPrivilege from the restricted token, and by default including it from unprivileged users?

Backward compatibility?

Sounds unlikely given that you could already give SeCreateSymbolicLink to any users - and the thought that someone could rely on a privilege not being there in the restricted token sounds pretty terrifying to me.

The primary motivation for adding the SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE flag is to not disrupt the behavior of existing code.

Because of pre-existing behavior, existing apps are likely to be built to assume that symlinks cannot be created when the app is running without admin rights. If the behavior changed such that the app could now create symlinks but not have admin rights, the app and/or its dependent scripts etc. will likely misbehave.

Therefore, we added a new flag - SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE - which allows developers to expressly indicate that their code is specifically built to handle the ability to create symlinks should their code be running on a supporting platform (i.e. Win10 Creators Update, or later) even if the app does not have admin rights.

HTH.

Oh, and just so we're all clear, if there are any problems uncovered in this implementation, we can all blame @felixrieseberg ;) :D

Thanks a lot for making this. I found out about this while installing Git 2.11 when it asked if I'd like to enable symlink support. Makes life a lot easier.

> Now in Windows 10 Creators Update, a user (with admin rights) can first enable Developer Mode, and then any user on the machine can run the mklink command without elevating a command-line console.

So an Administrator-equiv account still needs to be in the chain of events to make this useful on a per machine basis?

If this is the case I'm not sure how this makes symbolic links more useful to me in a Windows environment... ?

By default access to remote (network) symbolic links is disabled . You can enable it with fsutil [0].

Between the need for admin-equiv for mklink and the per-computer requirement to fsutil-enable sym links on network shares, symbolic links are unusable in a Windows environment.

I don't see how this feature changes that.

[0]https://blogs.msdn.microsoft.com/junfeng/2012/05/07/the-symb...

[edit: fixed typo on 'fsutil', 'sym' and 'symbolic links'. The curse of posting on a mobile device]

If you're provisioning lots of Windows machines, why not just make these two commands part of that process?

For me the issue is more that software like version control systems and archiver utitlities that need to understand and create symlinks, might just continue to not support symlinks on Windows. When the software is running as an unprivileged user, it will still fail be able to create symlinks by default, which doesn't make the feature very enticing to add and support for the developers of the software. Git has been in that category, I don't know if this is enough to trigger a change or not. Here's hoping...

I'll take some a kick at provisioning this via GPO on Monday.

I suppose the issue at hand here is that - at minimum - it appears that use of symbolic links requires a Group Policy that specifies both support for mklink and fsutil to be configured, the former requiring admin-equivilency to enable symbolic links to work for a non-admin user.

As someone who grew up using 'ln -s' in userspace to create symbolic links, this seems to be a rather high barrier to use of a very convenient user function - "Want to use symbolic links for convenience? Have your admin create GPOS to facilitate this" seems a half-assed answer....

In my opinion the magnitude of these problems is massively increased by locked down environments where playing with something has time costs for multiple people. The time for someone to play on their machine vs the time fire b that someone to work with an administrator to request, create and test.

I absolutely agree.

How much upstream effort must occur to facilitate downstream userspace day-to-day function(s)?

[edit]

I'll perform whatever action is required to enable user functionality. That's what I'm here for.

But I don't agree/understand why the user-use of something as basic as symbolic links is dependent on an Admin having a clue.

I'm assuming that this initial admin step is due to the plethora of edge cases in Windows that exist when it comes to symbolic links (old APIs being the most probable cause, but there may be issues in the current FS permissions model). If MS just enable this functionality now, with no extra steps required, in the main stream release, then they will leave themselves open to a bunch of potential security issues and nasty bugs.

Hopefully they'll act on the feedback of the insider-track release users, and those enabling developer mode in the more stable releases, and this will iron out the vast majority of problems before they just enable it by default for all users.

I know this means enterprise users and main-stream (non insider-track) users will be stuck without this functionality (obviously available in unix land for decades) for a while yet, but I'd rather MS rolled this out in a responsible manner rather than just open the flood-gates of potential malware. Symlinks may be basic, but they have been the cause of many nasty vulnerabilities in Linux over the years. Hopefully MS is just being cautious and in a year or so all Windows users will be able to create symlinks without any admin interjection at all.

I agree with you, it shouldn't require admin intervention. But I think the reason it annoys you, other sysadmins and developers alike is because the security model is broken in terms of UX, velocity and innovation.

From strictly a security perspective we need to lock down machines and remove admin rights because clients are a serious threat to our internal networks. But we pay out the nose in terms of productivity, employee happiness, innovation, etc. Even if Microsoft fixes all of these little things that currently require careful thought by a sysadmin before deploying that's just Microsoft. There's still plenty of poorly written software out there that requires admin rights. And that's new software, not legacy. Requiring sysadmin intervention every time someone wants to try something, especially just on a local machine is a huge drag on everyone involved.

Right now the idea is still young and as far as I know hasn't been productized yet but it looks a lot like Google's Beyond Corp[0] system of shrinking the perimeter to exclude clients, treat them as potentially hostile by default and have access to resources based on a dynamic threat assesment seems like the best way forward. Users have flexibility and freedom and the company is protected. But I think it's still out of reach for most companies. It will take projects and products to make it feasible and even then a lot of companies are still struggling to do basic "hard outside squishy interior" properly.

[0] http://research.google.com/pubs/pub43231.html

Do you mean unusable in an environment where you are not a privileged user? Like locked down work/corp environments?

Yes - absolutely. Why does Joe/Joan User have to rely on their Admin having a clue to use this Feature?

We're talking symbolic links here. This is a very handy and convenient feature in userspace. Or, it should be.

In those scenarios I imagine it can be enabled via AD group policies.

There are a number of security issues which can come with symbolic links, especially with how user/group permissions are handled. There is also a lack of support for them in the ecosystem, which could introduce some strange problems (for example, whether specific apis follow them, or not). So I can understand why they're putting this behind a flag.

Like what? I can make symlinks all the live long day on unixy systems without it security problems and without having to log in as root.

Regarding the "interest over time" graph, it's fantastic to see people designing graphs that are just as meaningful for the colour-blind as they are for those of us with normal colour vision.

I noticed that too — without a legend, who am I supposed to decipher what each color means?

Because it's a direct screen capture from Google Trends, and for me, it's pretty clear which are they: CVS, SVN, Hg, Git. Here, I've re-created it for you: https://www.google.com/trends/explore?date=all&q=%2Fm%2F05vq...

Luckily there was a link to the google page where the picture was taken, where you can examine the graph more in detail.

https://www.google.com/trends/explore?date=all&q=%2Fm%2F05vq...

This is great; I love how Microsoft have become so responsive to community demands and wishes.

Personally, I've used mklink /d for quite some time, in particular to manage game installs. This became necessary once they started hitting the 30-50GB mark; suddenly those 500GB SSDs / NVMEs didn't feel so big anymore! (Not that I have much if any time to play anymore..)

Do you know why you have to do it as Administrator? It seems like mklink shouldn't need special privileges.

The worry was Time Of Check To Time Of Use attacks, but I guess they've decided the balance has changed.

Also you've always been able to make junction points without admin.

Unfortunately I cannot find the link now, but recently I have found a funny article related to these vulnurabilities:

If you set up VPN on Windows 10 Mobile for some reason MS has decided to remove the option to resolve DNS at the far end of the VPN. (AFAIK this was present on WP 8.1)

I have found a guide how to enable it: Create a symlink on an NTFS sd card pointing to a specific location (an ini file which is hidden in Windows 10 Mobile from you), insert the card, open the file in file browser with an editor, and change a setting.

Mklink is also the recommended (safest and easiest) way to deploy JVM patch-releases in a lot of settings.

Interesting; will read up on that, sounds like something that could be useful elsewhere also.

Just got a new preconfigured laptop which was almost full on system. Small 120gb SSD, but had a 1TB spinner. Found that there was 30GB worth of MSSQL DBs, so being lazy I simply moved the files to D and symlinked the directory. The only gotcha was making sure MSSQL had permissions to the new target folder.

It would be nice if they added this to Powershell. Last time I tried, mklink was only valid in cmd.exe.

Tbh I don't see the point of this policy change anyway. Developer Mode means it can't be used in Production, so what are we saving exactly? The 0.000001 seconds it takes to launch a console in admin mode, assuming you didn't already disable UAC like most devs do...?

Two options:

    cmd /c mklink ...
    New-Item -Type SymbolicLink -Path symlink -Target target

As for me, I rarely need to symlink files, so directory junctions work just fine, which don't need administrator privileges.

Wow. As a side note, this will also enable real symlinks in cygwin. There is already a setting CYGWIN=winsymlinks:native, but this doesn't work because you needed the rights to create native symlinks, and you usually don't run cygwin elevated.

(Actually, the situation was pretty messed up. There was a group policy that allowed regular (non-admin) users to create symlinks, but this never really worked, since UAC made processed drop a bunch of rights including symlink creation. So you needed a) either an admin account or a regular account, with the policy set, on Win Pro, and b) you needed to request elevation (which is subtly different from Run As Administrator) or disable UAC (which is dangerous and borks Metro apps in Windows >= 8).)

I'm so so glad they fixed this. Next step: Proper GUI for this in Explorer (and also for extended attributes, etc.).

To be fair, the symlink functionality is present in NTFS since XP (though MS never provided a tool to make use of them) and there is a filter driver (Open Source, both 32 bit and 64 bit) by Masatoshi Kimura, see here (search on the page for "driver"):

http://schinagl.priv.at/nt/hardlinkshellext/linkshellextensi...

that provides symlinks on XP and possibly Server 2003 (additionally without any issues with UAC, which is simply not there).

+1 Link Shell Extension is extremely useful!

The problem before was that the symlink creation privilege gets removed by UAC. So, if you gave the privilege to a non-admin user they could use it, but an admin user would always require UAC elevation to create symlinks. https://stackoverflow.com/questions/15320550/why-is-secreate...

Sounds positive, but the article skirts over whether there are any downsides. Surely there must be lots of ways this could get abused? (presumably that's why it sits behind Developer mode rather than being default)

Symlink vulnerabilities are pretty well understood. Though, people still make mistakes now and then.

Simple answer about one of the most common/dangerous: A symlink could be made to point to a system file. If a program running with elevated privileges could then be convinced to manipulate the contents of the file pointed to by the symlink, whoever could use that program on the symlink could cause it to modify the system file.

There have been real world vulnerabilities like this (I've shipped a few), but they're not super common and they're often limited in their capability to cause damage...they require an elevated privilege app, that is poorly written, that has untrusted users that can poke at it. It probably requires some kind of shell access, or some sort of local account (email account, maybe), in addition to the poorly written program that has elevated privileges.

https://www.exploit-db.com/papers/13199/

There's an easy fix for that: allow unprivileged applications to create symbolic links, but by default, only allow applications to access/follow symbolic links owned by 1) the same user that owns the target file, 2) the user the current process runs as, or 3) an administrator. Since people currently can't usefully use symlinks at all except as an adminstrator, that should not introduce any compatibility issues.

Linux has a similar protection mechanism (/proc/sys/fs/protected_symlinks), though it only applies to sticky world-writable directories like /tmp.

I applaud them for trying, but decades into it, Windows scripting still makes me want to cry : ( Habits aside (I typically enjoy working with unfamiliar languages), with a decent shell you feel like a superhuman flying at lightspeed. In cmd.exe and even Powershell (and sometimes even Cygwin) you feel like you are talking to a 3-year-old, having to switch to "simple English" and grimacing to get a very simple point across...

Have you tried the new cli and Linux compatibility in Windows 10?

I've not... does it also require mklink in "developer mode", or can you just do ln -sf? ...

I just successfully ran `ln -s /mnt/c/Windows/System .` in my home directory in bash.

It didn't pop up any dialogues, but I'm a local admin on my machine so it may be different otherwise. I can create a non-admin account and test it, but it's late and I need to go to bed :-)

Well, this is cool. Kudos to the folks that made this happen.

It was always a tragedy that the Windows Shell team insisted on creating their own "Shortcut" mechanism and not shaping the existing FS team symlinks to give a unified mechanism. It used to be a big ask to get teams to work with each other.

Could this be ye oldest piece of functionality to arrive in Windows yet? We also had ANSI sequences at the cmd prompt this year. And native ISO mounting in 2011. But non-admin symlinks go back way further. Cool.

I think the original shortcuts come from pretty early windows systems (.lnk files may have been around 95, but .pif files seem basically equivalent and existed by 3, but probably earlier).

Windows has multiple forms of shortcut. .PIF is for running DOS applications with a particularly configured environment. .LNK is for shortcuts to files, folders, and applications. .URL is for links to websites or other URLs (Steam uses this, interestingly).

Existing FS symlinks? Did FAT support them? .lnk came about in Windows 95.

MS is doing good in my book. They should keep it up. Eventually people will start giving them a chance again.

The problem is that MS has such an amazing record of strategic underhandedness, it would take many years of consistently good behavior for me to trust their intentions.

By the way, exactly _which_ of their patents do they claim Linux violates, and exactly what data are they collecting from Windows 10 boxes?

Yep very true. I'm a little dogmatic about too. I haven't even installed VS Code even though I know it's supposed to be pretty good. Burned too hard plus looking at what they did to others before me makes a pattern that is hard to ignore.

To be fair, VSCode is open source so you could go look if you were interested: https://github.com/Microsoft/vscode

It's always one step forward, two steps back with Microsoft.

Every time I start to warm up to them again, they make me feel stupid shortly afterwards by blowing it.

It's an abusive relationship, in a sense.

We all have our stories. MS has done a number on me.

I think they are less doing good and more scared shitless that they put off a huge swath of computer savvy users with their forced upgrades and mandatory tracking. The nuggets they throw to the dev/unix communities are smoke and mirrors. Strategic pleas of "please don't abandon our creaky ship!"

Finally. This was strange to request admin privileges. I wish they find a solution around the path length issues as well.

Is this solution not fitting your needs ?

https://mspoweruser.com/ntfs-260-character-windows-10/

I tried that in the AU: let's just say this is better than nothing. Well. Hm. Well, actually I'm not even sure about that;

First not only programs must opt-in (through a manifest) -- and that is quite natural given the way some existing Win32 functions are modified, but that won't be taken into effect unless the computer is also opt'ed in. That second condition both makes little sense, and is improperly documented (the doc tells us: a manifested app or the GPO option; while in reality this is: AND)

Then a single example so that you can understand the new "quality" of what MS is shipping: SetCurrentDirectory() https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

Once you've manifested your program correctly (mainly by extracting what is interesting from powershell.exe, like usual because MSDN is shitty and does not describe the procedure to create the manifest with enough details) and configured your install with the good GPO (home edition users: go to hell) you happily starts to try the good old SetCurrentDirectory without the 260 chars limit.

It fails.

2 hours latter you understand that it fails because the MAX_PATH LIMIT IS STILL THERE IF lpPathName DOES NOT END WITH A BACKSLASH ‘ß‘¬ð’€¬ð€þ¬ð€

You then take a look at CreateProcess and understand that there is just no way to bypass the MAX_PATH limit with it on some of its params.

So you stop trying to use that half-backed and poorly documented "feature". The MAX_PATH (=260) limit in Windows is not gone. Maybe it's 5% gone, but that insufficient as hell.

You don't need GPO though ( https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... ):

> A registry key allows you to enable or disable the new long path behavior. To enable long path behavior set the registry key at HKLM\SYSTEM\CurrentControlSet\Control\FileSystem LongPathsEnabled (Type: REG_DWORD).

As I am sure you can imagine, untangling the long-path limitation throughout the OS code, throughout RTL libraries, 3rd party libraries, .NET Framework & CLR, Windows Shell, etc. is a detailed, time-consuming process.

This was never going to be "oh, just change this flag and ... BINGO".

Rest assured we've several teams throughout the company (including mine) currently digging into this long-path effort.

You still need an account with admin privileges to enable the developer mode, so I'd count it as still requiring admin.

How far till windows ships with a whole linux's kernel down there?

The NT kernel was always heavily modelled after the unices of the time (but avoided quite some flaws from the get-go, and of course had quite a lot of differences as well due to that and other necessities, like the driver API).

I thought it was more DEC-ish (VMS?).

It is, Cutler famously hated Unix.

They already emulate a Linux kernel, in recent Windows 10 releases: https://msdn.microsoft.com/commandline/wsl/about

Considering that NT kernel is a much better design, I'd rather get Linux Mint (with UI) on top of NT.

Could you provide some link to read about this? First time I've heard about it.

My respect for NT grew tremendously after reading one of the early editions of Mark Russinovich and David Solomon's "Windows Internals". I haven't seen more recent editions but I'd suspect they just as informative.

The Wikipedia article on NT architecture is decent enough (https://en.wikipedia.org/wiki/Architecture_of_Windows_NT), and some quick searching brought up this slide-deck from 2008 that's pretty good too: http://www.cs.fsu.edu/~zwang/files/cop4610/Fall2016/windows....

I'm most impressed w/ the kernel object manager and the pluggable user-mode "personalities". I've always dreamed of an NT "distribution" that didn't have Win32 but, instead, shipped with the Interix (aka Service for Unix) POSIX personality.

For those who are interested, the next version of Windows Internals is available for pre-order from Amazon :)

https://www.amazon.com/Windows-Internals-Part-architecture-m...

Oh, and @EvanAnderson - you might want to take a look at the Windows Sybsystem for Linux (WSL) - which allows you to run unmodified ELF64 Linux binaries directly on Windows:

https://blogs.msdn.microsoft.com/commandline/2016/06/02/lear...

This is just a policy change. No new features.

I never thought I'd see the day. Now, if only they'd get * in filenames right, I'd be able to check out all of my git repos on a Windows system.

mklink FFS? Even tho cmd.exe is not default shell any more.

I can't understand such a big failure! What ? This thing was hard to do ? https://learn-powershell.net/2013/07/16/creating-a-symbolic-...

Please tell me this is going to make my Vagrant issues go away...

ln -s

a generation ... goes by ... and we get

mklink

but still no hard links and no use-counted file system. Haven't the Bell Labs patents expired by now?

huh? https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...

isn't this documentation a bit dated?

AFAIK as of Win10 hard and soft links exist, and also junctions, which are a third, similar but different concept.

Isn't that exactly what the documentation says? I mentions directory junctions, hard links and symlinks.

Mind you, hard links and directory junctions have existed for quite a while (going back at least to Windows 2000). Symlinks have been added in Vista.

Not really, they were added to NTFS in XP times, see my previous post.

https://news.ycombinator.com/item?id=13095172

windows has hard links.

"Those who do not understand Unix are condemned to reinvent it, poorly."

Those who do understand it, just use it and ignore everyone else.

Reply on news.ycombinator.com