Fix 260 character file name length limitation: Declined
visualstudio.uservoice.com
visualstudio.uservoice.com
Many can guess what happened next: I went to delete the erroneous top-level .svn folder and was given an error, the gist of which was: "Cannot remove. Path too long." There's more to the story, but it turns out the real answer was to boot into a Linux LiveCD and delete from within there.
I found it strange that one set of Win32 calls allowed TortoiseSVN to create paths of a certain depth, but the other set, used by Microsoft in their tools, would not allow me to delete them. These days I understand the reasons behind it, and so instead find it merely annoying.
eg.
rm -rf C:/REALLY/LONG/PATH/
doesn't work but
cd REALLY
cd LONG
rm -rf PATH
cd ..
rm -rf LONG
cd ..
rm -rf REALLY
works (windows equivalents of course)
In past projects I've ended up rewriting most of of System.IO that dealt with paths to fix this issue. Basically all you have to do is prefix your path with \\?\ and voila it works. (Except in .NET which detects you're doing this and stops it)#!/bin/bash
while [ 1 ]
do
cd .svn
doneThe other looks like this
#!/bin/bash
while [ 1 ]
do
cd ..
rm -Rf *
doneNow just run the first until it can't cd any deeper, and then run the second to clean your hard drive very throughly.
(note I know darn well what script #2 does ... this would probably make a good debugging question for a sysadmin interview... in case you don't get it, for gods sake don't run script #2 unchanged)
Down voting a guy who does have a warning? However for safety's sake, I would recommend prefacing the code with a warning instead of having it at the end.
The good thing about nix platforms is that you have nuclear options at your fingertips. The bad thing is that you can't be impulsive or an idiot because you have nuclear options at your fingertips ;)
So VLM... nice to meet you, here's the first version of our software upgrade script I was wondering what you'd change. I started laughing, pointed out a few things, he said I passed.
It had a lot more wrong with it. Weird quoting mistakes (single vs double, also something along the lines of grep "-R blah blah"). And there were basic conceptual issues, like copying the binaries to /usr/local/bin before replacing the old distribution with the new one, such that you'd get old ones. And the path contained a version number and the script tried to change user PATH env variables ... to the old version path.
The proper interview-style fix for my creative solution would be something like rm -Rf .svn instead of rm -Rf *, or bothering to check the output of pwd is longer than X characters using grep -c and some comparisons or playing games with chroot. Its a great interview question because you get a feel for the candidates style, are they most comfortable with chroot, or doing text manipulations of pwd etc or are they a minimalist who likes to make the smallest possible change or into rewriting the whole blasted thing or...
\\?\ is an extended-length path.
See GP's link for more details.
If they're not going to fix the problem, and I respect why they're not going to, at least make it easier to work around. It just seems like on Windows you often have to deal with silly, incredibly annoying problems like this that other platforms don't have.
robocopy c:\empty c:\so_many_subdirectories /mir will do the trick.
On my Ubuntu PC, I created an encrypted folder in my Dropbox using EncFS. Turns out EncFS produces very long folder names and file names if you use the default, most secure setting with per-file IVs. Still, everything worked fine until I booted my Windows PC. As soon as Dropbox started trying to sync with the Windows PC, it ran into the 260-char path length limit and quietly began to truncate their names. Result: corrupted EncFS folder.
Microsoft has had many chances to introduce clean new APIs: for instance, .NET could have been built on top of a simpler platform, or Metro could have been an opportunity to take out some cruft.
In the end Metro breaks everything (if you think the median programmer could write non-trivial apps with asynchronous I/O, think again.) Amazingly, Metro almost feels like a tablet operating system when it is running on top of the line hardware, but the ultimate attack on latency is to remove things from the stack and not to add them.
Personally I think Windows today (even Win 8) beats the pants off Linux and MacOS as a GUI operating system, but I seriously have to ask questions such as "will Intel be making new x86-compatible processors 10 years from now?" when I see Microsoft and Intel consistently painting themselves in a corner.
I'm uncertain they kept cross-platform systems running around (whereas I'm reasonably certain Apple has a full OSX running on ARM internally, as they had OSX/x86 long before they left POWER behind). Although NT was built to support a number of architectures, it's been almost 15 years since NT last ran on non-x86 systems (AlphaNT, the Alpha port of Windows 2000; MIPS and PPC had already been dropped by this time), that's a very long time, and more than enough for x86 dependencies to creep in.
On the other hand, WinRT is supposedly a full NT kernel on ARM, so...
Thus, every version of Windows NT has been released on at least one non-x86 platform.
- NT 3.1-3.5: x86, Alpha, MIPS
- NT 4.0: x86, Alpha, MIPS, PowerPC
- 2000 through 2008 R2 (7.0): x86, Itanium
- 2012 (8.0): x86, ARM
Notice that I listed Windows Server 2008 R2 as 7.0 rather than as 6.1. And that I listed Windows 2000 as 2000, rather than as 5.0.
In other words, I was giving the equivalent workstation OS -- not the internal NT version number.
http://blogs.msdn.com/b/b8/archive/2012/02/09/building-windo...
WinRT is a full NT kernel on ARM so your statement is incorrect. Also, Windows runs on IA64 (Itanium).
With Visual Studio you can currently compile code for IA32, AMD64, IA64 and ARM.
NT is very well done in regard to cross-platform compatibility: all platform specific code is centralized in one location and if you need platform specific instruction, you must you the Hardware Abstraction Layer (HAL).
For instance, I'd say the tablet experience is more defined by having an SSD than having a touch screen. The hybrid hard drive that has been pushed in the WinTel ecosystem (including ReadyBoost) is a joke.
Customers don't feel minimum latency, mean latency, or median latency. They feel maximum latency. An SSD reduces maximum latency, but half-baked caching schemes don't. Yes, they improve measurable things like boot time, but a quick boot is little solace when your web browser regularly locks up for 20 seconds. (Even if the machine could reboot faster than the time the browser is locked up)
If there was one fundamental tenant of the anti-customer corporate ideology it is "throughput computing", or, the idea that nine woman can produce a baby in one month.
Customers don't perceive throughput, they perceive latency. When you're stuck on the 405 going 5 mph you aren't going to be philosophical and multiply that 5 mph by the density of vehicles, you're going to feel like it's slow for me...
Had the Windows 8 requirements included "no HDD", people would be saying "Wow, Windows 8 is fast!", instead a touch screen adds $100 to the cost, as does the Windows license, so users get shortchanged on the fundamental performance of the machine when they are trying to hit a given price point.
That's really clever. I was very underwhelmed by Windows 8 but I was moving from an SSD-only Windows 7 system, which are still relatively rare. If W8 black-desktoped on a system with less than 1000 IOPS on the system drive the way it does if you have an unsigned driver, there'd be a whole ecosystem of modern hardware and software that can assume fast storage IO opened up, with no backwards compatibility loss at all.
The combination of moving from Win 7 starter to Win 8 (meaning remote desktop and all the goodies get unlocked) and going to an SSD has made it a really awesome machine. Win 8 really does get a lot out of an SSD, but if you hamstring your machine with an HDD or a "Hybrid" Hard Drive it doesn't matter how fast your CPU is if I/O makes the machine go out to lunch.
I've never heard of NT for the i960 before. Are you sure you don't mean the i860? Very different machines.
¹Not that one, this² one.
The i960, on the other hand, had IMHO a great instruction set (even after losing the capabilities¹), with clean orthogonal 3-address instructions combined with programmer-friendly addressing modes.
He's on the Xbox team now. He helped design the OS for Xbone, which, at this point, can't possibly tarnish his reputation that badly.
Does it actually break anything? In the sense that programs written 10 years ago don't work.
So many opportunities to settle down the MS development ecosystem into something sensible and consistent... so many pathetic missteps. Linq2SQL getting deprecated almost as soon as it was launched, Click Once installers just being an unusable mess (and now they've killed the old-but-usable installer projects), the endless shuffle of GUI frameworks, etc.
So many talented people firing out projects that are almost good.
I currently have in my hand an ARM tablet running Windows 8.
http://stackoverflow.com/questions/1880321/why-does-the-260-...
http://www.codinghorror.com/blog/2006/11/filesystem-paths-ho...
Summary: The assumption has been baked in forever and changing it is Non-Trivial. Workarounds exist though! (Kind of.)
Some Windows APIs support 32k paths but since not all do, very few applications can claim to support more than 260.
Do people actually want deeply-nested project hierarchies?
For example, from a real-world, full operating system, the deepest path I can find[0] is in a build directory and 184 characters:
> "./ports/lang/python26/work/Python-2.6.1/portbld.static/build/temp.XXXXXX XXXXX-vHEAD-amd64-2.6/XXXXXX-home/XXXXX.git/src/ports/lang/python26/work/Python-2.6.1/Modules/_multiprocessing"
(Names changed to protect the "innocent.")
[0]:
$ mkfifo ../names
$ find . > ../names &
$ python
>>> with open("../names") as f:
... l = 0
... nm = None
... for n in f.readlines():
... if len(n) > l: l, nm = len(n), n
... print l, nm C:\Documents and Settings\Joe Random User\My Documents\Some Cloud Software\Company\Projects\2013\156 Big Customer Inc\Locations\Someville Foostreet\Problem reports\2013-10-07 product not working\images\video of product not doing what it's supposed to do.avi
Also: find . | python -c 'import sys; print max(sys.stdin.readlines(), key=len)' $ find / -mindepth 10 | wc -L
273
I seem to have some slightly longer file paths on my home computer. Do I need that? Probably not. I could move those files somewhere else. But should I really have to worry about such things? find -mindepth 10 | awk '{ print length(), $0 | "sort -n" }' | tailI could be completely misunderstanding this though since SBT is fairly complex.
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_v8/v8/work/v8-3.21.3/out/x64.release/.deps/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_v8/v8/work/v8-3.21.3/out/x64.release/obj.target/v8_base.x64/src/extensions/externalize-string-extension.o.dC:\Documents and Settings\____._______.________\project\node_modules\package\src\tests\somelib\helpers\test123\file.html
What made this troublesome for me is that my networked profile refused to sync while the file was there (and couldn't be removed without renaming each layer to single character folders. Additionally the profile sync failure copied back files I had deleted confusing the heck out of me :)
Problems like this aren't problems when everything is working normally. But that's beside the point. When the shit hits the fan and the system is only getting in your way and adding to the problems instead of helping solve them, it's really really really not fun.
</sarcasm>. The complaint is about Visual Studio's decision in particular, not the weakness of (some) (old) Win32 APIs.
My understanding is that it's not worth going over all of their code to use the newer APIs since they also rely on third parties that they can't force to update. Those would still break when dealing with longer paths.
Still sucks (the path length limit is indeed problematic, especially when dealing with Java code), but let's not exagerate things shall we?
Even if VS restricts itself to 32k-aware APIs, you'll hit this eventually when shelling out or calling any third-party code. Never going to happen.
Even, if they did make the change and somehow managed to re-test and fix all their software, they would still have created the potential of breaking every piece of Windows software in existence.
Making the change has to potential to cause untold chaos, the likes never before seen in the software industry.
It would make Y2K look like a paper cut.
Even if they're just looking for a null terminator, they'll crash when they overrun the bounds if written properly.
wchar_t buff[MAX_PATH] = {0};
int (*callback)(void);
if (NULL == ::PathCombine(buff, /* ... */)) {
return; // Error, don't continue
}
callback();
If this is compiled with a small MAX_PATH and run on a system with a large MAX_PATH, then the system can write a value that's too long, overflow `buff` and overwrite `callback`, which would then execute the buffer contents and probably crash. Building an exploit out of this is left as an exercise for the reader (e.g. supply PathCombine with some shellcode)I picked a random API, but there are several win32 functions which accept a character buffer without an explicit size, and mention MAX_PATH in the MSDN documentation.
if (NULL == ::MySpecialPathCombine(buff, /* ... */)) {
return; // Error, don't continue
}
There is no way Microsoft can patch that, but the problem is still there.Also back to the PathCombine example. When the third party executable was compiled, the Windows SDK would have defined PathCombine to take a string the size of 260 (or whatever value MAX_PATH was at the time it was compiled).
If Microsoft did patch the call and then return more than 260 it would just crash the calling third party executable as it would only be expecting at most 260 characters.
In effect Microsoft would effectively re-defined the static PathCombine API signature at runtime and making it some sort of dynamic signature.
So if you updated Visual Studio to support it, a ton of other tooling and UX would fail. This needs to be a corporate wide, product wide, and 3rd party initiative. The scale of which is likely not worth the result.
[1] http://msdn.microsoft.com/en-us/library/windows/desktop/aa36...
[2] I don't heavily use Windows anymore, but when I tested < Windows 8 machines with > MAX_PATH, Explorer did not work.
/Applications/Xcode.app/Contents/Developer/Documentation/DocSets/com.apple.ADC_Reference_Library.DeveloperTools.4_6.docset/Contents/Resources/Documents/recipes/instruments_help-stack-trace-help/Displaying_Your_Source_File_That_Contains_a_the_Symbol_in_a_Stack_Trace/13_Displaying_Your_Source_File_That_Contains_a_the_Symbol_in_a_Stack_Trace.html
Please have now at least the decency to do the same without complaining.
Maybe this might be resolved in the gradual move to ReFS which I am under the impression is the long term future of the windows world.
NTFS allows paths up to 32767 UTF-16 code units (after expansions) with each path component (between `\` separators) up to 255 code units, and this is available through UNC paths.
MAX_PATH is a Windows API limitation, and thus a limitation implicitly inherited by many windows-based software which use MAX_PATH as storage limits and buffer sizes.
Changing filesystem will not fix WinAPI issues.
Hardlinks are extensively used in WinSxS to link the DLLs and executables to places where they used to live. Symlinks are used for many of the legacy directory names that some applications have hardcoded, e.g. C:\Documents and Settings links to C:\Users.
By default.
This is a security restriction, like UAC. And so you can turn it off, just like you can turn off UAC.
There is an alternate parser, but the name has to start with \\? and then it takes up to 32kb of text.
https://en.wikipedia.org/wiki/Subst
Gotta really stupid-long base path? alias it.