Fastest Way to Delete Large Folders in Windows (2015)
mattpilz.com
mattpilz.com
The correct method is to immediately move/rename the target, then launch an expensive delete in the background. That frees up the location you are trying to use.
I used to see people set up entire workflows that started with, essentially, "wait 20 minutes to finish recursively deleting the previous data" instead of just "move it out of the way and start immediately".
I mean when Shift+Deleting, or emptying the trash, Windows should:
- mark the folder as "garbage" in the filesystem and rename it to something impossible to free up the name
- hide the folder from the UI (apps should not see the file anymore)
- schedule a real deletion (reclaiming the space) with low priority in the background
- in case of a crash, chkdisk should detect garbage files and delete them.
Turning off Windows Defender makes Windows Update run much faster.
And that's using rmdir.
The FAT entries are cleared, so if the file is fragmented, reassembling requires knowledge of the format. Also, Wikipedia tells me that FAT32 directory entries also have some bits cleared, so the right data on disk may not be found.
https://www.codeproject.com/Articles/9293/Undelete-a-file-in...
If this flag has bit one set, it means that the file is in-use, else it is deleted.
DOS (FAT) used an allocation list on disc. CP/M couldn't corrupt the free-list (didn't have one). Unix is a different animal. CP/M didn't have "folders" (user areas instead).
> Wasn't at some distant time in the past deleting done by just renaming a file and changing the first character to a question mark?
That is just the entry in the directory listing - it would still walk the file allocation table and mark the relevant blocks as unused. The FAT operated as a collection of linked lists - the directory listing mentions the first block and each entry in the FAT states which the next block is for a file or 0 if it is the last block. Really a sub-directory was just a special file containing filenames and other properties so deleting an empty directory follows the same process. The root directory is a special case, being of fixed length (12 blocks IIRC) in a fixed position of the structure.
> I think that also applied to folders: Delete a folder's "file", and all the contents are gone, recursively.
No, you couldn't delete a directory with contents by default. Ordering a recursive delete would perform a depth-first search-and-delete on each individual object.
> Not sure how they found free blocks when creating a file though.
As the FAT had been updated, it was a simple first-empty-block search.
Unless something had deleted something just by editing the directory entry, in which case the blocks would be left alone as they would still be marked as in use. I did see this used as a way to try hide sensitive information without it getting overwritten, both as a naive copy prevention mechanism and a "hide my porn" technique.
echo Y | format Z: /FS:NTFS /X /Q
Also, on Windows 10 you can use Ubuntu bash and run rm -rf. I've read that it's faster, but haven't tested extensively.
Update: I did a quick test with 100,000 files and the 'del /f/s/q foldername > nul' approach was about 50% faster than 'rm -rf' on my machine.
Calling CMD.EXE from Explorer without double-quoting can have unintended consequences when ampersands and parentheses are present in the filename too. I'd hate to "Fast Delete" a directory named "& rd /s /q %SystemRoot% &" using his shell context menu entry.
If I was hell-bent on doing this I'd probably add it to my user registry w/ the command:
reg add "HKEY_CURRENT_USER\Software\Classes\Directory\shell\Fast Delete\command" /d "cmd /c rd /s ""%1"""
My current way to delete any folder regardless depth of the tree is to use robocopy (robocopy D:\EmptyFolder D:\FolderToDelete /MIR). It is actually pretty damn fast, might be faster than using RMDIR /S /Q.
I wanted to run a command as admin on startup recently and had to create a scheduled task for this, which of course will be lost the next time I reinstall. Same for services, etc.
Doesn't that mean that the "." pattern won't match everything?
https://blogs.msdn.microsoft.com/oldnewthing/20071217-00/?p=...
[1] Seeing as I fell into it too, it's probably the intuitive formatting codes that messed it up, not the parent poster themselves.
*monospace it*Also how do you create lists?
Wasn't this always possible? even from DOS days, in fact?
In DOS, the filename is always FILENAME.EXT, and dot is not an arbitrary character is the name, but a separator. In FAT16 directory entries, you have two separate fields, 8 chars for filename, 3 chars for extension, both space padded. The dot isn't actually recorded, and only appears in reconstituted filenames. Consequently, it's not possible to distinguish "FILENAME" from "FILENAME.", and they are considered equivalent.
Similarly, when you write something like * .* in DOS, the dot is a special symbol as well, and this really means "match any filename and any extension". So this will match FILENAME, because it still has an extension, which just happens to be blank. On the other hand, if you just use * , it will be treated as * . and hence only match files with a blank extension.
In Win32, filenames are just strings, and dot is just a character. But the last dot in the name is still considered as separating extension from the name for purposes where it matters, like determining the app to open the file - i.e. "foo.bar.baz" is "foo.bar" with extension "baz". And if there are no dots, then the file is still considered to have an empty extension rather than no extension, and will therefore match . and similar globs. On the other hand, * is no longer treated as * . and matches any filename now.
They will also trip up in the same way Explorer does, on paths that are too long.
rm -Recurse -Force $path
EDIT: As a test, I cloned my local Maven repository a few times. This resulted in 48,613 files, 16,590 folders, and 2.71 GB on disk. Here's the result of:
Measure-Command { rm '.\.m2 - Copy' -Recurse -Force }
TotalDays : 0.00196887874421296
TotalHours : 0.0472530898611111
TotalMinutes : 2.83518539166667
TotalSeconds : 170.1111235
TotalMilliseconds : 170111.1235
EDIT2: Tried it again, with RMDIR and using the timing script found here:
https://stackoverflow.com/a/6209392
timecmd "RMDIR /S /Q .m2c > NUL"
command took 0:2:40.15 (160.15s total)
I might try it one more time after lunch with the DEL followed by RMDIR combo to see if that changes anything.
timecmd "DEL /F /Q /S .m2c\*.* > NUL"
command took 0:2:57.60 (177.60s total)This might be helpful if the filenames and sizes make you realize that you're trying to delete the wrong folder so that you can abort immediately. Other than that, it's just eye candy.
I'd probably try to cheat in this particular scenario either by keeping a best guess for the recursive delete complexity in the file system itself, or by simply showing a worse progress indicator such as a counter of files without a total count. The progress indication doesn't necessarily need to include time remaining, especially when the cost is this high.
Exactly. I don't particularly care which file out of half a million it's on -- I just want to know at a glance if it's still running or has somehow frozen/locked up.
http://technet.microsoft.com/en-us/sysinternals/bb896655.asp...
An alternative would be to use dir + attrib to make all files visible (I don't know that stripping out the system flag by default is a good idea) before running rmdir.
https://antibody-software.com/web/software/software/wiztree-...
Deletion tools that don't know how to distinguish junctions and folders may then find the original files via the junction and delete them...
(del and rmdir don't suffer from this. I did get a strange error message from rmdir, though, and it didn't actually delete the junction. GNU-Win32's rm blithely follows the junction and deletes everything in it.)
That's one hell of an indictment of Windows!
I was just trying to copy files between an old system drive and a new one but I kept getting infinite recursion issues with the "Documents and Settings" (which is linked to Users) directory, permission issues, etc. Even as the SYSTEM user. I wound up having to learn a little robocopy for it.
Incidentally, it's amazing that Windows still doesn't let you specify a custom location to create the profile directory of a new user.
So, I don't see how these commands should be considered fundamental to those users.
This is an age-old argument (GUI vs command line), and I think each generation coming into computing via the GUI will initially agree with you, but after some time trying to do things that are slow and frustrating using the GUI, will break out into song and dance on discovering the ease and speed with which the command line can do them.
It's all about using the right tool for the job. GUI is awesome for many things, the command line is better for many others.
Unfortunately, GUIs are still more difficult to build well. Once that is no longer true, the command line will soon cease to exist.
And even if such GUI would be created, I don't think it would be any easier to understand than learning CLI commands. And it would require significant overhead to use; consider for example how you would search files for content matching a regex - in CLI I simply type in the regex one-liner, but in GUI I guess I'd have to click around visually building the regex? This would be a nightmare to use.
GUIs definitely are not a good tool for vast range of tasks.
Regex is also not the CLI, it's a text-pattern matching DSL that you enter into your CLI as an argument. There's nothing stopping anyone from using regex patterns in a GUI.
If I had to type regex patterns I'd much, much rather type them into a GUI instead of a CLI because when I do this, the results can be instantly actionable without any further thought. Instead of having to do more CLI-programming to act on the results, I can typically act on them immediately in a GUI.
Better than regex though would be a simple GUI for handling the most common cases (match case, match whole-word, etc - the options that most IDEs offer) and a yes - visual builder for making complex rules for the more complex cases. I'll take a well-built visual builder any day of the week over a textual representation of a regex pattern that I have to use rote memorization or external references to understand.
Today's GUI systems are definitely not a good tool for a vast range of tasks. That's not my point though. GUIs have the potential to be way, way better than any CLI but unfortunately there are a lot of things holding them back like market forces and unimaginative people. I fully expect those things to change at some point, but probably not in my lifetime.
Well, you wouldn't, I would. Much better to get the work done, perhaps save it in a small script, and next time simply run it. Yes, I know GUIs tend to offer macros, but no way I'm going to trust them doing something without being able to see exactly what they try to do.
> Regex is also not the CLI, it's a text-pattern matching DSL that you enter into your CLI as an argument. There's nothing stopping anyone from using regex patterns in a GUI.
So you're not arguing for a full GUI, you still expect parts of the input to be entered as some cryptic text commands (which essentially is the same as CLI).
> I'll take a well-built visual builder any day of the week over a textual representation of a regex pattern that I have to use rote memorization or external references to understand.
That's fine, but it'd still be much slower to click out a complex regex in GUI, than to just type it out in CLI (not to mention I'd have to check the text regex generated by the GUI anyway, to make sure whatever I clicked out is actually what I want). You could say it's a good tradeoff of convenience vs speed, but that doesn't make GUI nowhere near "objectively better for every single task".
If they are objectively better for every single task, there will be metrics and studies that prove this objectively. Kindly cite these studies and metrics. I don't think they exist.
I refute the quoted statement with two words: headless servers. Oh, and REPLs. And CLIs embedded in so-called GUIs. In many cases, GUIs contain command-line emulators. Think of, say, Wireshark's filters, or any JavaScript console.
Maybe you are railing against text-mode displays, as opposed to terminal emulation programs that use graphics to emulate a text-mode display?
> Given a proper choice... No true Scotsman...?
Why: People found it interesting.
The quick answer to your question is that 16 votes in a short time beats more votes over a long time. And there are also various penalties.
Similar to the popularity of this topic: http://blog.zorinaq.com/i-contribute-to-the-windows-kernel-w...
> There is, in fact, a significant amount of of overhead when you trigger the standard delete action in Windows including when either emptying the Recycle Bin or directly deleting files via Shift+Del.
> Upon deleting the ~46,000 files from the NDK package, it took 38 seconds with console output enabled and 29 seconds with output disabled on a standard non-SSD hard drive, scraping off a quarter of the time. By comparison, the same deletion process via a standard Shift+Del in Windows Explorer took an agonizing 11 minutes.