WebDAV (2010) | Hacker News Reader

WebDAV (2010)

webdav.org

62 pointsskilled48 comments

A few years ago I buit a personal file storage server [0] that was interoperable with OwnCloud and sync based on WebDAV. Over a decade ago I worked on a site builder that used WebDAV under the hood. What I found was that WebDAV is mostly fine if you control both client and server, but then there's less point in using WebDAV. But the allure has always been there: let your users mount your app as a filesystem and update your app's content locally using their own tools, with syncing for free.

In practice, the syncing was never reliable enough for most users, and different clients implement the standard differently. My hope was that it would eventually converge on something interoperable like CSS did, but I don't think there's enough usage for that to happen. And things are moving away from filesystem paradigms anyway (iOS, etc) so WebDAV is slowly becoming even less relevant.

0. https://github.com/mnutt/davros

In practice, the syncing was never reliable enough for most users

I wonder if that's why iDisk went away. Wasn't it based on WebDAV?

Yes iDisk was WebDAV.

More or less proving your point, the ownCloud client changed it's requirements, and Davros hasn't worked with it in a while. :(

I used it for an Adobe Acrobat project years ago. Acrobat Pro let you store comments via WebDAV in a way that you could collaborate on a single PDF and see all comments from all users.

But, you needed to set everything else up on your own. I built a system that filled in the gaps that was used to proof corporate documentation and clinical drug trial materials.

Worked great for years.

Oh WebDAV, the protocol I loved and hated the last 20 or so years...

- There were proprietary implementations NetDrive and rebranded as WebDrive for winNT which set up a WebDAV mount as a drive letter. Worked nicely for code editing in a CMS but completely corrupt files once in a while. Unless you had a SMP machine, then it would regularly blue screen

- WinXp had the first native ‘NET USE’ support I think. It was built on top of the HTTP support in the OS but didn’t persist state properly, so if you didn’t know to disable automatic proxy detection in your internet options, every WebDAV action would take 2 seconds to do a proxy detect, ruining the experience.

- Actually getting WebDAV to work was hit or miss. It helped to specify the password on the command line to NET USE for some reason

- office supported WebDAV too, and depending on its mood would either treat a WebDAV drive letter as just a drive, or speak WebDAV natively

- I’m pretty sure the initial Vista release completely broke WebDAV. Never got it to work properly. Sad given that MS was supposed to be invested in this standard

- Getting it to work safely on OSX was another challenge, as OSX heavily caches content client side and if the server side content changed, either remount to see the changes or write a tool to send a poorly defined control command to WebDAV to flush the cache. At one point we automated the server to inform a local tool on the macs to do this flushing

- oh, and automating mount_webdav to log you in requires speaking some property protocol over a fd to the application. Which broke at around OS X 10.10. Got to learn my way around the WebDAV implementation in Darwin back then to figure out both this and the cache flushing. Thank apple for at least publishing the source to this

Well, fortunately I don’t need WebDAV as much anymore as I have in the past.

Were these problems you experienced a failure of a bad spec or a collection of software poorly following a good spec?

- office supported WebDAV too, and depending on its mood would either treat a WebDAV drive letter as just a drive, or speak WebDAV natively

It's kind of funny that Office still sometimes shows this behavior with OneDrive for Business. If both are reasonably up-to-date (2016+ on both sides) it appears to use the modern new (Microsoft Graph) API that isn't WebDAV at all, but every now and then it sometimes gets confused and seems to fallback to WebDAV or just normal folder operations.

It's probably the Office team's devotion to backward compatibility and supporting the weird number of configurations that enterprises sometimes have between Office client applications and SharePoint server backends. But yeah, it's still easiest to describe as "moody".

Not all that related, but we found an exploit in Blackboard (the "virtual learning platform") by smuggling null bytes (0x00) via. their WebDAV protocol.

This made it possible to hijack other accounts, including our professors'. So we hacked our own grades and then reported it.

Blog post here: https://bustbyte.no/blog/how-we-hacked-blackboard-and-change...

Less exciting, but I found with some reverse engineering that School Loop (basically the same thing as Blackboard) has a WebDAV interface for their "Locker" (file storage for classes) interface. When I was working with it, I was finding that my queries did not work correctly; apparently their client for it did not work either because implementation was borked (which took a while to figure out!). AFAIK they never did fix it once I reported it to them, but the official client ended up being pulled at some point leaving my implementation the only interface to it that I know of…

Student here. We use SchoolLoop at my high school and the locker feature has been a pain. Just wanted to thank you for your work on Break. I recommend it to people all the time (plus, it's open source!)

Thanks; it's nice to see that people still find break useful. It's gotten somewhat difficult to work on, as I no longer have a live account to test with nor do I use the app anymore, so there are some bugs that have driven me up the wall for years but I can't figure out how to fix. I'll try to keep it running for as long as I can; maybe someone else will be able to figure it how to fix the issues ;)

Since I have a live account, expect a few PRs soon, once I get familiar with the code :)

We implemented WebDAV (rust/actix-web) for our product and it's great that you can mount it as a mostly native file system on all major OSes, but there are a lot of issues with Implementation:

* Windows sometimes appends a `/` at the end and sometimes doesn't when looking for folders, so you better make sure you return the same results for either.

* Windows WebClient service is buggy as hell. You have no event logs (that I could find) to find out what's wrong and end up restarting the service continuously. this includes if you get your username/password wrong, if there is an interrupt to the server (i.e, wifi drops), etc.. Wireshark is your friend, but it won't tell you why it's not working.

* Forget about Basic Auth on windows, credentials are not persisted across reboots/logouts, and you need to adjust registry to support non-ssl sites at all. Using Kerberos is really your only option, but if you want to use it on non .local domains, then you need to add in a registry key to allow this, as windows assumes that only .local domains would use SSO.

* Last time I checked, Adobe Lightroom won't even save to a WebDAV drive on OSX as it thinks there is no space.

* Lots and lots of noisy connections on both windows and OSX that aren't really needed, but happen anyway. Don't get me started on the amount of temp files that are saved/used, and don't filter them out either as things start breaking like Photoshop and folder uploads.

* Weird error messages that don't give you an idea of what's going on. Error code 100000041? Oh yeah I know what that one is!

* URL size limitations in Windows, make sure your entire path isn't too long!

All in all I am glad we have WebDAV for native-like access, but it is a massive pain to deal with some of the niggly bits between implementations.

I remember playing with WebDAV in the early 2000s. Microsoft Windows and/or Office and/or Sharepoint Client had (two!) implementations which supposedly would be able to present a WebDAV share as something that would look and behave more or less like a mapped drive, which would have been a killer feature for anyone developing an online file/document/collaboration service. (For example, imagine exposing your web CMS media library or even page structure as a mounted drive via HTTP). This was long before FUSE was a thing.

Unfortunately, the amount of crazy bugs combined with the unnatural and surprising behavior of Microsoft's two(!) implementations (which one got used depended on which part of the UI the user came in through) made the whole thing a flaky disappointing flop.

Also, the whole XML protocol feels totally over-engineered and overly verbose.

See for example:

https://www.greenbytes.de/tech/webdav/webfolder-client-list....

https://www.greenbytes.de/tech/webdav/webdav-redirector-list...

Almost anything XML is over-engineered. It's sad because WebDAV is cool and useful as a "public http drive", and better than FTP as far as reliability.

> Almost anything XML is over-engineered.

I hear this a lot, but I feel that with many XML protocols (including WebDAV), if you reformatted them as JSON, people today would think they were pretty reasonable. XML was just the popular style back in the 90s / early 2000s.

And it surely beat the other common alternatives at the time for directory listings: raw /bin/ls like formats for FTP or a HTML dirindex for HTTP

Did it though? At least microsoft usually managed to parse ftp and html dirindexes. The bugs they had in their webdav clients (see link in parent post) was something else... incorrect/lacking xml entity handling that almost indicates substr'ing for "&" instead of using a real parser, and converting percent-escaped utf8 to percent-escaped latin1..?! You almost have to try to sabotage on purpose to get bugs like these.

The XML spec is just so complicated, with tag attributes, xsd, dtd, etc. Although it has nice capabilities, it can be tricky to map some random person's xml document to another data structure. But JSON seems like the pendulum just swung to its opposite apex

I never really got webDAV to work properly but I gave it a good go. Today I would never think to use it, I am more of an sshfs type of guy and rsync is my friend.

However, what does disappoint me is how things like DropBox and WeTransfer took over with some marketing that just had instant appeal to people. I prefer to have my own 'sovereign' server space and if anyone wants to send me stuff then I want it on the server, not on my desktop. It doesn't matter who it is, but if I give them a network drive of sorts, they can have webDAV, FTP if they insist, ownCloud if they want it, SFTP and all these methods to put whatever the files are on the server. They can even understand that the files need to be on the server and not on my local computer, but no, I get some email about a weTransfer link.

With ADSL the download is fine but upload is that bit slower. So you have to do the WeTransfer type of thing and wait for that, then upload the file to the server. It is never me that really wants the file on the server in the first place, it is always them.

Anyway, I think webDAV died as a thing not due to the flakiness of the implementation but more to do with why it is that people use website builders like Squarespace, why people prefer cPanel instead of some standard issue OS, why people prefer whatever is marketed to them as 'easy' rather than what takes two minutes of critical thought to learn.

As a user of cPanel, there's some assumptions you're making in this statement. I can and do build servers, at work. When I'm at home, I just want to do things with as little work as possible. Sometimes these choices are made because it's easier to not have to support every last bit of an architecture.

I learned something a few days ago that made me both laugh and cry:

LoadLibraryA will happily load a DLL from WebDAV through a UNC path. Something like \\example.com@80\path\to\payload.dll.

It goes without saying that this has been abused by viruses to surreptitiously fetch their malicious code.

Is that much different than a get and load in the end?

It's a pretty useful attack vector since you can get an arbitrary program to load your payload under certain circumstances, so you don't even need malicious code running if you can find a vulnerable target. cough SharePoint cough

Yes -- you may not be able to convince a program to download a file, but you may be able to tell it to use an improperly sanitized plugin name via a static, non-executable document that someone downloads and tries to view.

What is this submission attempting to express? The main thing I noticed is that the webpage hasn't been updated since 2009 and the WebDAV working group appears to have abandoned it for unspecified reasons. I frequently use WebDAV via a very popular iOS app and never had any issues with it, so I'd be interested to know why it has been abandoned.

Because the clients/server quality varied (very fragmented standard compared to it's relative use amount) and for what it did plain HTTP or some pre-existing "proper" file exchange protocol likely met the need better anyways.

I have the same question. What is the purpose of this post?

During a week in 2013 I setup http://sabre.io/ to work with storage backed by S3, I think it was about 40 lines of PHP to add user authentication and the S3 storage plugin. It's so seductive since every file manager on every main GUI OS has some support for WebDAV, and you'd think "what could go wrong?" with just file I/O over HTTP... Some months later I even tried setting something up for my old high school teacher for his CS students to copy files over and receive results back, alas the district network policies got in the way and I'm not sure if it was ever resolved.

I was recently working on an nginx lua script to aid with impersonation (setfsuid, setfsgid, setgroups) so I could implement a multi-user webdav server, mostly with the goal of pointing it at some existing file mounts and making sure permissions were checked/honored correctly - especially file ownership. I need to test it a lot more, but in case anyone is interested:

https://github.com/lsst-dm/davt

FastMail gives me 10GB of storage that's accessible via WebDAV. It works perfectly to back my collection of papers and textbooks in Zotero. That way I only sync the metadata to my Zotero account (which is basically just a sqlite database anyway), and can pretty easily get access to my entire reference collection from wherever I am.

Mirror https://archive.is/O0pi9

I am using WebDAV as a virtual file system for a personal project. It was the easiest way I could find to make a cross platform virtual file system without invoking FUSE which seemed slightly unreliable on Windows.

It works reasonably well though it is a bit slow. It was pretty difficult to find good documentation though.

I use a WebDAV server on my Android phone and assign a drive letter to that from my Windows Vista machine so I can use BeyondCompare to copy my pictures off. I also use it to explore my phone files at a level slightly lower than what the standard apps allow. Probably pretty basic cases, but for me WebDAV has been simple and reliable to use.

I implemented a personal notes app recently using WebDAV. It's kind of mind blowing how many simple applications you can use WebDAV for and actually end up writing no server-side code whatsoever. Many things where you might think you need a database for, consider using WebDAV instead.

Just used the elementaryOS files app to access my Nextcloud server. A while back it used to be terribly slow but it's actually incredibly useable now. It's a shame this protocol has been shunned. I believe Microsoft Exchange used to support WebDAV for email access...

I’m not sure I’ve ever seen it anywhere in Exchanges protocol specs. Would seem an odd fit

Perhaps you’re confusing it with Sharepoint?

WebDAV wasn't a native Exchange protocol. But it was officially supported for Exchange server 2003/2007, and used heavily by Mac Entourage 2004/2008 to connect to Exchange servers as a supposedly "native" client.

In my experience, Mac Entourage clients connecting to suitable Exchange servers worked, but only for very few clients.

Vaguely remember that the Exchange servers would often get overloaded with the quantity of sync requests from the Mac clients.

This doc mentions that WebDAV was a previously-supported protocol for Mac Entourage: https://blogs.technet.microsoft.com/exchange/2009/01/20/ento...

Nah definitely Exchange. My old BlackBerry could connect to Exchange not via a BES server using the WebDav interface.

What does this do that fish://<ssh connection> doesn't?

"Files transferred over Shell protocol (FISH) is a network protocol that uses Secure Shell (SSH) or Remote Shell (RSH) to transfer files between computers and manage remote files."

"The protocol was designed by Pavel Machek in 1998 for the Midnight Commander software tool."

https://en.wikipedia.org/wiki/Files_transferred_over_shell_p...

Googling (or DDGing) for "fish://" is just maddening

It has native client support in Windows and macOS, which is kind of a big deal in some scenarios.

Fair, but typing fish://192.168.0.108 at home works out of the box for me in Linux (KDE).

We built a Ms word version control system with webav to s3 for our saas product. We couldnt find a better solution for check in / check out. Is there something better?

anyone used https://dokan-dev.github.io/ to mount Linux drives on windows?

For those that weren't around, WebDAV is one of those historical horror stories, like SOAP and CORBA. "Great in theory..."

Yes, WebDAV, XML, SOAP, CalDAV, etc. are from an era in computing where interoperability and common standards were important and demanded by customers. What have XML bashers to show for exchange and long-term archival of documents?

Reply on news.ycombinator.com