A way to distinguish form submissions (like a credit card purchase form) which must not be repeated after submission, from things like "upload a new version of this file" which you can safely allow to be retried if they time out or fail.
This is potentially useful for a bunch of different HTTP clients and client libraries, and potentially for some proxies too.
Edit: actually PUT and DELETE are useful for caches - although not because of their idempotency, but just because they communicate the fact that the resource being PUT to or DELETEd may change as a result, hence the cache key for that URL should be purged. IIRC some caching proxies do do this.
What they should do though is purge their cache for that URI when they see a successful PUT has happened.