SHOULD is not MUST
As per RFC2119 (http://www.ietf.org/rfc/rfc2119.txt):
3. SHOULD This word, or the adjective "RECOMMENDED", mean that there
may exist valid reasons in particular circumstances to ignore a
particular item, but the full implications must be understood and
carefully weighed before choosing a different course.So caching DELETE is against the spec.
I, however, agree that this behaviour shouldn't happen.
It does not mean that if you receive a DELETE request you can just substitute the value of what is a completely different request (the GET request). I believe eloisius is not correct about the spec banning this behavior with that line. It's "banned" because it's just broken, nonsensical. It's not the same request in the first place, any more than you can just substitute POST results for GET.
Yes, you're right.. the spec doesn't mention what should happen when the developer does something completely irrational, like returning a cached GET response to a DELETE request.