It would be possible to clearly watermark a PDF file with the serial number of the person who checked it out of the library. That way if it is leaked it shouldn't be too difficult to find the person who leaked it.
I would definitely use such a library if it existed, provided that it wasn't too expensive and had novels or books I could not find at my local (and free) public library.