According to this no one has ever been prosecuted just for sending false DMCA takedown notices:
https://law.stackexchange.com/questions/51541/has-anyone-bee...
They do mention a case where the defendant acted in bad faith and was found liable for damages though.