This feels like a strawman: As evidenced by the fact that everyone would agree your example sentence is ungrammatical, clearly "less" and "fewer" are not strictly interchangeable, either prescriptively or descriptively.
I guess what I'm trying to say is that I don't see how your example is relevant to the cases where people do in fact use "less" to mean "fewer" (or vice versa).