I think the outcry here is that government has much bigger fish to fry than hounding taxpayers over a few thousands bucks worth of free food every year. Your slippery slope argument isn't too far fetched though: this same sort of tax policy is exactly how the U.S. ended up with its ridiculous employer-paid health insurance scheme.
[1] In some cases. See 91 F.3d 72