I get the feeling that's because gluten-free food is - for some people - medically necessary, while people with, say, allergies to some animal product e.g. eggs will usually just buy normal food and be careful not to buy anything containing egg. As a result, "vegan" food has a negative perception that "gluten free" escapes.