I may be killing GP's joke, but here it goes. Potatoes are part of the regnum vegetabile, also known as vegetable.
But in the context of food, at least in the US, vegetables are understood to be things like broccoli, bell peppers, zucchini, asparagus, celery root, parsnips. Potatoes are not considered vegetables.
Of course, you don't want to get into the whole controversy about tomatoes being vegetable or fruit. Some people even take this controversy further, to say bell peppers are actually fruit, and not vegetables.