Castle garrisons were small. Even a large-ish castle probably didn't have garrison larger than 25 - 30.

And food doesn't take that much space either. Consider how Age of Sail ships would spend months at sea without resupply, and how a castle has much more space available for storage. And even with medieval technology, you can make food that keeps long: salted, smoked or dried meat; dried fruit; marmelade; hardtack; and so on.