These days, yes! Gemma4 is only a few gigabytes and is surprisingly capable and can run on normal consumer hardware. You could certainly run it off a thumb drive