The unemployment rate only counts people looking for work. If they quit the labor force completely, they're no longer counted as unemployed.

Minimum wage workers are usually the lowest skilled workers in the economy. When they're displaced they have the fewest alternative jobs available to them, so they tend to drop out of the labor force. Either temporarily, while they go and learn new skills, or permanently.