The engineer wouldn't have been able to waste the week feeling fancy fiddling with GPUs in that case.

Since the variables are categorical, won't PCA have to be modified to use it effectively? To my knowledge, PCA can only be used for continuous variables.

Yes. I think so. Or maybe some correlation plots. I doubt that a proper EDA was performed pre-modelling.