If the first step of your build process is creating a new pip venv from scratch, it's entirely believable to me. Pip resolves packages sequentially, so there's a lot of low hanging fruit even without yoinking a state of the art constraint solver.

As a concrete example, I have a raspberry pi 4 I use for "what's the most pathetic hardware a user might try". It takes 5-10 minutes to build up a fresh venv even when it has all of the packages cached. If I nuke the caches and make it download, it's 20+ minutes. For comparison a modern laptop takes ~1 minute.