The length of time notarization takes depends primarily upon how large and complicated your app is, and how different is from previous versions of the same application you've previously notarized. The system seems to recognize large blocks of code that it's already analyzed and cleared and doesn't need to re-analyze. How much your binary churns between builds can greatly influence how fast your subsequent notarizations are.

A brand new developer account submitting a brand new application for notarization for the first time can expect the process might take a few days; and it's widely believed that first time notarizations require human confirmation because they do definitely take longer if submitted on a weekend or on a holiday. This is true even for extremely small, trivial applications. (Though I can tell you from personal experience that whatever human confirmation they're doing isn't very deep, because I've had first time notarizations on brand new developer accounts get approved even when notarizing a broken binary that doesn't actually launch.)

And of course sometimes their servers just go to shit and notarizations across the board all take significantly longer than normal, and it's not your fault at all. Apple's developer tooling support is kinda garbage.