The middlemen you're talking about do two things: buy lots of reserved compute on the hyperscalers, and then pit the hyperscalers against eachother to get better pricing.

If you're reserving thousands of GPUs from the same hyperscaler, even if they're the only cloud you run on, you're not paying the price shown in the calculator. If you have other suppliers, you'll get an even better deal. Then you resell that reserved compute as on-demand compute, somewhere between your costs and what your customers would pay a hyperscaler directly.