Indeed it would be more like Fargate in terms of container runtime vs lambda runtime, but the line between those is pretty blurry. Fly.io for example does containers but they spawn faster than some cloud function platforms. It's not really relevant for the discussion.
> Heroku on an allocated server with fixed overhead at all
I don't think you're familiar with Heroku then? I'm not talking about dedicated or managed servers. That's exactly what their 'dynos' platform has been doing since ~2008, though less granular: https://www.heroku.com/dynos/scaling
"All you have to do is handle the routing inside of the lambda using regular ASP.NET Core" also applies to both fly.io and Heroku. That's what I'm getting at. If you don't have a collection of independent cloud functions to execute, and you are spinning up a little monolith for every request, the platform seems like overkill.