GitLab Product Manager for Verify (CI) here.
This is really valuable feedback, thank you for it. It's something we are thinking about and working on fleshing out a vision for. Two epics we have around these ideas include [1] Making CI lovable for monorepos and [2] Making CI lovable for microservices.
Both of these challenge our current assumptions around the project::pipeline relationship and will help make room for improvements around that model and provide flexibility to build more complex models that work for "real world" problems like the ones you've stated. However, I don't believe we have all the answers yet so I would love more feedback on these and the issues attached.