Unless the staff operating the public service receive no compensation at all, then there is profit involved.
I mean, I prefer to envision a world in which state-sponsored armies (not necessarily navies) are deprecated entirely, or at least relegated to non-standing status (much like the US constitution originally prescribed).
But even holding this somewhat anarchist view, I can still make a distinction between the incentive stuctures of a for-profit model and mere compensation. If you're so sure-footed about view point, walk us through the deconstruction that makes these things indistinguishable in your mind.
The canonical example is that the US purchases hundreds of tanks a year even though there is no real current military need. There are at least a thousand on hand mostly sitting in storage while relatively few are used for training.
Maybe there's an interesting thing to say here - some people people are against for-profit prisons because they think they encourage imprisoning more people as it creates more profit.
But if you have a non-profit, government-run prison in your local area, you may still be encouraged to imprison more people as it creates more profit for the local suppliers to the prison.
So yeah it doesn't matter if the final user is for-profit if you think there is a risk that profit is driving something you don't want.
This is a gross simplification that erases the very real differences between the incentives of a government vs. a for-profit company. Not only is the government accountable to its electorate, but a government that wants to save money can take a much broader approach - for example by funding access to mental health services - to keep people out of prison. Prisoners are expensive to keep locked up!
In the final analysis, all prisons are non-profit. None of them generate any value. They're all funded with taxpayer money that could be spent elsewhere. It's just that in the case of private prisons that money goes to a company that has no interest in saving the government money by reducing the prison population. Sure they'll try to run the prison efficiently, so they can pocket the difference, but they want as many prisoners as possible to get a bigger check from Uncle Sam.
Now the three groups aren't equal, but they do exist on the same scale. For profit prisons are, IMO, the worst while the other two groups are of debatable ranking depending upon the size of the relevant organizations.
In a "nonprofit", the paid staff cannot be paid contingent on the fundraising efforts of the organization. The nonprofit organization has a separate legal existence from the totality of its membership. A "not-for-profit" is identical to the totality of its membership, and may pay its staff or members out of general funds. Nonprofits are 501(c)(3) tax-exempted, not-for-profits are probably 501(c)(7) tax-exempted.
For example, a religious organization might be organized as nonprofit. It has a high priest, several underpriests, and some knights on its payroll. These are all paid predefined salaries and wages. They get paid the same regardless of whether the temple collects a lot of tithes, or none at all. If a nonprofit could not hire paid staff, it would likely have insufficient volunteers to accomplish its stated purposes.
A performing marching band might be organized as a not-for-profit. They go out and sell candy bars stamped with bars of musical notation to raise funds. They can spend the proceeds of the fundraising on themselves, or their staff, but it has to be in furtherance of the organization's stated purpose. They can hire a band director, arranger, and transportation coordinator, and book hotel rooms for themselves. If they raise more funds, they can spend it on themselves, and also give their staff bonuses. But they can't just distribute excess cash; that would be profit. They could buy touring buses and instruments and uniforms that are more expensive and worth more at resale, which would be suspiciously similar to profit, but the rules for "not-for-profit" are more lax than for "nonprofit".