Close. It's not publication by the government that is important. It's authorship.
If a work is a work prepared by an officer or employee of the United States Government as part of that person’s official duties, then it is not subject to copyright in the US.
It might be subject to copyright in other countries, though.
If the government pays for a work to be created and published, but it is not a work prepared by an officer or employee of the United States Government as part of that person’s official duties, then it will be copyrighted and the copyright owner will be whoever it would be normally. If there is an agreement between the author and the government to assign the copyright to the government, the copyright will still be valid, and owned by the government.