Any government willing to fund a large infrastructure stimulus project should be willing to fund a large scientific research project. It is not just throwing money into a black hole.
(Of course, whether any kind of stimulus spending is desirable is a different question.)
Some of the infrastructure needed to support a large particle collider (roads, power generation and distribution) may be of general utility, but with current technology, most of the money will literally go into digging a "black hole" and filling it with superconducting magnets and detectors. None of that is good for anything other than particle physics (and at this point, even that is questionable; the most plausible result of a Higgs factory would be just another confirmation that yes, the standard model works, with a few new decimals tacked on to various constants).