Looks like Microsoft dropped the ball on this one - it's a government, they could probably have charged billions. Hopefully, the UK government will realise this, and either a) go open source b) get their house in order much earlier next time.
They only have option b) if they won't to avoid this in the future. Open source or not.
In any case, if they have the usual volume licensing deal with Microsoft, they can use any version of Windows without charge.
A big example is Gov.UK which is mostly mostly open source as well as a few other big UK gov services like police.gov [1]
A large part of the NHS was moved off of a proprietary oracle system just a few months ago [2]
[1] http://github.com/alphagov [2] http://www.theregister.co.uk/2013/10/10/nhs_drops_oracle_for...