I don't understand why these should be mutually exclusive. Starting from a common parseable markup you can have both responsive HTML for browsers and formatted PDF for print. Why don't decouple content from presentation?
However gov.uk already does both so it's not an actual issue here.
Having hard pagination and consisstent layout is, for me, a cognitive gain, esspeciially for longer documents, say, 20+ pages. (I frequently read 500+ page docs.)
Other formats such as ePub are frequntly compact in space utiisaation, but again, the free-flowing text lacks the mnemonic framing of even a basic print book, let alone the expertise of a masterpiece of layout & typography such as Tufte.
Not that HTML isn't well-suited to other cases, or that PDFs can't be awful. But there's a place.