When the likelihood of the risk is small, but the potential damage bill could be financially catastrophic, I think it's perfectly rational to buy insurance if the premium is so low.
When I bought my current house my solicitor suggested that the seller should pay for insurance cover for the fact that there was no planning paperwork for a small side extension. Of course I said OK and of course the seller also agreed. But that extension was even shown on the land registry plan and was obviously 20+ years old. Since planning breach action is limited to 4 years there was no chance of any problem. Free money for insurer, commissions, etc.
Like, are the chances of Belway complaining and landing us in legal trouble close to zero? Yes, they are, but like said above - it was £50 and I got the seller to pay for it. When we sell the house I'll do the same just to avoid any bumps in the process because of it.
Another thing: You already say that you would be fine paying for the person you will buy the house in the future. However, many indemnity policies run in perpetuity so in principle will still apply and there will be no need to buy a new one.
If it's only a small one-time payment, then it probably makes sense to pay someone that much just for doing the work to verify that it's not going to happen.
In my case, the development became immune after 4 years, that's the most common case (e.g. extensions, conservatories, fences, etc)
[1] https://www.gov.uk/guidance/ensuring-effective-enforcement
There was that one guy who covered his castle in hay bales for just over 4 years before revealing it and they took enforcement action successfully, but that's a bit different. I'm guessing that's why you said "became obvious that building works had been completed" but actually the rule is simply completion date but the courts decided that this guy was taking the piss so much that he still violated the intended meaning of the law. Your odd agricultural shed example may well fall foul of the same thing but I don't see what that has to do with the parent comment.
It's 10 years (not 12) in some cases but they wouldn't concern most domestic developments.
Any insurance of the type referred to by the parent comment is 100% definitely a swindle.