When I started my UK Limited company I immediately registered for VAT, so I could claim VAT paid on purchased startup equipment, which was mostly computing goods. I could then obviously offset that against VAT collected in the first months of business.

I quickly discovered that quarterly VAT returns were a pain, and accountants aren't cheap (for some definition of cheap).

I am also just a one man band at the moment, and de-registered for VAT about a year ago. Unless there are significant savings to be made, or you're over the £82k per annum revenue threshold, factor in the cost of the effort to submit these returns. They really are just a distraction from the main focus of your startup business.