You can, but it's not always a good idea. The 'caution' here (our equivalent to the US's Miranda warning) is "You do not have to say anything, but it may harm your defence if you do not mention in questioning something which you later rely on in court. Anything you do say may be given in evidence". The intent is to prevent 'ambush' defences (introduce an alibi at trial, for example, giving no opportunity to investigate it) but it has the consequence that a "no comment" interview can be tactically risky. (And that we have no real right to silence, of course!)

In principle they shouldn't be arresting you unless they have sufficient grounds for suspecting an offence, though there are the usual accountability problems in enforcing that. The recent case of ST is a good illustration of that[0]: a 14-year old was arrested at dawn on a flimsy case of participating in the robbery of a mobile phone at school. The arrest was mostly to search his room, as well as to ensure attendance at interview, which is legal under PACE, but he was willing to attend voluntarily, as well as to agree to the search, and the timing of arrest and decision to hold in custody rather than immediately bail were also probably unlawful. He sued for false imprisonment. He lost the case because the first instance judge decided that 'whilst the arrest and detention were "reprehensible", the way the police powers had been exercised "lamentable" and it was doubtful that the actions would be "justified in the court of public opinion", there had nevertheless been reasonable grounds to suspect the Appellant of an offence of robbery and there had been reasonable grounds to believe that the arrest and detention were necessary.'

(This was overturned on appeal to the High Court, but it does show what a high bar there is for holding to account the use of powers of arrest).

[0]: https://www.bailii.org/ew/cases/EWHC/QB/2022/1280.html