In the US, if someone has your account number and routing number you can attempt all sorts of fraud via the eCheck system. People and companies are wary about giving out their account numbers. Providing an account number here is more often used to withdraw money than deposit. These payment services offer an extra level of authentication and authorization that does not appear to be easily accessible in the traditional banking sector.
Meanwhile, in my experience, in the UK sharing your account number is primarily used for other people doing a personal transfer to you. The institutions that can withdraw money directly from your bank account are limited to those in the Direct Debit system that has some vetting procedures and guarantees similar to what you would get in a credit card agreement[1]. This significantly reduces the chance of fraud, even with incidents like when Jeremy Clarkson printed his acct # in a newspaper on purpose [2].
[1] https://www.directdebit.co.uk/DirectDebitExplained/pages/dir...
[2] https://www.theguardian.com/money/2008/jan/07/personalfinanc...