It's the same in the UK but those filter paper looking ones actually used to contain small amounts of plastic.

I believe most have transitioned away now... or, at least, the only brand that matters if you're British has transitioned[0]

[0] https://www.yorkshiretea.co.uk/brew-news/plastic-in-tea-bags...