I bought a console as a gaming device, but now my family mostly use it for YouTube and other streaming video. Similarly, relatively little of my phone time is used on phone calls. I think the distinction is mostly just locked in by history.
But sure, nothing on a technical lecdl stops you from using a console as a general computer. It's just that that's not what the overwhelming use is as of now. Use cases play a large factor in rulings like this.
Consoles compete on games. Phones compete on specs, and then they happen to have an app store on the side. Thats a difference.