So I really dislike Jupyter, and I've tried using this[0] before to ssh into Colab and do work in a terminal setup.
You have to be careful to back up your code frequently (what I did was push to Github) since your ssh session goes away when your "kernel" (or whatever your colab session is called) goes away. You wouldn't have this worry if you were just using the web interface, since the code is always saved.
I'd imagine that, if you want to use VSCode, it's remote editing features, which I keep on hearing a lot about, would come in handy to edit over ssh!
[0] https://github.com/WassimBenzarti/colab-ssh