VS Code does become slower for me on my MacBook M1. I typically don't restart the app unless I have to. After few days of usage, scrolling the code area becomes so slow (<20fps) that I have to restart the app. I haven't experienced such slowdown in other IDEs, such as Xcode or Qt Creator.