Midi controllers are great, but setting them up can be a negative experience. The other big issue with software synths honestly is DRM. Some have DRM that is just as bad as video game DRM. I got locked out permanently from a VST I was using to generate melody data, and when I reached out to support they told me nothing could be done and I would have to re-buy the VST again. What happened was I had to keep inputting my license because occasionally, the license info would become disassociated with my VST. What I didn't know was this counted towards the max number of installs my license was capable of.
The other big issue with VST is you end up in a situation where you cannot update your operating system because over time, VST's become unsupported and will not run correctly when you update your OS, DAW software, or audio drivers. This is an absolute disaster in terms of security.
I would rather work within the limitations of synth hardware than deal with all the pain that comes with software. I still use VST software though. Aside from VCV Rack Serum is outstanding and will do almost anything you could possibly need a synth to do, and it does not have crazy licensing issues.
I think most artists use a hybrid hardware/software workflow, but there are definitely disadvantages no matter what path you choose.