it's an up-front cost, but it's one that you'll benefit a lot from over the years.
I recommend starting with soft plaintext, like a diary. Over time, you can migrate to using it to edit code. My conversion took a couple months, and followed this pattern.
I started by editing plaintext diary files. Then, once I was very use to the in-file navigation, I installed vim-mode on my editor of choice at the time, so I could benefit from efficiencies there, while keeping all the IDE benefits.
Then, I began the process of replicating all of the features I used in my IDE, in vim. that took maybe another month, and was motivated by how painfully un-snappy my IDE was relative to vim.
This is all to say, it doesn't have to be an all-or-nothing cutover. That would be painful and frustrating.
While I love Vim and I use it most of the time, I don't really buy it when people act like it saved them a whole lot of time. I love the Vim keystrokes and philosophy, but I don't really know if I'm any more productive with Vim than I was with Emacs...I stick with Vim because I find that, once you "get it", it's relatively easy to translate stuff from your brain to the screen.
I wish most of my time was spent writing code, but a vast majority is spent reading it and swearing at my monitor because things don't work, so I don't think even a perfect editor would make me a lot more productive, at least not directly, except (as you stated) maybe through me liking the editor a bit more.
I personally spend as much time as I can in the terminal, and so my "IDE" is actually is typically NeoVim, tmux, some kind of language-server, and a command line. Am I more productive than my peers using IntelliJ because of this setup? Almost certainly not, but I find it very intuitive for me to do stuff with, since I'm super familiar with the tmux keystrokes, and I find Vim's design really intuitive for me.
Really if it weren't for the first challenge of learning Vim, I wouldn't have learned to program. Which, I know, sounds kind of backwards :P