I was using Google Voice for a few years. When I got Project Fi, I had to release my old number and get a new one. It was disappointing, but less annoying than I thought it would be (since I don't use my phone much).
Currently you don't have to give up your google voice number. You can even use it as your google fi number if you like, and you can undo that choice and get it back as google voice later.