What does "cover their bases" mean?
As them to explain what they are trying to achieve and find other ways to assuage their concerns.
The only legitimate thing is to have something in case you fail and they have "banked" on you. There is a legit way to solve that. basically if they want that tell them they should pay for an Escrow service - that will hold your code and they would receive it if you ever cease to exist. But it's important that they would need to pay those fees (they are significant.)
That should make them back down.
It's entirely unreasonable for them to demand access to source code.