The answer to that could be burning things, along with better storage tech that you mentioned. Not necessarily fossil fuels, but biomass and gas (e.g. "cow farts"). Burning biomass, you can still stay neutral if done right, as the mass will use carbon when growing. Things like algae farming as a fast growing source of biomass may become important. With some additional tech you could filter out some carbon and other emissions and store what you filtered out, and have a negative emission balance considering the whole process.
Biomass and other renewables generally have the other positive aspect that research, development and installation is relatively cheap and quick compared to nuclear.
Another part of the solution surely will be to just use less energy, by investing in more efficient tech. Reducing energy consumption by merely 10% would essentially cover what nuclear in Germany produces now[0]. A lot of companies in Germany already started investing heavily into modernization to reduce their energy need, and this process is speeding up now. A lot of this effort is driven by public policy - from tax incentives to negative incentives in higher energy prices - but a lot is also driven by the realization of management that modernization efforts like this can massively drive down costs mid and long term while also ensuring the production facilities stay competitive with tech upgrades. Another way of policy incentives e.g. are the mandatory labeling of consumer goods in regard to energy consumption and efficiency, as consumers will use this information to buy stuff that's not only "better for the environment" than the competitor but also saves money on the electric bill. Right now, within the EU, such labeling is mostly for electric appliances, but it could be extended further to other areas like food or services, and could cover the entire life-cycle of a product including production and disposal, not just the time consumers operate it.
By the way, I am not generally opposed to nuclear, just opposed to nuclear how we do it now and did it in the past. For one, it's artificially made cheap by subsidies everywhere you look, and that artificially cheap price does not even include the additional costs of long term waste management. And then the plants are rather unsafe, not because they have to be, but because of policy failures. Things like building nuclear plants in regions prone to earthquakes and/or tsunamis, the French having to shut down a lot of their reactors every few summers because they'd be boiling their rivers if they didn't and of course incidents like Chernobyl, Fukushima, Three Mile Island (thank the Air Force for having an airbase nearby which lead to TMI being build more sturdy than usual thus probably avoiding a really bad outcome), Windscale, etc... all that does not inspire confidence in me that humans and in particular politicians can manage the technology. When Belgium discovers "micro cracks" in high pressure boilers in nuclear power plants, and then their politicians almost immediately declare the plants to be still safe - in a timeframe where they surely cannot have thoroughly evaluated the findings - then that's not great already, then when I consider some people want to build massive nuclear power plants in areas of the world with far less stable societies than Belgium, I want to facepalm.
And on top of that nuclear is not renewable. The fuel has to come from somewhere, which doesn't bode well for energy independence. Right now, a forth of uranium is mined by Kazatomprom a quasi-state-owned company in Kazakhstan, which Freedom House rates a "consolidated authoritarian regime". Being dependent on Russia for gas isn't great; being dependent on Kazakhstan for nuclear full isn't great either.
PS: I would agree that getting rid of (most) fossil fuel energy production is more important than getting rid of nuclear.