It's the railgun problem: railguns let you trade complicated, unstable ammunition for cheap, stable slugs of metal. But the railgun imposes so much wear on its barrel, and the barrel itself is so much more expensive than in a traditional gun, that you've obliterated your cost savings.
Likewise, it's irrelevant if fusion reactors can provide infinite energy from a single gram of hydrogen, because the reactor housing itself will be an impossibly complex machine with an extremely low lifespan from dealing with the energies involved.
Is fusion possible? Probably. Will it ever be more economical than solar panels? Nope.