Furthermore matter is just condensed energy, so why bother with some wonky immobile megastructure when you can just decay your everyday trash into pure energy.
True, matter is an energy... however the only known way to convert matter into pure energy is to make it annihilate with antimatter ( akin to electron-positron). We first have to have a good and cheap source of antimatter! And the matter annihilation may still generate a lot of particles/matter - that may be a waste (or not) depending what you do with that further.
A Partial conversion of a matter into energy is also fusion and fission reactions, but they are limited by the difference between source and end element of reaction here https://en.wikipedia.org/wiki/Iron_peak#/media/File:Binding_... - e.g. H2 -> He3, That uses only a tiny portion of matter, no energy that can be extracted from Fe56 (max), and all its close elements are also useless for either fusion or fission energy extraction.
It would be inefficient, but there is no physical reason we couldn't do it.
The other problem is scale: Dyson swarm power levels are orders of magnitude greater then the amount of deuterium in the Earth's oceans. At those sort of levels, you can't use fusion because you don't have the fuel.
Finally, based on what we know there's some real problems with compact fusion: ITER and other big vacuum vessels are because we're at the materials science limits of how strong magnets can be without destroying themselves.
Another big problem with nodes in Dyson swarm is heat, or the inability to get rid of it. They need to be close to the star so they can get as much energy as possible so they will heat up by position, but also modulate this energy and transfer it further, which won't happen without loses. So nodes can be constructed on gigawatt output but in reality be able to do fraction of that, otherwise they would fry itself up.
You would have to protect it pretty decently though..
Well let's compare star to the amount of available material in Solar System - I don't think that there is enough material to build one layer, let alone several + protective layer.
If you actually take every photon from Sol and point it in a single direction (think "LASER" rather than "mirrors"), the range is infinite, give-or-take the fact that you'll hit some objects on the way, and the intensity is on par with the intensity next to the surface of the sun.
The only hard limits I'm aware of are the engineering (which we're ignoring), and diffraction (which is small, I'll touch on). Diffraction-limited lasers have a wavelength / diameter term in the mix, so the relative increase in beam diameter is miniscule at any reasonable scale (doubling at 650M light years, tripling at twice that, quadrupling at 3x that, and so on linearly). Surface power scales with the inverse of that squared.
The engineering is pretty obviously going to be the dominant term. Even with some lasers we've already created (not to mention more exotic forms of wave propagation), dispersion (diameter) would be on the order of 750x at 1 light year, or ballpark 1.1e2 watts per square metre if I haven't screwed up too much arithmetic, contrasted with the 1.4e3 or so we experience on Earth when measured in a plane normal to the Sun and the roughly 6.3e7 you get on the surface of the sun and the roughly 5.6e4 where no matter your orientation if you didn't have protection or active cooling you couldn't prevent death by overheating just via sweating (that last threshold you hit from this hypothetical laser at 0.045 light years -- a small chunk of the way into the oort cloud).
How much better will we get the engineering between now and then? Who knows, but the math in that regime is _almost_ linear, and if you halve the angle of dispersion you double the distances in question for a given power flux.
Additional problem with this concept is, that if you will close a star into a sphere, won't it have effect on star itself, like destabilization of fusion process, because of excess energy? For example when star will start burning helium, it will get bigger. If you will prevent the star from getting rid of excess energy, won't it build pressure in the star, so it will start burning helium instead, expands and destroy the Dyson sphere in the process? Then after pressure decreases, it will go back into being a normal hydrogen burning star again.
Moving a star is kinda nice as it brings the system with it, so you get to keep your planets.
1. Being able to take your power source with you, so it also works behind nearest asteroid is even more important
2. Having your own source, so not to be dependent on one megastructure, because it is a single point of failure and single point of control of civilization.