The concept is known as a "vacuum airship". TL;DR: Materials aren't strong enough past very trivial volumes.
https://en.wikipedia.org/wiki/Vacuum_airship
This is a plot element in Neal Stephenson's The Diamond Age, as well as several other fictional appearances (Edgar Rice Burroughs, Azhar Abidi, Peter Watts, and Iain M. Banks all use the trope).
There are no materials known with sufficient strength to withstand compressive and buckling forces. Not even diamond.
Lightweight stiff structures (honeycomb, something resembling aerogel, perhaps) are other options, but seem unlikely as well.
Gas-filled airships or balloons benefit from the fact that low-density gasses (by either chemical composition or temperature in the case of hot-air balloons) exert a countervailing pressure to balance atmospheric pressure, but with a lower mass, hence providing buoyancy. The internal pressure actually provides some (or much) of the structural rigidity of most airship variants. Any vacuum airship would have to make up for this factor, again, increasing strength (and material mass) requirements.