Probably burned into an external EEPROM or flash, the FPGA fabric itself still has to be configured by some external means. It's pretty fast but on-the-fly reprogramming of the FPGA logic is still an issue where you want to very quickly switch the logic on the FPGA.
Most FPGAs I've seen go out of their way to allow you to reconfigure while it's running, you just have to jump through a few extra hoops and get down and dirty with the floor plan.
Older Altera CPLDs and Actel/Microsemi antifuse based FPGA families have nonvolatile configuration within the fabric. They enjoy the benefit of near instant start up since there is no programming phase to transfer a bitstream from an external flash part.
some have the configuration memory on-die as well to keep part count down. Also good to help protect your bitstream, but this isn't infallible of course.