It's a combination of disparate factors. Most labs were set up under different initial conditions, and their future contracts and direction spun out of their initial intention. There are specific nuclear security/safety labs (Livermore/Sandia/Los Alamos), specific fundamental research labs (Fermi/Brookhaven), and more generalist labs (Oak Ridge/Argonne). This is not to say that each lab doesn't have significant overlap.
Livermore may be fundamentally a nuclear security/weapons lab, but they also morphed into a HPC and biotechnology lab, while I worked in more of their general energy research directorate, which covers everything from hydraulic fracturing and carbon sequestration, to novel enzymes for breaking down free methane. To give an exmaple, I never interacted with the people working at the National Ignition Facility, nor was I allowed in their building except on a tour (it was an awesome tour).
Oak Ridge has a general Environmental Sciences Division, which ranges from genetic engineering, to ecology, to soil sampling, to water power technologies research. That's just in the single building where I work, and these are just small subsets of current missions, while labs have specialties in certain areas, such as Livermore being great for lasers, or Argonne being known for transportation research. Oak Ridge has a big campus, and it's easy to never interact with other buildings. I believe both Oak Ridge and Livermore have 7k+ employees.
The labs are also managed completely differently. Livermore is technically run by a corporate board after it was restructured away from the University of California system, maybe 10 or so years ago. Lawrence Berkeley Lab I believe is still in the UC system. Oak Ridge is managed by a combination of Batelle and the University of Tennessee. I want to say Fermi and Brookhaven are managed by national physicist groups or something, although I'm not certain.
And finally, of course, the overall lab culture is influenced by their hiring practices and regionality. Before I became involved in this system I had no idea they were all so different, and the really amazing breadth of projects across all the labs. Hope this helped elucidate it a bit.