My recollection is that the whole experience depended on which group you were in, and mine was fortunately very chill. Smart, friendly people who arrived and left more or less at the same time every day. Lots of matrixing and loaning of people from different orgs -- I had the feeling that if I were making a career there I would wind up slowly drifting around between projects.
The biggest surprise to naive me-in-my-early-20s was that "Department of Energy" is a euphemism for "Department of Nukes." Nuclear stockpile stewardship was a large portion of the activity there, and so a lot of your colleagues will be people who are at least vaguely comfortable with that.
There was a ton of "basic research" too -- some high-energy group had a daily experiment that would deliver a "whomp" of a shockwave around 3:15pm most afternoons, there was a room temp. fusion group, lots of interest in assisted driving cars and unmanned aerial vehicles... you just had to appreciate that all the first applications of all this tech was going to be military.
Also the security clearances.... the joke was that the "L" clearance stood for "Lavatory pass" because in our building until you got one, you needed a line-of-sight escort at all times, even in the bathroom. Even for the "L" the process was quite onerous, and I understood that the 'Q' clearance held by nearly all full-time staff was even more burdensome. I heard stories of people waiting for their clearance getting stuck in rooms with nothing to work on. One person in my group basically got sent offsite to some "think tank" or something for several months while he waited for his clearance - I only met him once the whole summer, at a conference.