There are tiny little converters inside.

sometimes they go in packs together with "blanks" so that in total they provide similar voltage as 3xAA

LiFePO4 chemistry is at about 3.2VDC so a blank/filler will make it a drop-in for 2X alkaline which is what I see a lot of devices use. You can get "AA" size ones in the garden light section of home centers as they're used for solar path lights.

Lithium primary batteries are available in 1.5V nominal — not sure if they have any circuitry.

A typical lithium ion "14500" battery has that issue, yes.

There are some more unusual ones such as Kentli, which have built-in buck converter to give you 1.5V as a substitute for normal AA batteries. It's a clever design where the top of the cell has two separate contacts, the normal button-top gives you 1.5V, and a recessed ring around that hooks up to the actual battery cell for charging purposes. So you need a special charger for these.

Images and more info here: https://lygte-info.dk/review/batteries2012/Kentli%20AA%20280...

I've never actually seen any of these (have a bunch of NiMH cells and one fancy charger already) but it's a cool system. I don't think the capacity is significantly better, but devices that have trouble with NiMH's lower voltage would do better with these.