The X100IV is awesome, of course, and if I could afford it, I'd probably own one. But it's more than 2x what I paid for my X-E3.
A fixed-lens camera is built around the limitation of having just that lens. To me, if I only bring the 27mm with me when shooting, then that is exactly like a fixed-lens camera. But it also means I have the option to take it on a bird-watching trip using my Fujifilm 70-300mm lens — something you just wouldn't be able to do with an X100. That flexibility is worth something, which in my opinion makes the lower price of the X-E range even more of a bargain compared to the X100.