It's called the hedonic treadmill [1]. There's plenty of things written about it if you want to go down the rabbit hole.
I do think that a lot can affect one's baseline though: health, drug dependency, stress from life, emotional baggage, etc.
And, I think that happiness can be achieved through practice (gratitude, letting go, etc..).
But yeah, that baseline - life gives some people a below neutral baseline.