I too ride titanium and have ridden my share of both time trial and racing bikes made of carbon.

There is no speed benefit for a non professional on a carbon frame, but wheels and positioning are a totally different story.

I ride titanium only these days because it looks good and its a tiny bit more soft than carbon, so for longer rides its more comfortable.

Bit I ride 40mm corima wheels or 60mm dt Swiss when I need to go fast in the bunch. And that makes the most difference along with my posture on the bike.

And if you lose 2 kg you can ride on a cheaper frame and get the same result due to your own weight.

Cycling for amateurs is more about looking good, egos, and spending money than it is to make the sacrifices around what it takes to go really really fast.

But ride your bike. It makes one happy.