IIRC, Einstein actually spent quite a bit of effort trying to reformulate electromagnetism as a geometric curvature just like gravity, but eventually published a paper admitting failure. I recently saw some physicist talking about revisiting that with some new tricks because maybe it wasn't such a bad idea after all.