As you say, batteries have been around for a long time. The rapid growth/improvement part of the battery curve happened back in the late 1800s/early 1900s.
The lead-acid grid lattice design, still the...errr...gold standard when it comes to the most amount of joules stored per buck, was invented in 1881.
(modern technologies like the various lithium battery chemistries win when it come to storage for a given mass -- thus their use in things like portable devices and cars, but lead-acid still wins when it comes to storage for a given cost).