You're not crazy. That's exactly what Japan chose to do for the latest block of Sōryū-class attack submarines. They removed AIP engines from the design and replaced them with a huge pack of lithium batteries. Underwater endurance is still limited, but there are significant operational advantages to the simpler battery based system for defensive submarines that will never deploy far from Japanese waters.