People are worse at mental arithmetic than they were in the recent past, so it's not clear that they aren't "dumber" in the sense people meant at the time.
We also used to be much better remembering things, when we relied on oral histories, our memory skills have degraded quite a bit. And there's a quote from Socrates criticizing how writing is a crutch that degrades our skill (https://www.perseus.tufts.edu/hopper/text?doc=Perseus:text:1... , the last bit). Over time, we've just moved to valuing other things more.