An admittedly intelligent and nice guy he wasn't a particularly competent president. Looking at his term through the lense of sympathy rather than history gives a distorted picture.
A US president never gets to see the results of his policies in the first mandate. Realistically, it takes a year just to get to grip with the duties, and by the time they're productive the midterms arrive and dramatically reduce space for bipartisan compromise. Once that's done, soon after it's time to gear up for re-election. Even if they manage to pass anything significant in those short windows of activity, new laws have to get into force (which can take a year), kickstart the various first-order and second-order effects (again time passes), and then go through some sort of statistical collection - itself typically a process that gives results over 6-18 months.
US Presidents effectively get judged first and foremost on the day-to-day executive orders, which is why they tend to be keen on starting wars. If they secure re-election, only then they'll get to be judged on the actual legacy of their policies.
In light of comparisons of the current President to him, the current meme seems to be to consider Carter “an actually very good, misunderstood President” in the hopes of neutering the insult.