Your architect is almost certainly right. I would bet that most of your improvements came from fixing the drafts, with the insulation providing a marginal improvement on top of that.
I’ve also dealt with insulating old homes, but I did draft fixes, wall insulation, and attic/roof insulation at different stages. The draft fixes provided the most improvement, followed by attic/roof insulation. Insulating walls had much less effect than I anticipated.
In friends’ houses I’ve used my thermal camera (which I didn’t have back then) and it’s easy to see where the heat or cold is coming in during weather extremes. These days I’d recommend anyone start with the thermal camera view before deciding where to spend money on insulation.