Probably either Skunkworks or Kelly: More than my share of it all. I recall that very soon after shipping the U2, they saw the writing on the wall for its operational limits and started working on the SR71 very soon after. Looking at the timelines, U2 first flight was 1955 and the first discussions for the SR71 were done in 1957 with work beginning in 1958. Powers was shot down in 1960 in his U2.
Every time one of these threads comes up I think I should reread Ben Rich's book, which I read over twenty years ago and gave to a friend about five years ago. It would be interesting to recall his perspective on this. I think he had something in there about how proud they were about keeping the costs of the reactivation and final flights of the SR-71, which were sort of mandated by congress, as low as possible. They were still a lot more expensive than, for example, recent programs to equip the U-2 fleet with more modern avionics.