(I don't work in the field, I just do photography as a hobby, but as let's say an "advanced amateur" interested in lens design and history:)
CNC optical design really hit big in the 90s. The problem space for lens design is obviously high-dimensional and large (number of elements, element composition, element placement, element size, curvature of each optical surface, cement between the elements...) and that made it possible to just brute-force a bunch of different lenses and tune it all.
For prime lenses, that pretty much solved "reasonable" lenses for "reasonable" angles of view, "reasonable" aperture/intensity, and "reasonable" numbers of elements. A 7 element lens of a particular focal length, designed in the 90s, is significantly better than one designed in the 70s, but probably about the same as today.
There has still been significant progress made on the "unreasonable" lenses. For example f/1.4 lenses from the 50s were poor wide open, from the 70s they are mediocre, in the 90s they were decent, modern lenses they are excellent wide open. Superfast lenses, superwide lenses, super-sharp lenses, and lenses optimized for beyond-visual light or wide-spectrum photography (IR or UV at the same time as visible light) have all benefited significantly since the 90s.
Lenses have been moving to more and more elements over time, as well as using more exotic shapes and materials, all of which gives more ability to correct the image. A lens from the 50s would have been around 5-7 elements, all spherical (all surfaces are a sphere of some radius), using basic types of glass. A modern superlens will incorporate several ultra-low dispersion elements (typically grown fluorite crystals) and use at least one aspherical element, on top of having a significantly higher number of elements in general (7 elements was a lot in the 70s, modern prime lenses might have 12-14, with 3-4 of them being exotic).
This may seem like an obvious statement but optics are a very complex materials science problem and on top of the optical design the materials and manufacturing have become much more advanced over time. At the dawn of photography, basic crown glass and flint glass was about all that was available. The coatings are also very important - every time a beam of light goes from air to glass or vice versa, it refracts a bit. Many of the modern optical designs used in the 70s to 90s were actually known quite early - the Planar lens type (used in your standard 50mm f/1.8 lens) was invented in 1896, and the Plasmat type was invented in 1918, but the number of air-glass interfaces meant the contrast was very low on those early lenses and it was not practical. Instead, fewer elements produced more contrast (because of fewer air-glass interfaces) and elements were often cemented together with balsam pitch so that you didn't have to have an air-glass interface between them (for example in the Cooke Triplet or Zeiss Sonnar designs). More advanced designs had to wait for the advent of coatings just before World War 2, and then there was an explosion of designs after the war. And multi-coatings (Pentax SMC was the early winner) did the same in the 70s, and the expiration of the SMC patents enabled another explosion in the 00's. The same applies to glass types and other components of the lenses, most recently aspheric elements and fluorite.
And yes as you mention it has always been hard to design good zoom lenses, they are as complex as a "super" prime lens even with a relatively modest zoom range and relatively slow aperture. "Superzoom" lenses covering a giant zoom range are even harder to do. The 70-200mm type has been around a long time and good 70-200 designs were available in the 70s for the professional market, then in the 2000s the pro market got 24-70 f/2.8 and 24-105 f/4 types, and more recently there's been some good consumer ones like 17-50 f/2.8 or 18-35 f/1.8. So the field of zoom lenses is still very much advancing.
so tl;dr basic prime lenses were "solved" in the 90s, really high-end prime lenses that do exotic things and zoom lenses are still making progress by increasing numbers of elements and using more exotic elements.