Here's what you are missing, the relative populations of native born and foreign born have changed over that time period. If you do the same thing for the population data (https://fred.stlouisfed.org/seriesBeta/LNU00073413) you see that the graphs are almost exactly the same. % of foreign born population goes from 17.5% to 22%, versus the 18.3% to 23.7% you called out for job change.