Sure, if we knew nothing about Earth's climate (and also didn't have plenty of other ways to measure historic sea level), only having 50 years of measurements might be misleading. But we know that sea level is rising and we know why it is rising. We have a very strong hypothesis as to what's happening and we see this hypothesis confirmed again and again across a very wide range of subjects that otherwise have no relation to one another.
On top of this, we do have plenty of other measurements for historical sea levels that all indicated yes, sea level is raising they just aren't perfectly apples-to-apples "actual measurements" so it wouldn't be perfectly honest to include them in this chart.