You can though, there is an options field in IPv4 that could have been made to hold extended address values.
It was used as part of the Extended Internet Protocol: https://www.rfc-editor.org/rfc/rfc1385
And the Address Extension protocol aka IPv7: https://www.rfc-editor.org/rfc/rfc1475