I understand that it's from Gulliver's Travels where it's about which end to start breaking an egg from - but without knowing this you can easily end up getting this wrong.
The word "End" can also mean any "extremity" and not just the opposite of "beginning". Otherwise phrases "on both ends of the spectrum" wouldn't make sense.
Thus, a positional encoding of a number has one side (end) where the impact of digits is much higher (big) than the other side (end) where the impact is lower (little).
Little end: the side with lower "weight" Big end: the side with higher "weight"
Being "little endian" is a property of the encoding or architecture, not the property of the word. The word is not "little endian", i.e. its "end" is not "little". The encoding is little endian in that it starts with the little end of the word. You're rightly confused because the fact we're now suddenly talking about the start of the word is implicit and based on the assumption that the reader knows Gulliver's tale.
If end means start, then why use this word?
Little endian is in reality little startian and big endian is big startian.
In fact, we could simplify this even further and just call big endian, startian and little endian, just endian.
It's also quite an old terminology which is not going to change.
If we could come up with a new terminology from the start we could find better options.
For example:
* Least/Most Significant First (LSF / MSF) * Low/High Address Least Significant (LALS/HALS)
Etc
u16 x = 1;
u8 * px = (u8 *)&x;
int* x; // x is an int-pointer
int *y; // dereferencing y gives an int
int * z; // int multiplied by z
(For the record, the second way is the universal and objective truth.)