"major hostilities of World War I were formally ended at the 11th hour of the 11th day of the 11th month of 1918, when the Armistice with Germany went into effect." https://en.wikipedia.org/wiki/Veterans_Day
So any attempt to 'move' Veteran's Day will likely meet resistance due to this connection. This is also why this day is a fixed calendar day (at least in the US) rather than a "first/second/third Monday of the month" type holiday like most of the other official federal holidays. It shares the same connection with specific exact calendar dates as does the 4th of July, Christmas, and New Year's day.