Albany is older than NYC, and anyway the concept of NYC as a single large city is barely 125 years old. The original capital of Illinois is (strangely) west of the Mississippi River and south of St. Louis. By treaty between Great Britain and France, the current area of Illinois was off limits to European settlement, until after the revolution when the US said “that treaty doesn’t apply to us”. Chicago wasn’t a big or important city until the civil war.
Los Angeles has been the wealthiest and most populous area in Alta California since like 1800 or so. But it was established as a puebla and not a mission, so it could never be a center of government under Spanish rule.
Miami was fairly unlivable until air conditioning and malaria control, and its geographic location was extraordinarily inconvenient until after the interstate system.
The US government established a customs house in Olympia when it and Seattle were little more than a handful of homesteads. And guessed wrong on which would grow faster.
Similar for the Washington Territory which was barely a collection of homesteads and tiny villages. IIRC Olympia was the first significant immigrant settlement in the northern Oregon Territory, the major (only?) early trade connection to shipping on the pacific, and connected to the few other settlers moving north in to the territory. Its not very surprising that it was the territorial capitol and remained so for the state. Yes, post statehood a lot of rail & shipping traffic moved north. But at least through the turn of the 20th century both Tacoma and Seattle were roughly equivalent in population and industry. Arguably the easier ship access for the Klondike is what cemented the prosperity of Seattle pre-war, and Boeing & related defence activity post war. But _at the time of territory & statehood_ Olympia was the major town in the region.