Why nobody knows what they're doing
sites.google.com
sites.google.com
That said, if I had to guess a motivation for the technique, it'd be to stop people getting attention by wildcarding subdomains and linking up nonsense like click-here-for-my-awesome-blog.domain.com, etc. Not that such a thing should avoid getting flagged anyway..
It's a kind of dated argument, since it doesn't deal with the client being able to look at the web and get an idea of what's possible or not possible.
In all fairness OP cites lots of examples where success is usually elusive, so it's easy for him to make the jump from "usually" to "always". This kind of thinking scares me, not so much because it's so myoptic, but because it becomes a self-fulfilling prophecy: "I have never seen success" becomes "Success is not possible" which becomes true because we stop trying.
There is so much to examine here, but I'd like to focus on OP's 10 examples of things that change. Mere mortals throw up their hands in futility, blaming the user, the state of the art, or the alignment of the planets, without ever understanding the fundamental principal that great systems accomodate great needs and the greatest need of all is constant change.
the company recategorizes its product line based on utility and lifestyle rather than brand and designer
This is incredibly common and can easily be accomodated with proper database design, decoupling products from their associated attributes.
the parent company divests itself of you and withdraws the connection to its parts and inventory database
So now you have to treat them like any other vendor? You do have some of those, don't you?
a manufacturer starts using more than one UPC per product because they changed the packaging
A simple database design consideration. "One to one" is just a special case of "one to many".
the law now requires that national holidays must be counted against vacation time if the employee takes a vacation day immediately before/after that holiday
Rule driven logic. What's the big deal? (Your logic was rule-driven from the outset, not hard-coded, right?)
a loan company files to become a bank and store deposits that earn interest
Additional logic to existing data. We do this all the time.
CSV files exchanged with a partner were always denormalized, but then one company starts using normalized records with mixed column mappings
Any decent batch file processor should be able to handle anything thrown at it. Companies that do a lot of this build uncertainty into the design of their systems.
another partner decides to ditch file transfer as a way to submit data, and creates a web service that you're supposed to invoke commands on
Sounds like you'll need an extra piece of software to create a feed. Not exactly like no one's ever done this before.
the shipping carrier reveals that their tracking numbers are all recycled at the end of each year
Not a problem if they were never used as the primary key to a table.
new rules require every credit card number to be encrypted, but you still have to be able to quickly search for matches in a database of millions
Ahh, but the last 4 digits do not have to be encrypted. Start your search with them and resolve synonyms. Pretty standard stuff.
a partner switches to mixed-case alphanumeric PO numbers
As an identifier, a PO number is always a string even if it looks like it's numeric. (When was the last time you added 2 PO numbers together?)
OP's roadblocks are experienced developers' speedbumps. Just because he hasn't done it doesn't really mean that "Nobody knows what they're doing".