A lot of ISPs already have policies for not having daemons on ports, so they could use that as justification for not having your own IP completely accessible from every port (which I don't believe would be possible without explicit port forwarding from the ISP itself).
What would this mean for end users' internet? Not much, but IPv4 IPs would be shared by multiple users (hopefully not many at first). A lot of websites have session policies and all sorts of mess based on the remote address, so it will be interesting to see how the aging webapps deal with the change.
I figure servers will get first priority at v4 addresses. Yeah, there's useless crap going on like IRC vhosts, but I figure once the market closes in on the addresses people will likely sell them (hell, I know people squatting on address blocks until the 'apocalypse', thinking they'll score a huge profit!).
Fun times.