IANAL either but I believe that if you release code under license A you can relicense it to license B, but anyone that obtained the code under license A will be able to keep on using it under that license.

A license is like a contract, and you can’t change a contract out of the blue without the permission of both sides. Otherwise licenses would be useless, as at any point someone could go “aha, I changed my license to the give-me-all-your-money license and you’re using my code, so pay up!”.

Since the license A in this case is an open source license, anyone that has a copy of the software with license A (such as amazon) will be able to share it again under license A. So effectively the old code base will always remain under license A, and only new changes will be fully protected by license B.

IANAL, but AFAIK while Amazon could ‘unpublish’ the code by removing it from their Github or something like that, there is no way to ‘unlicense’ it.

On a project without a CLA, you’ll have to ask permission to every contributor for a use that goes beyond the initially agreed to license. On a project with a CLA, the organisation to which the code has been licensed no longer needs to do that. The situation becomes similar to a project that only has one author and no outside contributors. So they can release a commercial version in parallel, or they can decide to change the license for new versions. But none of this magically changes the licence for old code: code once released stays under the license it was released under. Otherwise the whole system of Free and Open Source licences would be extremely brittle, if authors could just revoke their licences as they saw fit.

If one wants to contribute to something under an Apache license but exclude certain part, one is expected to explicitly point the exclusion out as in “this part is not delivered under Apache 2 license”.

You might want to read the Apache2 license text. It's not that complicated, at least not the in your case relevant section(s)