One of my kids' doctors left their practice, and we followed them to the new. The old practice sent us a letter stating patients were not permitted to do so, and they refused to provide records citing a non-compete. (I had more than one sweary phone call with them over this.)

HIPAA has no private right to civil action in these scenarios, so we had to involve the state, and that process took about six months. It's definitely not fast; we were happily in the scenario where the records weren't time critical.