That isn't an issue with Windows but with all file systems. When you delete a file it does not remove the data it marks the area it resides as "free" allowing anything to be saved there.

Even tails if not immune to this unless you use some secure wipe tool. Even they flash drives have wear level management that loves data around to make the wear on each chip equal so you can't be sure its even gone.