This happened to me with Citibank. But, I called citibank and got a new card number. Then, citibank gave the new card number to the fraudsters when they tried to charge me the next month.
Unfortunately, I’m going through a divorce right now so I can’t cancel that credit card. But, as soon as I’m free that’s on the top of my list of things to do to start living the rest of my life.