Rare earth refining traditionally generates a lot of toxic waste of various sorts. This could likely be mitigated but it would raise the cost of the finished product well above current market norms, so unless universal standards were applied, it would become unprofitable relative to China. There's a good overview of the difficulties involved (and some alternative approaches and replacement materials) here:

https://news.climate.columbia.edu/2023/04/05/the-energy-tran...