Hell, I’d argue a similar prompt could be useful for a lot of people. I use something similar for coding and ask it not to provide direct code and more a guide on coding principals with source links if possible.
I fear this will push even more people into deep rabbit holes they won't be able to get out of because they think this neutral AI has confirmed their suspicions/ideas/observations.
And you can see the influx of people on r/physics and the like who are convinced they've solved dark matter/quantum gravity/... because ChatGPT kept agreeing with them when they presented their ideas to it. Just recently there was a post by a guy who essentially "rediscovered" 17th century physics with the help of ChatGPT but was convinced his formula would explain dark matter because ChatGPT told him so.
The idea was to use ChatGPT as debugging ducky 2.0