Maybe take the approach many physics classes embrace (in germany) :
All tools are allowed in tests: calculators, books, as long as the correct answer is derived with "correct derivation path".
If chatgpt makes some problems too easy maybe it's a good thing because we can raise the bar. Find problems that require true understanding beyond auto complete / copy pasting.