If you want to test for memorization, you need a live test to do so.
If you want to test for comprehension and understanding, require sessions with a tutoring AI and grade the level of comprehension exhibited.
And then please, please test for prompting with challenging assignments that would usually be beyond a student's skills and make sure they know how to drive the machines to do valued work.
I totally agree that there is utility in memorization and comprehension. I also know that by the time students graduate, AI will be the job and if they can't pair successfully with agentic workflows they will not be much use in the work force for many roles.