Really it should be both, where the app asks you to input a code given from the person on the phone (confirms to you they're actually from the bank), and then gives you a code that you tell the person on the phone (confirms to them that you're really the customer).

Of course the more automation you put around this, the easier it becomes to MITM it, like a scammer simultaneously calls both you and the bank and passes the codes back and forth, pretending to you that the call is about a credit card offer, while using the call with the bank to drain your account. That's a lot harder to pull off with a human in the loop as the real bank person will get suspicious at the delayed responses, even barring some amount of stalling ("oh hang on I left my phone downstairs, let me find it oh god it's updating again, let me just get you that code, give me a sec here"). But it becomes trivial if the authentication is moved to IVR and by the time the human operator is on the line the call is already considered safe.