But on the other hand human brains are basically just black box neural networks as well.
You can just ask a human brain what it's optimizing for.
But the answer you get isn't necessarily the truth, because humans don't make decisions based on reason. 90% of the time, decision-making is based on emotion, and the reasoning comes afterwards to try and justify why the decision was made. Thus, when you ask the person for justification, you're getting post-hoc reasons.
There's a great book written on this - "The Righteous Mind: Why Good People Are Divided by Politics and Religion"
"Senator, is the safety and prosperity of the American people your highest priority?"