One of the oft overlooked, yet critically important aspects of the scientific method is the hypothesis. You don’t design an experiment having absolutely no idea what to expect. You have an educated guess in mind (the hypothesis), and you design the experiment in such a way that says “this result will rule out my hypothesis, while this other result might confirm it.”

Just trying two things at random and picking the one that makes some arbitrary metric go up, is not the scientific method. It’s gradient descent.