Yes, in theory. But it's testing against highly complex, ever-changing systems, where small changes can have big impact on the outcome. So it's more akin to "weak" science like psychology. And weak here means that most finding have a weak standing, because of each variable having little individual contribution in the complex setup researched, making it harder to reproduce results.

Even more problematic is that too many "researchers" are just laymen, lacking a proper scientific background, and they are often just playing around with third-party-services, while delivering too much noise to the community.

So in general, AI has also something like the replication crisis in its own way. But on the other side, the latest wave of AI is just some years old (3 years now?), which is not much in real scientific progress-rates.

except the area is so hugely popular with people who unfortunately lack the rigor or curiosity to ask for this and blindly believe claims. for example this hugely popular repository https://github.com/x1xhlol/system-prompts-and-models-of-ai-t...

where the authors fail to explain how the prompts are obtained and how they prove that they are valid and not a hallucination.

yeah, but it's a different type of science.

the move from "software engineering" to "AI engineering" is basically a switch from a hard science to a soft science.

rather than being chemists and physicists making very precise theory-driven predictions that are verified by experiment, we're sociologists and psychologists randomly changing variables and then doing a t-test afterward and asking "did that change anything?"