I recently discovered extra-enforcer-rules which can help with that, as long as your classpath comes from Maven dependencies.
If you don't care (eg [0]) then do nothing and let the user bang it's head and reinvent the wheel.
If you care - just sort it and mark it as 'platform dependent' or somehow. Nobody really cares for the exact rules and a simple \d\d\-\w+\.conf pattern for the filenames is more than enough in 99% of cases.
[0] https://www.zabbix.com/documentation/6.0/en/manual/appendix/...
This would be a problem for first wins too.
There are plenty of applications that read a set list of configuration files and use either the first or last setting found. I agree with the responder that the last setting usually makes more sense.