The "meritocratic" system is as big a fraud and every bit as bad (if not worse) than the inheritance system.
The majority definitely couldn't judge merits, since avg Joe knows nothing about surgery or economics.
The institutions couldn't regulate themselves as well, since this leads to corruption, where the most politically active are getting to the top and starting to regulate on the loyalty basis. We need external checks and balances for institutions as well.
We can't have a meritocracy without an external authoritative voice judging, what's the merit. Hence any meritocracy or even technocracy applied is ended up being a authoritarian, ineffective corrupted mess. We had this in soviet union, we had this in china.
The idea that people should be judged by merits and the best scientists, academics, writers should be at the top ended in soviet union with the distortion of the very notion of merits and values: careerists got to the top and ruled out actual smart people, turning institutions into corrupted mess.
Most of the accrual of wealth in the 'meritocratic' system also goes to the inheritance class - in your example, see a shitty sword and think you can do better? Get a world-renowned swordsmithing coach, have the finances to work at it all day for however long it takes, import the best materials, buy the best workshop, then reap the rewards - all while talking about "your hard work and dedication" and claim everyone else could do it too.
It's total, and utter, bullshit.
Meritocracy is a lie, but it's sold as a feel-good truth: you could just come from the ghetto, work hard, and by gosh, be successful! Of course that isn't the way it plays out almost ever - we tell ourselves that the supremely rare few who it does work out for could be everyone else if they just WORKED HARD. It's a LIE.
Meanwhile, we've accepted this disgusting fiction, and we place the blame squarely on REGULAR PEOPLE - any system that requires you to be exceptional to succeed (or even have food security!) is a failure, but one that cloaks the actions of the inheritance class as "merit" (as they are best able to, of course, "compete" - with coaches, perfect nutrition, educational trips, etc) is MORE insidious and evil - and it doesn't actually displace that same inheritance class!
The technically literate, ironically, eat that meritocratic shit up for some reason. The same people who built surveillance capitalism, destroyed the poor with the "gig" economy because it always "seemed to work for them!", built the methodology of tracking everyone's persuasions and beliefs online to be used for propaganda or ethnic violence, who repackaged sub-prime mortgage suffering as a "data science" problem.
Tech people, especially those who believe in "meritocracy", are so painfully blind, they don't even understand that the wheel will come around for our skill set - we're in a weird period of history where our technical skills are extremely profitable. Rather than making the world better, we've caused the kind of damage that drives a stake in the heart of our very society. Very Promethean.
If we were as wise as we were smart, we wouldn't be here - we're the brightest fuckwits in the whole world.
Of course it's a roll of the die. Of course you might end up with a cretin or a criminal. Of course it's made worse by inbreeding which comes with inheritance. But it's still the best roll of the die you have.
Just one example is worth having a meritocratic system, IMO. (Btw, meritocracy isn't based on hard work, just on output - whether it's genetic, or a result of effort, or whatever is irrelevant.)
Even with all the trash I listed about it? It's a lie and a shit belief.
These can be objective metrics. They can be measuring pretty much anything you want, and you usually find a way to get better at it.
I think finding the right things to measure is the most challenging part, almost because of the fact that you can usually 'game the system'.
But to deny merit completely or find it in contempt is in my unearned opinion, a fear response in denial of the massive amount of skill a small but influential number of people develop in all sorts of endeavors.