Yea it's a problem. You're trying to hire from a pool that includes very competent people, and also people who legitimately can't write any code at all and are just trying to sneak in for 6mo to get $100k before they get fired, or think that "fake it til you make it" is a real thing. But at the same time, unlike other engineering disciples there's no PE licensing exams (or whatever the FAA does, I forget), no equivalent degree requirements (for good reason - one can learn to code without it, and a huge portion of a CS degree is very theoretical), but at the same time is qjite difficult, nothing like a plumber/electrician/etc certification level with accompanying work requirements and an apprenticeship program, and has very high pay and demand.

For very small shops willing to incur legal risk or limit the application pool, you can do take home tests and/or lean on side projects, with an hour of explaining the nitty gritty details and choices. At scale it becomes untenable, because you open yourself up for discrimination lawsuits, and you can't convince a lot of people who already have jobs and families to do lengthier take home stuff.

If I had to bet, I'd bet some day it ends up like plumbing. You work as an apprentice, learn real stuff on the job, and then there's some kind of state certification which itself is a joke, but requires you to have gone through an apprenticeship program already where all the real shit happens.

Companies trying to hire "the best" or whatever are still going to have problems, so as long as there's not much regulation increasing cost of dev and uniformity by an alarming degree, we'll still have insane hiring (and insane pay too, regs will make it unprofitable eventually).