Peer validation platform for engineering skills (inspired by X community notes)

As a hiring manager know what you want and be careful what you communicate to determine your desired audience.

Secondly, skill verification is a form of leaky abstraction and may not be what hiring managers want. It’s most often not about finding the most skilled candidate but rather the most compatible candidate. This can commonly mean finding the least sucky person from a pool of sucky people.

Hiring managers can game that by setting requirements criteria for a job. If you want extremely skilled people then get down to the metal and find candidates that like to work without tools or abstractions. If you super versatile candidates find people with experience in a bazillion different tools. If you just want a body to fill a seat that is quick to hire/fire select for the latest trendy framework.

The best way to determine the right candidate is to ignore the nonsense on their resume and just talk to people. Dig in and see what they really want and then challenge it with questions only they can answer. Most people are bodies just wanting to be hired without bringing anything special to the table so that’s what most employers target for. The real challenge here is finding people that are highly skilled in a market where exceptional skills outside the bell curve are not commonly rewarded, because these people will not self identify as awesome when looking for work in compatibility driven system. The people that most typically do identify as awesome tend to not be as awesome as they believe.

This is a real insight "finding the most skilled candidate but rather the most compatible candidate", thank you very much, this is the kind of feedback I am looking for from engineers. I never looked at it from this perspective.

I talked to hiring managers and most of them say skills is good but what we want is a personality because skills can be taught and personality cannot be. Personality I have experienced that, very skilled teammate exceptional, but personality was not a fit, so the management was not happy.

"The best way to determine the right candidate is to ignore the nonsense on their resume and just talk to people.", that is true and this I have tried with when I was playing the role of hiring manager trying to find a team member to work with me in the company I worked for. I imagine the peer-to-peer verification framework like this talk to people and ask them personalized questions, this is how exceptional talent will be flushed out I think.

Given that, if you were to imagine a tool that helps surface those 'quietly exceptional' people—the ones who are highly skilled but won't self-identify—what would that tool look like? Does it even need to be a tool to begin with?

> I talked to hiring managers and most of them say skills is good but what we want is a personality because skills can be taught and personality cannot be.

That is correct, but its an extreme over simplification. What they mean by personality is among these:

* excellent soft skills - can listen and emotionally bond with others

* excellent written communication - the ability to put into writing high precision content with minimal revision that is well structured. It should not take advanced effort to turn meeting notes into a formal written recommendation.

* discipline / conscientiousness - this is awareness of the space outside yourself. It is the ability to balance 6 things at once with endurance throughout the day.

Yes, technical skills can be taught. Hiring managers really prefer to retain their people and not have to rehire, so teaching people is a long game of minimal continuous effort from the leader via proper assignments and continuous practice on the part of the individual contributor. If the individual contributor also does this work as a hobby outside the office that is even better, but it would be a massive ethical violation to impose this or it intermingle it with assigned tasks.

I am a hiring manager, by the way.

From a tooling perspective I am not sure. My organization uses a tool called GreenHouse. It contains a candidate's resume and credentials and allows HR to build out forms for interview feedback. The insight part is always a challenge. I am fully remote so I am actively watching everything a candidate does with their eyeballs as much or more than listen to the content of their answers, because I want to know what they are thinking and what they are looking at since AI prompts during interviews is now a thing.

When I do look at a resume what I am only looking at:

* Years of total experience

* Job hopping versus loyalty/duration

* Credentials like education attainment, certifications, and so forth

Yes absolutely, I am looking for framework that will be included in the platform to be able to asses someone and determine those accurately skills to have them in the CV it is not easy and it is a challenge, overall those skills can make a teammate be valued among others.

The AI interviews part is becoming very common as well, this is the issue I wanna solve since it is very easy to fake it while remote.

I have another question, have you ever faced with getting a perfect candidate during interviews this person passes fully it is great, everything matches but then after 1 or 2 months the just completely flip and are not the candidate(now employee) what you tought and everyone else tought they are?

Yes. So, I work for a defense contractor and we can hire somebody into a start date within a week, but it might take a couple of months before they can enter the office, even online. During this period is where most people fall apart. Either they cannot get their administrative requirements together, cannot get their required certifications complete, or end up with a competing job they find better. They have not even gotten to the social interaction part of the job yet, but we are already starting to get a sense of their ability to lean in and participate by the time all their roles and permissions come in.

Then we have a 90 day evaluation period where I write something up and sit down with the employee. I have never had anybody fail out by this point due to performance. With on-boarding so long everybody has time to really settle in. Even after they can come into the office/online there is still a honeymoon period. From a supervisory perspective it gets real at the 6 month or 12 month evaluation based upon the volume of assignments and quality of participation. I am super chill as a supervisor so as long as the employee gets their work in at the client's timeline, keeps the client happy, and is as helpful as possible to the rest of the team everything is kosher. I have released candidates for negative performance almost never, but it has happened.

I have had an Army general officer tell me in the past that to get there they just have to do the job they are assigned and don't put their boss in hot water. I heard the exact same thing once at Bank of America, and it works for me as well. The people that bubble up for promotion tend to be the people that accomplish the most from the least effort, not the harder workers, because those are the people that can be assigned their same job plus an additional job. That does not mean taking shortcuts either. Part of doing more means overwhelmingly helping people in the present for building out relationships to leverage into the future.

That is very interesting, it is like a starting filter for candidates which can catch candidates that might not be a fit for the job after all. Basically the candidate was lacking the discipline and patience to be able to handle paperwork and administrative duties to even start doing the job. Got me thinking how would it be possible to catch something like this before it happens, would be a very big challenge

So it all boils down to, handle the administrative challenge first, then make the boss happy and not make his life harder as well as everyone on the team and lastly for promotion simply be efficient, I like this framework it works so well.

I think these all work for any industry, since defense is a bit stricter then there is a higher chance to avoid a bad hire.

> Hiring managers really prefer to retain their people and not have to rehire

In my experience, the only time a hiring manager has had the authority to overcome HR mandated maximum percentage wages was when my manager was a director/CTO and I was a strategic employee. That’s what causes “salary compression and inversion.”

Line level managers who most junior developers report to are completely useless when it comes to almost anything except for advice about dealing with the politics of the organization and reaching across teams.

You may sincerely want to retain talent but (the royal ) you probably don’t control budget. If you are working in a large tech organization, the promotion process still goes through a committee and is out of your control. Even when you do get promoted, you’re probably going to be making less than a new hire coming in at your same level.

Getting back to the uselessness of line level managers, when I was interviewing to be a “strategic hire” for two companies between 2016-2020 before I got into consulting, one of my conditions was that I would report directly to the person who had signing authority for budgets and/or major decisions so I wouldn’t have to deal with powerless line level management.

Congratulations, you have just reinvented a feature that LinkedIn has had for years and not solved how easily it can be gamed.

And honestly, validating skills like that is strictly something a mid level (no matter what your title is) “engineer” would do.

Every interview I have had since 2014 - and I have had 5 since then including three as a strategic hire (still hands on keyboard) by a new to the company director/CTO and one at BigTech and now full time staff consultant (cloud + app dev) has been behavioral where I had to convince then that I was “smart and gets things done”.

No matter what generic set of skills you have Kubernetes, AWS, etc, everyone and their dog have those skills - or at least enough people to make it difficult to stand out.

If you do have a niche set of skills that not every one has (raises hand) you aren’t randomly spamming ATS’s and going through all of the standard interviewing processes. You’re being pushed through by a decision maker.

You aren’t going to get hired in today’s market based on a generic set of skills without selling yourself as more than that. Every open rec gets hundreds of applications within a day.

I am currently in the pool as the for lack of a better term and stolen from Amazon (former employer) as a “bar raiser”. I’m not going to ask you about K8s trivia.

I’m going to ask you “what project are you most proud of” and then delve deep in the complexity of the project, your role and scope, your decision mskinv process etc.

Endorsements are good way, but limited and they can be bought and can be faked easily, which does not bring much credibility.Do you think there is a way to improve the endorsement system from LinkedIn into something that will be taken more seriously?

Yes that is the main assumption, seniors would not be interested in getting their skill verified they will be doing the verifying to a junior or whatever entry title there is, in order to help them gain their credibility even mentor them, so it would be easier for them to stand out.

That is a fair point, also that is going to be part of the verification process, engineers would be able to get badges verified for their projects, atm I can AI generate a whole project and add it to my portfolio, without having to do the system design work, but if I get into the verification process, then I would be able to prove I did it if I really did by explaining my system design, decisions, compromises so on.

You can’t overcome human nature. Why wouldn’t I verify any junior that asked me? That’s just like I’m never going to give someone a bad reference

How are you going to prove they did it? And in the age of AI, I expect developers to use AI to code.

That gets back to why a system like this is worthless, statistics are that people only spend 6 seconds on a resume before they decide to interview a candidate, they definitely aren’t going to take time to look at a resume. They are definitely not going to look at a GitHub profile or a third party site.

I have an anecdote from both sides.

When I left Amazon in 2023, I interviewed for a position as an architect for a company where I would be responsible for leading the integration of all the companies they acquired. I was interviewing with the director and he did his standard spiel about what the company does and some of the architecture.

I knew he hadn’t looked at my resume then. I was the architect for one of their most recent acquisitions and he was explaining the system I designed and that was clear on my resume. I had left the company to work for AWS in 2020 and I was referred by a former manager that was still there.

On my side, I never look at resumes, I don’t really care what technology you know. AI can write code and you can learn new technology. I want to know how you think through problems, how you learn, how you adapt and how you troubleshoot.

I go into every interview purposefully blind.

It is not about overcoming human nature like HR tries to force things like that. > Why wouldn’t I verify any junior that asked me?

This tells me that there will be people signing up to do that and earn the benefits of verifying others.

AI is the hands, the person building it has to be the brain, they will have to understand what they told the AI to do so they can create accurate requirements and of course system design, so they AI can build things with less error margin.

> How are you going to prove they did it?

A good pilot is not the one who never uses autopilot. A good pilot is the one who knows when to trust the autopilot, when to override it, and how to land the plane manually when the screens go black. You can prove someone knows their job by looking at those moments of override and decision-making, not by staring at the autopilot screen

> On my side, I never look at resumes, I don’t really care what technology you know. AI can write code and you can learn new technology. I want to know how you think through problems, how you learn, how you adapt and how you troubleshoot.

This is how it should be, and I never said that I don't agree with this, the purpose of the skill verification process is to verify these things as well, as I call it for now "personality section you can put on your CV2.0", I am sure someone experienced as you can be a verifier and can easily tell how some junior, mid or senior whatever is going to be a person you wanna work with and have the qualities of critical thinking, proactive problem solving that bring value

> engineers who've worked with them (or reviewed their OSS contributions) could validate or challenge that [skill]

I like your idea of verifiable skills but I don’t think the proposal is solid. We need a trusted third-party as a validator so everyone is happy. Otherwise, we bring bias and corruption into this.

The problem is currently solved by professional certification — yes it could be gamed but it provides an official confirmation from a reputable source rather than someone’s word that you’re good.

Please keep iterating! We need better tools!

Thank you for the feedback, I appriciate it. The idea is to avoid the corruption and bias by hand picking a starter deck of super trusted few professionals who would be able to verify others.

For example to prove someone is an senior engineer from google I can ask for a pay slip and references and yes certifications, so we would become that rusted partner. Still working on the proposal and iterations, I wanna talk to much engineers, hiring manages as possible since this is going to be for everyone, to navigate this wild job market.

We definitely need better tools, at the moment it is wild that it takes 1000 of applications to just get an answer from any company, this not right at all

The model gets interesting if reviewer credibility is scored by calibration over time, not follower count. Without that, it will drift toward reciprocal endorsements and people gaming each other’s profiles.

The idea is great, but everything else is not. A few points:

- You say "engineering" verification, I checked and it's all computer science, monkey coding, full stack whatever, not only is that NOT engineering, it wasn't before generative AI, and it's definitely not now. This is programming. Programming is writing, and with AI it's more of storytelling rather than engineering.

- How would this platform measure real engineering skills? Critical thinking, problem solving, etc.? What about Electrical Engineering? Civil Engineering? Robotics design? Etc.

- I didn't open an account, but on the first page it asks for LinkedIn.. really? I don't trust most of what I read there. GitHub? Not only does it bring us back to software-related topics, but what if I don't use GitHub?

I think the best way to verify someone's skills in the engineering world is a portfolio that shows their projects and what was accomplished during them. A resume will never be enough, and interviews should be limited to personal interactions and assessing how the potential candidate communicates. Where needed, maybe an assignment to complete and return after a few days -one that mimics the potential project or role they will be working on- because asking questions during an interview is not enough to measure their skills at all.

Thank you for the feedback. It is meant to be all kinds of engineering, the vision is to have all type of engineers from software engineers(which you are correct it is focused full on that).

> How would this platform measure real engineering skills? Critical thinking, problem solving, etc.? What about Electrical Engineering? Civil Engineering? Robotics design? Etc

- This will be all up to the hand picked person to measure your skills, for example the verifier will be assessing everything required to determine if someone is really good or not, that includes critical thinking, problem solving so on. Let's say this verifier they work/ed for Google, they will bring Google's interview framework to this candidate software engineer in our case to asses their skills, the expected result would be "I passed the assessment, I could be good as an engineer at Google" which helps with credibility. The platform itself will store the final decision on your profile (you can hide or show anything up to the user).

> I didn't open an account, but on the first page it asks for LinkedIn.. really? I don't trust most of what I read there. GitHub? Not only does it bring us back to software-related topics, but what if I don't use GitHub?

- Linkedin, github or gitlab are all optional fields, I have pushed a fix on that to communicate it better, thank you. I am thinking of making it a free link field to be portfolio rather than profile from github, gitlab or whatever else. Personally I understand this, not a fan of github at all, I just put it there since it is the most used.

This is how I imagine it and actually tested this concept and it worked and you are 100% right it brings a lot of results. What I did was fist meet and greet talk, then assign take at home project, after the project was done, then we discussed the decisions made and approaches and how the problem was solved, and talked about past projects from the portfolio. This reduced the chance significantly of a bad hire, and the candidates liked the concept a lot.

What don't you like the most from the current hiring processes? For example in software engineering for me is solving meaningless problems during interviews like reverse a binary tree that have nothing to do with the actual role.