The goal of the interview is not to subject someone to a test, it's to become informed about them and their abilities to do the job.
The industry has this weird defensive position where everyone believes there's an army of imposters trying to invade them, and it's become a complete parody of itself.
Sadly I've even seen this from a company founder who felt that as the company grew they were no longer the "big kahuna" and felt a need to make up for it by showing off in an interview.
(experiences as a co-interviewer not candidate)
The most fun organizations are where everyone is glad to be working with all those other people and things everyone else is smarter than them (so with a lot of opportunities to learn). Organizations like that really do exist -- I've worked in a bunch, but the larger the organization gets the harder that is to maintain.
But when a founder is doing this it poisons the well right away.
The value of a sunk cost is zero. Who cares what you invested into the process so far? It's time to make the best judgment you can for the benefit of your organization. If you learn that the previous investment was misspent, great. Refine the process for next time.
You should also be providing a positive candidate experience if possible. They may be a customer, have friends you want to hire, or be in an anti-loop and actually good (but you can't tell, and that's okay, because, well, it happens).
You're there to (in order): 1) make the best judgment for the organization 2) represent the company well for the candidate (hopefully this is a faithful representation, since you can't work long term for the company if you're tricking the candidate)
It sounds like you do this already, since your interviews are challenging and friendly. Hopefully they're discriminating at and below job level to allow you to inform a hiring decision. Friendly almost always makes sense (excepting for threatening, dangerous, illegal, or otherwise red flag interviews).
Some houses cut bad interviews short at a halfway point. We don't. The candidate might show you something later in the day that points to another position or area of strength, and, either way, they should feel like they got a fair shake. It also helps with cross-calibration. I've had two candidates that I would have walked out at other companies (less than 1%) at the halfway point. 2-3 incremental person-hours is almost always worth it, though. I'm glad we put in the effort.
Generally the focus of someone hiring should be on the position they are hiring for, and they should actively ask themselves how the candidate with their own specific character traits and skills would cope in that position.
For this the person conducting the interview needs a profound understanding of the open position, which requires work that is often not done — and this results in lazy trick questions and ultimately gets the wrong people hired. Just like in a scientific measurement the person "measuring" the candidate needs to be aware of their own influence. And while taking of half of the stack of applications, throwing them into the bin and pronouncing that you don't like unlucky people certainly reduces your workload, it also reduces your chances of getting the best candidate.
There's pros and cons. For on-site, there's a point where you just don't want to waste time.
When I run an interview, if it's clear early on that a candidate is a no, I'll just move the interview along quickly so I don't hurt feelings.