You cause me to have an additional thought on the topic which is that as much as I expressed a sense of dread at the inevitable use of this sort of tech in hiring pipelines (not by agents, necessarily, but as a sort of HUD overlay on a video call between humans was my initial envisioned use case.) But I suppose that just as the AI interviewer bots that I thus far have refused to engage with will inevitably be unavoidable if one is on the job hunt, so will the use of this sort of multi-modal sentiment analysis be inevitable. (Same with the justice system use case you referenced in your metaphor, and probably therapists and such as well will follow.)
As such, I wish you the best of luck with this project - earnestly so - because if, as I suggest, it is inevitable... we want such a system to be as good as possible.
An aside: another inevitable use case just came to mind - that of the cheap, shoddily implemented and poorly tested (along with the insecure, surveillance-adjacent products that will proliferate) kid's toys with embedded AI and the sardonically-humorous privacy mishaps and unintended actions from such low-quality implementation toys being sold (see: the current LLM-enabled kids toys currently popping up routinely at retailers.) ha! Sorry I keep taking your cool demo to dystopian extremes. :)
Oh, one more thing... Upon re-reading my previous comment, I recognize that the description of my visceral reaction as on of being being "repulsed by the thought" could literally be read as me calling your system "repulsive", which was not my intent. I think your tech is cool, and was just trying to convey two conflicting feelings that occurred within me when thinking about the future commercial use cases. I hope your systems works great so that if it does find market fit with such use cases, that, well... if it's inevitable - as the last few years of "LLMs everywhere!" has forced us all to adapt (accept or reject it, it still requires new effort) - we should hope for a good and working system, so I hope you succeed in making one.
Lastly, to your self-driving/potholes analogy... I do think that that fits more in line with my "objective CV classification" category; I think a closer fit to what you're building would be "self-driving car having to handle the Trolley Car Problem", with the nuances of human value judgements etc; does the car swerve into two adults vs one child? And so on. Pothole classification is more objective while driving into it, swerving to avoid it, classifying pedestrians and choosing one to possibly collide with, etc are subjective and more complicated (as is your system and the functions it can perform.)
Best of luck!