Hacker News Reader

Companies usually hire editors to manually annotate the quality of the algorithms on anonymized samples. They don't need direct user feedback to gather quality metrics. You also can't tell which step failed in the pipeline: speech recognition, query understanding, or the actual search.

Why can't you know which step? It's their system..they can build as much tracing as they want into it.

Why would editors make user feedback any less valuable? It's hubris to think it's not.

I'm just waiting for my toddler to start repeating "Hey Siri, nobody was talking to you."

My preschooler does that whenever I’m working with a tool that requires me to bend my wrist backwards and the watch-Siri activates.

“Sorry, could you say that again?”

Nobody asked for you to interrupt my chopping, Siri.

They likely don't need the manual feedback as they would be able to tell that someone asked the same question twice in a row, and then look at the data to see what the issue was.