Ask HN: Making LLMs ask a clarifying question

A major problem that we are facing is that the model (GPT-4) is not aware that there are multiple answers present for the query in the context. It tries to pick one possible answer and expands on that. Ideally it should be able to detect the ambiguity and ask a clarifying question.

I tried to make this as an "instruction" in the prompt but that doesn't seem to nudge the model towards identifying ambiguity. I am not trying to improve the retrieval to make the context less ambiguous. Anyone has found a workarond?