I too once fell into the trap of having an LLM generate a confidence value in a response. This is a very genuine concern to raise.