Hacker News Reader

I think you are referring to the 'ask follow-up questions' toggle in the settings but it's not available on all devices to turn it off.

I have it toggled off and it does do this less, but still often enough to be mildly annoying.