Ask it to output the occurrences as items in a JSON array, then count the array items in code

Man, -people- are bad at counting tasks like this.

Not only that, asking an LLM a question like: "List all the occurences where John Doe fights someone in the book. Also list who he fights and why" is a surefire way of getting LLMs to hallucinate, and also give incomplete info.

I had issues with gpt4 last summer confusing which characters did what and hallucinating characters.

his may of been a context/chunking issue (if that particular section doesn't name the character performing an action), but maybe its better now.