I just tried it with phi3.5:3.8b-mini-instruct-fp16 - it didn't work with the base question, though interestingly the reasoning decided that strawberry was spelt s-t-r-a-w-b-e-r - which explains why the AIs have such a hard time with this question. I also tried it with my current favourite programming question too - What programming language is this whole line of code using? `def obfuscated_fibonacci(x)` - and like all the AIs, it was convinced the answer was python (the correct answer is ruby - python needs a trailing colon - but most LLMs will swear blind that it's python). It didn't even consider ruby as a possibility. Nobody uses ruby anyway :D

Thanks for the fork and the suggestions though - looks like I'll be having fun with this over the week!