Usage data and analysis from my AI chatbot

Hello all,

LLM hype is everywhere these days, but actual usage data is pretty hard to find. I see tons of companies pushing AI tools at me but I have almost no idea what user behaviour/preferences are.

I built a TaxGPT.ca, an AI chatbot for answering people's (Canadian) tax questions and ran a bunch of numbers over the questions submitted between March 4 to April 31, 2024.

Highlights include:

- Over 8,500 questions answered - Users frequently asked about basic tax info and business expenses - Feedback is not very common but surprisingly positive - Experiments in tagging questions in relevant and for 'complexity'

My verdict here is that these bots have potential but you have to have really quick feedback loops and pretty good product chops — you can't just throw AI at some problem and assume users will find their own ways to make it useful.

'Vouched' as this was marked as 'dead'.

Unless obvious, it would be nice if when flagging something to saying why. We can only guess motive as they left no comment.

I have no affiliation with this story. But also upvoted as it was quite detailed and valuable real-world case study.