> I won’t say LLMs are useless for this. But from my trials, I get about 1 card that’s useful to me out of 10, and even that 1 card still needs rewriting.
I don't know the specifics of how the author tried to do so, but from what I've seen the majority of attempts are, let me drop a chapter of a textbook and say "make flashcards." If that is what we are talking about, then yes, LLMs are useless.
In my mind, though, this is sort of like looking at the very first GitHub Copilot LLM autocomplete from a couple of years ago and concluding, yeah it's nice for one-liners, but it cannot write an app.
If you create a framework around your card-creation AI so that it can use tools, and verify its work to ensure common card-creation pitfalls don't happen, you can get pretty high-quality cards. In my experience, you go from a 10~20% hit rate to a ~90% hit rate, which in my mind is good enough. I got to ~75% quality just from a two extra LLM calls that would assess a potential card against a standard set of rules (adapted from [0]). There are huge Pareto gains to be had here.
I've generated thousdands of cards over the last few months this way. I let the AI add it directly to Anki via AnkiConnect. Then, if when I go to review I find a card that my AI created and I don't like it, I just delete it.
Removing the limitation of card creation is really quite compelling, and I think the area is still highly under-invested in. Would be cool to see a generic framework evolve that one could use. For now, I've been using a personal fork of clanki [1].
[0] https://supermemo.guru/wiki/20_rules_of_knowledge_formulatio...