how does Klarity scale with more complex models or larger datasets? Does it maintain the same level of insight and actionable suggestions as the model grows in size and complexity?
Great release btw
It should work with any type of model, obviously longer chain of thoughts will be more difficult to analyse by the evaluation model, because it will have way more reasoning steps to identify and separate. The quality of the outcome depends a lot on the chosen model to give you insights. We tested with Llama3-70B and worked smoothly most of the times.