So, why does your marketing material compare api.ai trained on 70 examples to Snips trained on 2000 examples?
We are comparing what you get by using an alternative service like api.ai (which you need to start by feeding manually) with what you can do with our solution.
Manually feeding 70 queries per intent already takes a good deal of energy and imagination, so we thought it was representative of a good effort from a user. This is why Snips proposes to its users to automatically to use generative methods to create high-quality datasets (over 2000 queries per intent).
All hypotheses are clarified in our blog post on performances which I encourage you to read! https://medium.com/@alicecoucke/benchmarking-natural-languag...
We even post all the underlying data and results on GitHub so people will be able to verify our claims: https://github.com/snipsco/nlu-benchmark/tree/master/2017-06...
Only if you have to manually make them up. If you have access to the query stream (which I imagine api.ai etc give you since they're online platforms), it's quite quick and easy.
I can definitely appreciate integrating things together to make things easy, but this is a pretty biased comparison IMO.
It's great that you've made the data available, but the headline metric that most people will see seems misleading.
Then you show the F1 score for snips on 2000 queries. Obviously you see improvements and that's good. What I don't understand is why don't you show the F1 score for the other competitors on 2000 queries? If you're able to generate 2000 input from the original 70, what prevents you from feeding those to the other NLP engines? Do they all get a 93% F1 score ?