rajhans

We have published some insights here. https://medium.com/snowflake/snowflake-arctic-cookbook-serie...

Arctic dev here. Yes keeping all experts in memory is the recommendation here and understandably that is a barrier to some. But once you have 1 H100 node or two (gpu middle-class I guess...?), then a few things to note: 1. FP6/FP8 inference is pretty good. How to on a single node: https://github.com/Snowflake-Labs/snowflake-arctic/tree/main... (vllm support coming soon!) 2. Small number of activated parameters shine in batch inference case for cloud providers.

One of the modelers working on Arctic. We have done no alignment training whatsoever.

On day one, Spoke's ML and NLP algorithm can respond with the right answer that is relatively directly related (i.e. has a good amount of keyword overlap with the query) without any training. For complex queries, if Spoke does not find the right answer, you can "train" Spoke in a very simple way and very quickly (as a part of your normal ticketing/knowledge management flow without any extra effort) by responding with the right answer. And to answer your question, Spoke's algorithm learns from these responses in real-time and Spoke's performance for your organization will get better over time as more responses are accumulated.

For full transparency, Spoke will not deliver 100% accuracy for complex questions because "open domain" (unlike closed domain stuff like pizza ordering that Alexa or Google Home handle) natural language understanding is still far from being solved. However, it will augment your ticketing/knowledge management flow nicely by automatically resolving an increasingly large chunk of your questions.

Great work! A few questions for the author(s): In the article, you have listed 9 feature extractors/templates. In the final model, what's the total number (or rough magnitude) of features? How much data (or ballpark estimate) did you train this on? Did you try to deal with potential difference in distribution across your data sampling sources?