Let's not forget Qwen 35B A3B MoE. It gets better performance than this in all the metrics for a fraction of the memory / compute footprint.
Sad to see all the non Chinese open source models being at least one generation behind.
Sad to see all the non Chinese open source models being at least one generation behind.