Sure, I think their quant format is pretty basic, something similar to bnb q4 - my plan will to be scripting a framework for testing, so should do that as well since the omniquant implementation is in mlc-llm anyways.

i was trying to get this to work with mlc-llm. i'd appreciate any pointers

more specifically on a non-cuda gpu - mali on orangepi via opencl