One thing I'd like to try is MoE offloading: I have 2x32 GiB of VRAM and 128 GiB of DDR5 running at 4800 MT/s (only 2 channels though). I've seen people post difficult to believe MoE offloading results albeit a decently long time ago with older models. Maybe there is a quant that would fit with MoE offloading?

That said, I am guessing my problem is not enough RAM - but this poor consumer platform struggles to do memory training with 128 GiB as it is.

Now I surely regret not having gotten Threadripper and 256 GiB of RAM in the before-times.