I’ve been sidetracked with work but planning on tuning llama 65B to produce alpaca 65B, will distribute via huggingface or torrent..

FWIW running the 30B alpaca-lora model quantized to 4-bit via llama.cpp has given me great results, and while I don’t expect much of an improvement from 65B at FP16, 65B will probably perform better than 30B when quantized

The interesting next steps in my head are more focused around curating a better instruction-tuning dataset using GPT-4, then fine-tuning again, and integrating the LangChain project with the resulting agent