Leaner LLMs could enable efficient local use on phones and laptops

Compressing Large Language Models using Low Rank and Low Precision Decomposition

https://arxiv.org/abs/2405.18886