Tabby's philosophy is to achieve a completion rate comparable to Codex/Copilot by using a model size of less than 1B, with support for BF16/FP16 that reduces VRAM requirements to 2G or less. This may seem impossible, given that the model size is 10 times smaller than that of Codex, but it is definitely achievable, especially in an on-premises environment where customers want to keep the code behind a firewall.

Related research works include [1]. (Hints: combine code search and LLM).

[1]: RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation https://arxiv.org/abs/2303.12570

This also reveals Tabby's roadmap beyond other OSS work like Fauxipilot :)