Hacker News Reader

it seems that BERT can be run on the llama.cpp platform https://github.com/ggerganov/llama.cpp/pull/5423

so presumably those models could benefit from the speed ups described in OP article when running on CPU

llama.cpp only support BERT architectures for embedding but not with classification heads - although there is a feature request to add that

ah I see, thanks, did not read the PR closely!