BS=1 also means you can only decode one token. This is not always what you want, and then you're back to matrix-matrix multiply again. Most of the from-scratch llama implementations start with matrix-vector because it's simpler & easier. But eventually, you will want to serve multiple users, or compute multiple beams together...