Yea definitely. But in that case, you could train _much_ faster in pytorch, then convert to ONNX, and load in the browser for inference (as the transformers.js docs recommend)

EDIT: (I responded before your full edit with the bullet list). This next comment is orthogonal to the slow training performance topic I think, but the use cases you reference there don't seem to be well-suited at all for an autoregressive decoder-only model architecture.