I know that this is Transformer, and not LLM per se. But isn't this the same idea that PaulG said the other day, and many of the comments criticized him?
What I intended in my comment was that, may be starting from scratch (like you did many months before, and what PaulG suggested recently) is the way to go for future job prospects and startups. Most of the comments in the thread I posted was negative for PaulG saying that.