If it's just a toy, sure. But even frontier models reach a tipping point of complexity fairly early where they start introducing bugs quicker than they fix them.
A (good) framework abstracts away the complexity so you hit that tipping point much later in terms of codebase size, and reduces your token spend to boot.