You will rarely, if ever, deploy an application running CDK, so the bundle size is more or less irrelevant. It’s basically a glorified YAML compiler used to bundle infrastructure.
Not only that, but you'll also not really be running it anywhere that isn't "on AWS", like a browser.
The cloudformation job will more than likely dwarf the tsc overhead from cdk stuff being a mono package.