When did I say anything like “it will free me up from so much labor!”? This is not even remotely close to what I believe. You are responding to a caricature of me you have in your head.
Again, you are excited about the future of a technology that is rapidly making you obsolete. It is not going to go how you think it is going to go.
I think you basically have to accept there will be one of two options -- either it scales at something like O(n), in which case you have some number of highly skilled professional which can use AI to do the work of 100x of a SWE today (that would be a pretty cool job). Or it scales with something like O(1). I think in the second world the only logical conclusion is that AI is so incredibly capable that the world is profoundly and utterly transformed. If AI can do the work of any SWE without oversight, it can basically do anything -- which would mean that any need you have could be satisfied. If you are hungry, food could be grown. If you are thirsty, water could be procured. Or you would need to propose some crazy future where AGI is achieved but for some reason trivial things like this can't be solved.
I think the thing that makes me the most nervous is some enemy actor going rogue with AI and killing us all, but somehow we've lived for decades with enemy actors having access to nuclear and bioweapons and somehow we are all still here.
I don't know how to make it any clearer that you are the horse in this analogy. Hoping that you're one of the 5% of horses that gets kept around for racing, dressage, or farm labor when the other 95% of carriage horses are put out to pasture does not make this future any less bleak if you're a horse. If you don’t understand that decimating demand for your profession (and knowledge workers in general) overnight will do the same to wages, you are in for a rude awakening.
That you think AI doing the work of any SWE means that "any need you have could be satisfied" is fantasy. Not only are most of the things people want not the result of software engineering, but the notion that the billionaire owners of AIs intend to use them to fully provide for the wants and needs of the masses of humans who have suddenly been rendered economically useless is absolutely wild.
For perhaps a more salient example, take a look at what happened to laborers during the industrial revolution. Wages collapsed by 60–80% over four decades. It took something over eighty years (and that's considered a lowball estimate) for things to recover. By all indications AI will be even more disruptive, particularly since it is disrupting high-income professions. Notably, artisans whose jobs evaporated were not suddenly awash in a sea of abundance, as you seem to anticipate.
It can't be both ways. It will either be bad enough that most of us will still have jobs, or good enough that most of our needs will be solved.
AI will certainly be used to enrich the lives of a select few. It’s great to be in that group, assuming you don’t give a shit about the people who aren’t.
Self-driving cars are going to supplant human drivers. What do you think this is going to do to Lyft and Uber drivers, for whom this is already a job of last resort?
Do you think the ability to be driven around by robots is a net improvement for their lives? Do you think they will, now unemployed, be given these vehicles for free?
I’ll be honest, your comments strike me as you being completely ignorant of the existence of poverty. It quite literally took generations for the labor class to climb out of poverty brought on by the Industrial Revolution. Do you not understand that free gifts from the proceeds of new technologies are not simply handed out to those whom it displaced? The United States is one of the richest countries on earth, and an absolutely staggering percentage of its people live in poverty despite this enormous abundance. How do you square the existence of its poor? How do you have any expectation that you, yourself will not become one of them when 90%+ of people in your career are displaced, cratering wages?
Are you completely unaware of the accelerating rate at which wealth is being acquired by and hoarded by those who already have the most of it? Can you not recognize what the eventual conclusion of that process is, if not only left unchecked but allowed to accelerate?
Honestly your argument sounds detached from reality.