AWS is not just GPU + electricity. It's GPU + a server + a rack + electricity + network + a physical data center + real estate + labor + reserve capacity + automation, etc...
For cost effectiveness, one route (mostly in the ML community) is to rent people's machines via services like vast.ai, who can get around NVIDIA preventing use of consumer cards in data centers by making it so they're only selling/buying time on people's computers.
These end up being a lot more affordable (IIRC ~$20/day for a 2x 3090 system) but in exchange have the unreliability that comes with renting time on someone else's computer, like tasks randomly getting killed (which is pretty frustrating to wake up to, as you get charged for time rather than usage), some people having poor upload/download speeds or other gpu performance issues.
Update — I do recall correctly. Source: https://www.digitaltrends.com/computing/nvidia-bans-consumer...
I don't think this was ever the case, at least, not in the short term or at smaller scales.
I always thought the value proposition for AWS was the fact that you could scale up and down nearly instantly and that you didn't have to run your own data center, which was a monster of an expense.
AWS made it so a single person could be a startup. With just a couple clicks, you've got a server running, and you only pay for the time you use.
Now, maybe you're asking about GPUs specifically...in that case, yeah, I don't know why people would use AWS unless they really only need a couple hours of GPU time per month or something. GPU instances are so expensive that the ROI for buying them yourself hits pretty quickly. You have to REALLY need the rest of the AWS infrastructure to be directly connected to your GPU to make it worthwhile.