Your job isn't to build the smoothest, most optimized, experience for your customers. It's to build features customers want or need, which in turn create value for your employer. This generated value pays your paycheck. Of course if you do a bad enough job, it's counterproductive, but to the extent it's not bad enough your job is to focus on features and development and not on optimization. Once performance is bad enough, your job becomes optimization -- via instrumentation, measurement and profiling.
Every minute you spend waiting on a build or waiting for an app to launch or waiting for some long-running task to finish is a minute you're not creating value. Compounding this problem, it impacts morale if you spend your day looking at spinning beach balls while your boss won't get off your back about when you'll be done.
This is your employer's perspective, more than likely.
[edit] Another way of looking at this is to say that you and your customers are doing different jobs. You have all the extra cpu and memory overhead of developing a product — all they have to do is run it. By definition your workload is more demanding.