HNHacker News
TopNewBestAskShowJobs

kziojzwsndppqgg

166 karma · joined April 8, 2015

submissionscomments
kziojzwsndppqgg··on Apple's Software Quality Crisis
(Digging out old alt account for reasons)

I'm a former employee from the SWE org of ~13 years, left around a year ago.

This is a huge problem that the company needs to address ASAP. If you're in the Apple SWE org and reading this, please go up the chain as far as you're able to make things like this understood: Apple needs a bugfix release. They need all hands on deck going through radars and fixing things. No new features until these things get settled.

Care about error logs. Look at the number of Error logs happening per second on a customer build. Every one of those was considered by someone to be important enough to notice that it likely needs fixing. There are showstopping bugs buried in there.

Figure out your concurrency bugs. The whole company seems to be using the swift Concurrency framework wrong (at least a year ago, I doubt things have changed since then.) Stop abusing semaphores and DispatchGroups to work around async/sync barriers. It makes the compiler shut up but causes deadlocks later. Every "Just a sec" on Siri on HomePod is very likely caused by this. Stop putting off the important refactors needed to make this work.

Start caring about compiler warnings. Get the swift team to allow warnings to be disabled on a line-by-line basis so you can work from a zero-warning baseline and attack it from there.

Fix the build system. It's horrific that coordinated changes to multiple frameworks are so god damned impossible to do, and result in broken builds so often. You probably don't need to go full monorepo, but if you're going to continue with thousands of individual projects, make it so coordinated submissions is possible.

Fix Xcode. Or at least just jettison it. Pay a boatload of money to JetBrains or something and get an IDE that works internally, bless it as the way to go, and announce publically that Xcode is deprecated. You don't have the resources to fix it any more, it's time to take it out to pasture.

Fix the development milestones. The current system is designed for tentpole features to reach a certain maturity level before a certain date before a punt decision: But it just encourages punting (slipping to the B or C or E release) to a milestone with less scrutiny. Allow for bolder changes later in the process if they're in the name of improving stability. Allow for groups which are not part of a tentpole feature, to fix things at any time without having to deal with bug deadlines. Zero bugs is a joke, it's just denial, encoded into process.

The milestone system also lacks the mere vocabulary necessary to describe "time dedicated to fixing bugs in shipping code". Like it doesn't even exist as a concept. "Escape" makes it seem like it's a rarity, it's not. Nobody seems to follow the pact any more. Probably because the pact pretends that escapes are rare. They're everywhere. The development process needs dedicated time spent not doing feature work so that old bugs can be addressed.

There are a lot more things I could go on about, but people already know this. The problem is that senior leadership doesn't care enough. They don't foster a culture of excellence where they actually sweat these small details. They only care about features, and it's a disease. Get somebody up there who gives a shit.

kziojzwsndppqgg··on The Google incentive mismatch: Problems with promotion-oriented cultures
Can confirm, it’s a great choice. Step one, convince a FAANG/MAGMA to let you work remote at the same salary level you already have. Step two, move to an extremely cheap cost-of-living area. Step three, avoid promotion at all costs, and just fix bugs.

Provide value, but _never_ get promoted. Every year come review season, talk about how your goals for the next year are to improve latency/perf/whatever. It doesn’t have to be specific, just something that’s important enough to be useful, but don’t be ambitious. Just keep your head down and make enough improvements that people consider you to be a good “grunt” worker.

Yeah yeah, inflation sucks and technically I’m making less because my RSU’s have mostly vested and my yearly grants haven’t made up for it, but I don’t care. I’ve saved up enough that I could retire now if I really wanted to.

Every day for me, I work 9-5, fix a few bugs, provide some value, then work’s over and I get to play with my son and completely forget about my job. Then once he’s in bed I smoke some weed, watch shows with the wife, talk about the good old days. It’s heaven.

(By the way, I say all this, but I kinda failed at it because I got promoted last year anyway. I got hella worried because I thought they’d start expecting more out of me, so I actually slowed down my pace even more. I just got a mid-cycle bonus last month. Maybe I’m not so great at keeping my head down after all…)

kziojzwsndppqgg··on Apple confirms it uses Google cloud for some of iCloud
Nothing in public, no. Apple doesn’t talk about this stuff generally. But I know because I helped build a lot of it.
kziojzwsndppqgg··on Apple confirms it uses Google cloud for some of iCloud
Apple runs absolutely ludicrous amounts of its own storage. The idea that they use GCP for 100% of anything is insane.

Saying they “use GCP for iCloud” sounds intentionally misleading... there’s a missing qualifier there. But I guess “uses GCP for some stuff in iCloud” isn’t nearly as click-worthy I guess.

kziojzwsndppqgg··on I Quit: What really goes on at Apple
> Apple used to be a lot less awful in this respect, but their design and usability was still great (arguably better than today).

It's interesting that you say that because from everything I've heard talking to people I work with and in other departments who have been here a while, the opposite is true.

The conventional thought is at least over the past 2 years or so, the stress has gotten lower. Most of this is from the departure of a particular executive (no, not Steve) who was known to crack the whip really hard on people. I also hear those same people talk about how quality has suffered as a result of his departure, and they actually long for the higher-stress days when they had to work longer hours but were happier with the finished product.

kziojzwsndppqgg··on I Quit: What really goes on at Apple
I've responded elsewhere in the comments about my positive experiences (just ⌘+F) but I think the main reason people aren't typically in "public" about their role at Apple is because Apple has a secretive culture. You're just expected to not talk Apple's business, including what you work on, to anyone outside of Apple (or even people inside Apple if they're not disclosed on your project.)

Trust me, complaints about this come up a lot inside Apple. I'd love to go to conferences and talk about the tech we're working on. (Even if just to get more hires!) I'd love to blog about it. I'd love to contribute to open source projects. But it's just not what we do here, in general. (Hell, I had to make a throwaway just to post this on HN.) It's definitely my biggest complaint about my job, and it's definitely kept us from being able to hire good talent.

kziojzwsndppqgg··on I Quit: What really goes on at Apple
I'm not really sure that article even paints Apple in a bad light. Yeah, it talks about how much you work hard, but even just listen to them:

> I mean, it’s not that it’s not fun, it’s not that it’s not fulfilling, it’s not that you don’t get to work around all these brilliant people. The bad side effect is they’re all, like, workaholic, psychotic brilliant people.

For me, one of the best things in life is the opportunity to work hard at work worth doing. Apple gives me that, and I feel extremely fortunate.

I've been at jobs where the stress is very low and I had time to browse the internet at all hours and didn't have much expected of me. This is 100% the opposite of that, and I'm happier now than I ever was then. It's been 4 years and I've never had anything close to burn-out, because I've never felt that the work wasn't "worth it". (To me, burn out is a result of feeling like you're not getting anywhere, and that just plain isn't the case at a company where we consistently ship things with huge scopes, on time and to resounding success.)

Yeah, there's exec demos, and yeah there's dry-run meetings for them, but you know what? The execs here are fucking smart. Every meeting I've had with execs has left me awestruck at the level with which someone can be simultaneously so big-picture oriented while still being able to sweat little details. There's a reason they got to where they are, and I really feel that at Apple it's a true meritocracy.

It sounds like it's even harder work if you're in the direct email line-of-sight of an exec (which I am not), and I feel for the authors of your linked post about how hard that must be, but to me that comes with the territory of being so close to the success or failure of a product that you are under that kind of scrutiny. What else would you expect?

kziojzwsndppqgg··on I Quit: What really goes on at Apple
FWIW, as an Apple employee I have no idea what the hell he's talking about.

I've worked here 4 years and I absolutely love. my. job. I take days off to work from home whenever I feel like it. We drink at the office on occasion. I'm never harassed for not being constantly online. I don't have endless meetings. I'm constantly praised for the work I do, I get great reviews, with large bonuses.

My impression from reading this article is that he either had a shitty manager (it can happen, Apple's a huge company) or his department wasn't very well-run. (It is customer service, that's never known for being a great environment almost anywhere you work.)

I feel bad for the guy, in a situation he described I would've left too. Fortunately I'm not in that situation, and neither is anyone else I know here. People have their issues with small things but at the end of the day I think everyone I work with loves what they do.