Former Zynga Engineer doing AMA on reddit
reddit.com
reddit.com
Yes, gaming companies are extremely data-driven and they measure everything they can. I'm in charge of dashboards that have over 300 KPI's (key performance indicators) on them. What do we measure? A whole lot of shit. ARPU, ARPPU, HC (hard currency) spend/gained, SC (soft currency) gained/spend, Retention, DAU, Spend DAU, Gained DAU, Downloads, Activations, plus funnels for every page, every user action you can think of.
Do I think measuring all this is creepy? No. Why? Because when I come in the morning, I look at my stats. I'll be able to tell within an hour if the build the engineers shipped the night before had a bug that affected 30% of our users in a certain location. I'll then report this to the Engineering team, they can create a hotfix, and users can be happy again. Engineers are happy knowing that we have actual data to validate and confirm their gameplay suggestions (made during scrum/sprint planning sessions). I also report monetization metrics to the VP's. They want to know if the boatloads of money they are spending on user acquisition are actually paying off. And the PM's want to know if users are playing the game, or how far they get into the game.
Yes, we maximize for users who spend over $10k. These users even have names. They're called "whales" in industry parlance. Why wouldn't you want more of them in your game? This is a business afterall. I feel just as bad as the next person wondering whether some of these "whales" are addicts. I don't want to be creating products for addicts. But what if these users are just rich people who have tons of money to spend off the cuff? There's no way we can know because we never get that data. All I see is what's reported in ItunesConnect.
Do we ever record any personally identifiable information? No. Do we ever sell or distribute this data to outside parties? No. Can any of this data ever be used maliciously? No.
Personally, I like the way game companies work (hence why I work in the industry). It teaches you to measure everything. You make decisions quickly. And no one can argue with the facts. You'll know real fast whether the product is shitty and you can do an about-face, gather the troops, and push out something new.
I thought that term sounded familiar.
http://www.worldfreepoker.com/poker-glossary.html#W
"Whale: A poor player with a lot of money to lose."
I don't want to be creating products for addicts. But what if these users are just rich people who have tons of money to spend off the cuff?
Sure, just like at casinos.
You're right, in a way; there are people with more money than brains who are happy to blow a good chunk of it gambling and are really no worse off for it because they're rich. But, realistically, you and I both know that's never the whole story.
You and I also know that this isn't a problem inherent to just casual mobile games. It's a chronic problem to the gaming industry as a whole, and its not going to stop people from making entertainment for the millions of users who aren't addicts.
He is pointing out that, like casinos, these games are specifically targeting these addicts, and these games probably couldn't even exist without the addicts to support them.
>its not going to stop people from making entertainment for the millions of users who aren't addicts.
These games aren't made for the millions of users who aren't addicts. These games are made specifically for addicts.
>Yes, we maximize for users who spend over $10k
It's like the difference between selling alcohol and selling cigarettes. Alcohol is mostly sold to normal people, and percentage of the population abuses it, but virtually every single cigarette made is sold to an addict.
Alcohol manufacturers could exist without addicts, cigarette companies couldn't.
If you make a Farmville type game there's a good chance you are depending on "addicts" to keep you in business.
I'm not declaring casual games to be immoral by the way. I'm just pointing out the difference.
Targeting is your word, not mine. We cohort users by spending activity and learn from them in the same way a Macy's buyer will learn from their most loyal customers. My friend works as a buyer for children's clothes and they make the same business decisions I make about inventory and store layout.
We maximize for users who spend large amounts of money (incidentally none of ours have spent $10k, more like $100) because this and engagement data is the only data we have in determining customer loyalty. So we ask ourselves questions like "If this user found enough value out of this feature to spend $100 on it, we should make more like this".
However, I do think the closest moral analog to running a casual Farmville type game is running a casino. If you're fine with running a casino, I'm not trying to stop you.
If you really want to find out if targeting these users to maximise profits is harmful or not, I think it's a good idea to ask these spenders, perhaps have them take a survey. If you don't want that, I'm pretty sure there's research out there on gaming and spending habits that can shed some light on the question.
[Edit: I see that you sell through iTunes connect, and I don't know if Apple permits you to have anonymous user surveys in you app. Even then, I'd guess that you wouldn't risk your highest paying customers reconsidering their gaming habits by asking them uncomfortable questions in a survey. Given that, I would think actively looking for published research papers would be a good idea.]
Don't expect not to get judged when you work for those pushing the envelope in this direction though.
>I don't want to be creating products for addicts. But what if these users are just rich people who have tons of money to spend off the cuff?
Ahhh, ignorance is plausible deniability bliss.
I once wanted to work at a Lithium Ion battery start-up because I wanted to help create an energy source free of geo-political and climate turmoil. Then I discovered those sources were some of the most toxic to the environment both in the harvesting and the disposing of the material.
I then went on to do graduate research in explosives detection. We created machines that would help detect bombs in Iraq and landmines left in a peaceful villages. But then TSA used the machines to look at people's private parts and there was a mass privacy scare.
My point is: you can't go life worrying who might use the things you create for harmful purposes. You may be ignorant, but the alternative will drive you insane.
If you don't want to make games for addicts perhaps you should stop working for companies that make games for addicts. You're not fooling me.
Isn't this the point of any business?
> If you don't want to make games for addicts perhaps you should stop working for companies that make games for addicts.
I don't believe we make games for addicts. When I show our game to my friends and family they say things like "oh this game is cute", "this creature is funny", etc. The vast majority of our users are people who enjoy casual gaming when they're bored sitting on a plane or on the train.
That is not the point of business for me, most businesses offer something of value.
Would you feel the same way if we changed the business model from virtual currency to a fixed-cost one?
No, you get to decide what the point of the business you work for is. No one else does, you do. There's no rule that says the point of any/every businesses is to get people to part with as much money as possible.
In a way, but at my last employer, the people being socially engineered were employees of our clients who then had to jump through several hoops to justify their plans - often numerically - and get business spending on our services authorized, not putative millionaires trickling down their personal hard-earned wealth.
Did you ever stop to consider perhaps money isn't the worst fallout? The families of addicts suffer as well–and I suspect, when we look back years from now, we'll see a pattern of spouses and children ignored & damaged because companies like Zynga analyze/optimize everything in order to maximize engagement. A single person can't compete with that.
You're not just engineering people to part with their money–you're using your skills to get them to part with their time.
Sorry, but I consider what you do (& Zynga does) unethical.
I have friends who play Starcraft II a couple hours a day. I've known people who will skip work and play 8 hours a day. I've seen a 60 minute story on someone who was addicted to World of Warcraft. This is a chronic problem of the industry as a whole, but its not a reason to stop making games because the majority of users can have fun responsibly.
WoW vanilla was an MMO, WoW Burning C was basically moving towards the disney land model and away from the mmo. The overall business model has acknowledged the power of the Zynga model, and all actions so far have been moving towards Zynga's model and not away.
Imagine if film makers stopped trying to make good movies and instead focused on making movies where you had to pay more money to see the ending of a movie, or perhaps to pay to see a different ending to a movie. You'd sit down in a theater and then right at the climax of the movie you would have to pay more to continue watching.
Then making movies would be less about making a "good" movie and more about creating the best hook to get people to pony up more cash.
Limit everything except going to the gym? Maybe limit that too?
Zynga games and slot machines are an easy target for because they are time sinks that aren't common in HN crowd. But imagine some outsider looking at your day under a lens and regulating how you spend your time.
What should be the metric by which you classify an activity as being "good"? Inducing happyness? I bet if you ask Zygna customers they'd all say Farmville is their favorite game. Productivity?
Should people be allowed to spend large amounts of money to buy cars they don't really need? Electronics?
"Yes, we maximize for users who spend over $10k. These users even have names. They're called "whales" in industry parlance. Why wouldn't you want more of them in your game? This is a business afterall. I feel just as bad as the next person wondering whether some of these "whales" are addicts. I don't want to be creating products for addicts. But what if these users are just rich people who have tons of money to spend off the cuff? There's no way we can know because we never get that data. All I see is what's reported in ItunesConnect."
I think the somewhat layered nature of it is what creeped me out. It kind of reads like an internal dialogue of somebody trying to justify their own actions to themself.
The question that confronts me when I go into work everyday is how can we make this game fun for everyone, and how can we make money in a non-destructive, ethical manner?
I have an answer, provide people with tools that make their lives better. Does the service you're providing actually make people's lives better. And I don't mean does it make their lives more entertaining. There's no objective way to measure this, so it comes down to a VALUE judgement. YOUR value judgement. And it's not about making money, it's about making products, services and tools, for people. For people, like you, me and your mom, and my mom.
Also you said 'These users even have names. They're called "whales"'. No that's not their names, their names are bob, joan, mike, susan blah blah blah even if you don't have those names in the data, they exist. seriously, its easy to forget when you abstract them away, but there are actual humans at the end of each one of those clicks.
I'm obviously aware of what goes on in the industry and its similarities to gambling are something I think about a lot. But as an analyst, I feel like I have a lot of power in helping to create games that are fun and engaging, plus learning how to drive a product to be successful. I don't think of it as a spam machine or a front for taking money from the poor.
Maybe the business model needs to be changed. If its virtual currency that makes it creepy, then maybe switching to a fixed-cost product is something the industry as a whole needs to move to. We're still in the infancy here in learning how to harness mobile devices to make better products.
I came from a web company. I was a data analyst there too. But I really wanted to work in the mobile industry and figure out how to deliver insights on a completely new platform also at massive scale as well.
What this unpacks to: Let's make some money before people realise what's up.
That effectively means the only way to "win" is by becoming a whale; casual play is basically not rewarded. That's the problem I have with most online games; the greatest benefits go to those spending unhealthy amounts of time and money on them. IMO a mature gaming industry will be one that is able to reward a similar playing commitment to a weekly trip to the cinema; 2 hours play a week is still rewarding. That would make games attractive to a much larger audience IMO and be sustainable; you're not pushing players to burnout.
Just out of interest; do you track KPIs around hours of play per day? How many hours is a typical whale spending?
The answer may be no for now (that you know of). You can't guarantee that it will stay that way forever. Given Pincus' general shadiness to make a buck, I can totally see a slow quarter or two driving a decision to "update" the privacy policy to sell data to 3rd parties.
Similarly, it's not a big programmatic change to start tracking PII as well as cross-referencing to existing information. So while you may not have any ill intentions, I doubt you can speak for all of Zynga's actions over time.
I would emphasize the "over time" component since it tends not to be taken into account by most people which is a big reason why we are being bombarded by SOPA-like legislation. (Didn't win this one? No problem, we've got ACTA, etc. And should ACTA fail, I'm sure there's something else over the horizon)
So even though I don't believe in it (I deactivated FB for this reason), I'm not sure how much financial benefit you can gain from <UserID> <StrawberryFarmID> TSV files which most of this data is anyways.
FB is merely more blatant about it. It is my sincere wish that the US adopt a strong privacy policy like the UK's Data Protection Act (with the loopholes fixed).
Why aren't the engineers doing this themselves? (Honest question, not snark) Analyzing performance/results data is (should be? :) )part of all engineering.
Now to pick your brains a bit. :)
What are some good resources for engineers to pick up some advanced analysis skills (specifically to do the kind of work you do)?
Is it as simple as going back to college to do some courses in Advanced Statistics? Any particular sub branches of stats/math to focus on? Any good books you can recommend ? Thanks in advance.
PS: I work in Machine Learning,and am fairly knowledgeable in maths/stat but often have difficulty recommending the right sequence of books,courses etc, to people who want to follow a similar path - it doesn't help that I am entirely self taught - and often have to mumble vague generalities like "learn all the math you can and learn to code really well" etc. Recently I am getting questions about how someone can get good at "data science" and "behaviour analysis". Just wondering if there is a standard learning path for something like "casino science"
But more importantly, its learning how to be a detective. For example, I'm not strictly a math person. I also know Python and PHP and use it to write scripts when I need to validate a theory, often working with raw logs.
You also need business acumen. Analysts are like consultants in that they work with a variety of teams and stakeholders. Everyone wants to be sure that the decision they're making is the right one, and data can help them do it. So you need an understanding of what the core issue is, how the stakeholder wants it solved, and the engineering capability to get it done in a timely fashion.
One of my favorite books is Collective Intelligence. Its a soft-intro to machine learning, and having read that and gone through those exercises along with taking a formal class on machine learning, has helped me see the importance of statistics applied at web scale.
EDIT (to the PS): I don't think there's a standard teaching for data science. Everyone screens for it differently. If you apply for a job as a data scientist at LinkedIn for example (I did) they'll expect you to have a fairly formal CS background and will throw you questions any good software engineer should be able to solve. But they also ask you questions like "design a news feed" or "whats a good algorithm for a spam filter and how do you score it" or "how do you make an algorithm work with little data". I think "data science" is actually learning how to apply statistics at web scale problems, so my advice would be to look at things like recommenders, spam filters, classifiers, NLP, etc.
Which isn't to say I think terribly poorly of you, but it isn't entirely non-creepy.
Out of curiosity, do you offer users a feature to prevent themselves from spending money if they self-identify that they have a problem?
Oh and that is the raw cost of the models what it woudl be if you had to by that painted is anyones guess
If you're not maximizing that above all else, that is why you are hated.
Creepy stuff:
Spying on players. Getting intimate gaming data, their habits, their networks, and how to effectively monetize given X.
This never happened. Zynga doesn't do behavioral targeting at all.
Creepy/bad stuff in my view:
All decisions are short-term. Let us build X games in Y months.
Bad code quality. There is a team which optimizes for the bad code written by engineers at high speed.
Good stuff:
Speed - They are really aggressive. And they never think twice to throw away something if there is a better alternative.
Decisions - Right or wrong, decisions are made with least and things progress.
Meritocracy - I wouldn't call it pure "meritocracy", but when they know you are a smart guy who can ship stuff, they'll love you in $ and positions. I know several young people at VP/GM positions.
Data-driven - I see this as huge plus. Lot of companies are clueless because of not enough feedback-data. A/B testing, phased rollout, metrics are mandatory for any product/feature.
Zynga is 100% data driven. Zynga is not a tech-company, it is a product-manager-company. Product managers are more statisticians than PM. All decisions are from numbers.
This statement seems fundamentally at odds with the following. Could you explain further?
Data-driven - I see this as huge plus. Lot of companies are clueless because of not enough feedback-data. A/B testing, phased rollout, metrics are mandatory for any product/feature.
Zynga is 100% data driven. Zynga is not a tech-company, it is a product-manager-company. Product managers are more statisticians than PM. All decisions are from numbers.
Behavioural Targeting: User "A" loves lot of "Strawberry crops" let me show him more of that.
I can't tell if his answers are given out of anger against Zynga or if they are a reflection of reality.
"Google and Apple teams are total cunts. Fuck em, and their hipster bullshit."
Followed by:
"I like Google as a whole, but they hire too many wanna-be-devs. And their SC2 team can suck a big herpes-infested cock. FUCK THEM."
Hard to take this AMA seriously.
> Brogrammers are mostly Silicon Valley / Harvard type douchebags who got into programming. YCombinator drop outs. Years of experience in managed and web languages, but who have no idea how to setup a build system nor work in native code.
I almost feel like "real geeks" are a rare breed these days. It's so rare nowadays to meet a hacker that likes to hack, as opposed to a coder who's really in it so he can afford to rage at the club every chance he gets.
The best 15 minutes you'll ever spend on your career, almost as important as your dubstep collection.
Eh, hes just talking like every other former marine: honesty riddled with swears.
The bad news? That's where the herpes comes into the picture...
He mentioned he quit in August (I wonder how many engineers quit in August? Single digits?) and uses curious language quirks like "[they gave] nay fucks". As a Brit he sounds potentially British to me.
I know people in Zynga's management - they'll relish the opportunity to fuck this guy for contravening some part of his NDA. I'm sure Dani is pissed.
If you are going to 'whistleblow', people, be careful.
I've often wondered, how do you balance the two? When you have that much data, wouldn't it be insane to go by what any single person thinks is "good design"?
I wasn't privy to all the details, but the impression seemed to me that it was a cost-cutting measure (no benefits for comparable pay), and a hiring-cap dodger, since at this company they didn't seem to count.
The increasing use of contractors-as-full-time was part of the reason I left. I personally knew two people who were getting strung along by HR with vague promises to turning full-time, and one who was unceremoniously let go when he was getting close to the legal limits. This seemed like a regular occurrence. I hated working next to people who had no benefits, no job security, and were doing precisely the same work as full-timers.
One particularly nasty one that I ran into would arbitrarily delay payment to employees for 90 or 120 days, unless the employees agreed to pay 10% for a weekly paycheck. The bad ones are even nastier to folks on temporary work visas.
If they are a good place, they will pay people whatever the fully-loaded cost of an employee is. Contractors aren't getting rich.
As for pay, we were hiring people right out of college at $65 an hour plus overtime, which works out to over $130,000. That's more than I made as an employee.
Brogrammer - Brogrammers are mostly Silicon Valley / Harvard type douchebags who got into programming. YCombinator drop outs. Years of experience in managed and web languages, but who have no idea how to setup a build system nor work in native code.
Maybe he's getting Brogrammers confused with web dev scenesters. Awesome Tumblrs, lo-fi keytar side band, scruffy hair, interesting glasses, polaroid cameras. I don't know if those guys have a name yet. Hipster is too easy...it's never really been "hip" to have a desk job.
If you can't handle a pointer, you have no business being near a compiler. If you don't know why you (usually) don't want to handle a pointer, you have no business calling yourself an engineer.
Real "engineers" of course write in Fortran and have no truck with this la di dah hippy pointer stuff :-)
Though I did work on a big billing system for Dialcom systems that was mostly written in Fortran 77.
>Brogrammers
http://www.quora.com/Brogramming/How-does-a-programmer-becom...