SpikeGronim

Hey Yuri long time no see. Facebook me if you're ever in Seattle.

I'll share my experience, which may differ from other people's. The largest system that I've worked on was Amazon S3. At the time that I worked there we were doing 100,000+ requests per second (peak), storing 100+ billion objects (aka files), and growing our stored object count by more than double every year. The most important skills for that job were distributed system theory, managing complexity, and operations. I can't explain all of these skills in depth, but I will try to give you enough pointers to learn on your own.

For distributed systems there are two main things to learn from: good papers and good deployed systems. A researcher named Leslie Lamport invented a number of key ideas such as Lamport timestamps and Byzantine failure models. Some other basic ideas include quorums for replicated data storage and the linearizability consistency model. Google has published some good papers about their systems like MapReduce, BigTable, Dapper, and Percolator. Amazon's Dynamo paper was very influential. The Facebook engineering "notes" blog also has good content. Netflix has been blogging about their move to AWS.

Every software engineer needs to manage complexity, but there are some kinds of complexity that only show up in big systems. First, your system's modules wil be running on many different machines. The most important advice I can give is to have your modules separated by very simple APIs. Joshua Bloch has written a great presentation on how to do that. Think about what happens when you do a rolling upgrade of a 1,000 node system. It might take days to complete. All the systems have to interoperate correctly during the upgrade. The fewer, simpler interactions between components the better.

The best advice I know of about operating a big distributed system is this paper[1] by James Hamilton. I won't repeat its contents, but I can tell you that every time that we didn't follow its guidelines we ended up regretting it. The other important thing is to get really good with the Unix command line. You'll need to run ad-hoc commands on many machines, slice and dice log files, etc.

How did I learn these skills? The usual mix of how people learn anything - independent study, school, and building both experimental and production systems.

1. http://www.usenix.org/event/lisa07/tech/full_papers/hamilton...

You can do that quite cleanly in Python:

import commands print commands.getoutput("your system call and args")

There's no point in adding however many MB of download for an add-on that so few people use.

It is good for my mother. She fell 37 Windows Updates behind because she canceled them. Chrome is stable and most UI changes are configurable, what's to argue with? Google's disciplined six week cycle[1] is what makes Chrome get better faster. It is good for you, the browser consumer/developer.

1. http://google-chrome-browser.com/how-google-speeds-chrome-re...

Doug Cutting works at Cloudera. Also, this is non-tech media reporting on tech so if they are even sort-of correct then good for them.

The difference is that I notice when Windows updates, and I don't notice each time that Chrome updates.

If your backup provider elects to delete your data due to an obscure ToS clause then it is their fault for sure. The article does not say anything new about cloud providers - you have to trust their competence. But this is a critical failure by EMC / Mozy and is nobody's fault but theirs.

They are really deprecating whitelisted REST apps in favor of their streaming API. The streaming API is WAY better, especially for time sensitive apps.

I do not trust comscore's numbers at all. They rely on opt-in sampling of user data. There is obvious selection bias there. Large companies will never allow such tracking software, so anybody browsing from work will never be counted. There are many other large demographics that they just plain miss. They have a history of goofing badly, for example being directionally wrong about Google's ad revenue [1].

1. http://seekingalpha.com/article/72911-why-was-everyone-wrong...

Wavii is hiring in Seattle, WA.

We need frontend developers to work on a rapidly iterating consumer facing website.

We are also hiring backend developers interested in natural language processing and the infrastructure needed to support large scale data processing.

TechCrunch coverage:

http://techcrunch.com/2010/07/21/time-to-meet-wavii-the-supe...

http://techcrunch.com/2011/01/17/wavii-waviimania-investors/

jobs@wavii.com

I doubt that YC has $15 billion in opportunities every year. They also focus on using their expertise and networks to help their startups, so having 1,000 YC startups wouldn't work.

It's nasty when the ISP returns an A record for an advertising filled spam page instead of NXDOMAIN.

I always put my keys in the same place. Very simple. Works.

I think that's a bad example. AFAIK you can't legitimately buy Viagra online, so what results should Google be returning? Where are the awesome web pages about Viagra that you want to see? They don't exist, so the query comes back with spam.

That's very much a risk for Quora. But if they're still showing me my friends' activity that could ameliorate the problem. My friends are still going to be asking/answering questions that I am likely to be interested in.

(concentric elite growth strategy || answers filtered by my friends) + email notifications == win

Quora has so many respected tech industry people using it that I want to use it. That's similar to how Facebook was first Harvard only, then Ivy League, etc. - you want to be where your perceived social betters are.

My Quora feed shows the topics that my friends are interested in. This is super valuable for finding information slightly outside what I would search for myself.

Their email notifications keep me clicking back to the site so that I stay engaged.

This article just covers the downsides of selling equity to raise money. Obviously there's an upside - having money in the bank with which to grow your business. Some projects need a team of people and that costs.

Nope. From what I've heard it is standard practice for later rounds to dilute the investors from previous rounds.

Parsing speed depends on the markup. One example is text layout, which is O(N) in the size of the text area. So 1 MB of text might be a lot slower to parse than 1 MB of no-op tags.

Any of the The Culture novels by Iain M. Banks. You can start with the first, "Consider Phlebas". They're space opera with a lot of dark wit.

You're right and I would prefer to link. It comes up in The Economist from time to time. I can't find the links right now.

Thanks for the citation. That same article says that "at least one study has found that it's possible to teach these visuospatial skills." I can interpret the result "males in the study did better at mental rotation tasks" to mean "males brains are wired better" or "males have better math educations". Both interpretations fit the study evidence as presented.

The 3rd part of the linked article even says:

"So it appears that there are a wide variety of social factors that affect (or are affected by) sex differences in math and science. Because of limitations in the way these studies can be controlled, it's difficult to say that discrimination or differential treatment cause the sex differences we see in math and science."

Overall, the linked articles do not support the hypothesis that "men's brains are wired better".

To me, it's not interesting at all to ask who can write a 100-1,000 line program faster in which language. The real interesting question is how efficient developers are once you're talking about a 100,000 - 1,000,000 line program that has been in development for years. Solving small problems is relatively easy.

The citation was [1]. It is typical for women that attend open source conferences. That really sucks, for everybody involved.

1. http://www.networkworld.com/community/blog/there-sexual-hara...

You are being sexist - you're overgeneralizing from one example (your girlfriend).

I didn't mean to insult you. I did mean to say that the aggregate behavior of many men has discouraged many women from participating in CS. That's not disingenuous, it's a fact supported by evidence.

Yes, I have a citation[1]. I've also got anecdotal evidence from studying computer science at Stuyvesant High School and Carnegie Mellon University. To be clear, I didn't mean that programmers everywhere disrespect and harass women. I meant that sufficiently many programmers act this way for it to have a negative effect on female CS participation.

1. http://www.networkworld.com/community/blog/there-sexual-hara...

EDITED TO ADD: where's your citation for the "brain wiring" comment above?

Several studies have reproduced a correlation between diversity and organizational success. I don't know of such a study for programmers specifically. I also haven't seen proof of causality - do better organizations have diverse hiring practices, or vice versa? But I've never seen a study that said that lack of diversity was a good thing. So far the evidence weighs towards diversity being better for your organization, though I wouldn't call it "proven".

How many times were you groped by a peer while learning to program? When you were in high school and first learning to code, did you have peers whose presence inspired all the women to leave the room because of their obnoxious behavior? These patterns of behavior are a real, measurable problem that discourages female computer science participation. That shrinks the talent pool and benefits nobody.

Most people aren't good at programming, at least until you train them. It's hard to convince women to spend ten years becoming excellent programmers when their peers disrespect and harass them. I've never seen credible evidence that women's brains "lack the wiring" - that's an awfully Victorian era sentiment. The fundamental problem here is with the behavior of men in the field of computer science.