g9yuayon

> If there was any feature that would sway existing Golang developers to switch to Java, it would be this.

Not sure about this, but a trajectory question: why does the Go community have so few concurrent containers but Java community does? I mean, even `sync.Map` is specialized for two specific use cases instead of like Java's ConcurrentMap that is of general purpose. And In Java there are concurrent sets, queues, barriers, phasers, fork-join pools, and etc. I'd assume that even with Go's go routines, we can find great use of such containers, right? At least fork-join is not that trivial to implement. And using mutex everywhere seems just so... low level.

I understand that there are 3rd-party implementations, but concurrency is so tricky to get right that I would be really hesitant to adopt a 3rd-party package, unless it is as mature as Java's JCTools or Google Guava, both of which are also backed by a large number of users and developers.

It means they are still courses instead of seminars, and they are not necessarily cutting-edge research materials. Many of the courses are taught to senior undergrads too.

Strangely, I find theories, at least up to junior graduate level, amazingly useful in building systems.

I remember when we were taking an algorithm course, the professor gave us a proof question on the CLR book. It's a property of universal hashing and it used to be a well known corollary. It took me multiple hours to crack the insight that led to the proof. That level of teaching - the ability to select the right problems to nail deep concepts and to push us to grow - is hard to come by if we teach ourselves. At least not for ordinary students like me.

Different styles of college? My college shaped me and what I learned carried me until today. Case in point, I don't think I would learn how to write a database engine, albeit a single-threaded one, from scratch with progressive difficulties, if my professors didn't design the course properly. I wouldn't be equipped with intuitive understanding of maths and stats that I can express using the proper language such as linear algebra, if it were not for the rigorous training given by college. I wouldn't be able to use queuing theory to analyze systems fresh out of college if it were not for tons of fun of studying queuing systems in college. I wouldn't have no difficulty picking up any programming language if it were not for the courses on programming language theories. I wouldn't even know the advanced mathematical logic and formal verifications and computability and a long list of other things if it were not for excellent teaching and unwavering demand from my professors. Heck, I wouldn't even be able to deeply appreciate English classics like The Great Gatsby and Mobi Dick, given that I could barely pass TOFLE when I went to my college, if it were not for the excellent and rigorous writing course given by a demanding lecturer.

I'm forever indebted to my college

I always wonder why we lose the efficiency and intensity over time that we enjoyed in college. When in college, we took five pretty intense courses a semester. We didn't have to wait for "a big chunk of time" to work on our homework. Instead, we got tons done in library or in the computer lab in between classes. And we still managed to find time to date, to party, and to read volumes of fictions. Years having flashed by, and yet somehow I always find half an hour is too short for me to read something interesting, and it always takes more than I can admit to focus on what I want to learn. Yes, we have family and kids and etc so we have less time, but if I check my screen time, oh s%*t! I certainly have time for intense study, but somehow I manage to squander them on social media and what not.

> We have become so detached from nature so much we have forgotten seasons exist,

True for me as I enjoy reading and programming, but I'm not sure if it's true in general, at least not in the metro area where I live. It looks outdoor activity is a big deal in American life. Hiking, camping, fishing and etc are so popular that one has to book a camping site in a state park months ahead of time. The trails are always full of people. Stores like REI either have heavy traffic or run out of stock.

I'm not sure if a process can be a cancer. Instead, an institution that uses processes, Scrum included, to hide their ineffectiveness and inefficiency is.

I grew up in China and the guess culture was predominant there. It was quite a culture shock after I came to the US. In a training session back when I was in IBM, the VP of marketing told us a story about the ask culture: he was an American-born Japanese. When Lou Gerstner asked him what he wanted, he instinctively tried to be humble. Lou cut him off and said: I can't help you if you don't tell me what you want. Come back in X months when you know the answer. The next time they met, the VP told Lou that he wanted to be an executive, and he got promoted soon. Another thing I learned in the training session was that leaders have different styles but all the executives demonstrated only one of the four key styles: direct and decisive.

As time went by, I found it was much easier to adapt to the ask culture. I also found consistency matters more than the styles. When I consistently ask with good intention, people would not take offense.

Hi, Jordan! :-) I consider myself lucky to have worked with you.

My own experience is that there was enough ambiguity in what we needed to build. Take building a CDN for instance, what tech stack would you use? How would you deploy them? What's your caching strategy? How would you partner with ISPs? What kind of hardware would you use? A large company may spend months approving every single such decision.

> Were failures punished by low ratings, bonus?

Netflix's culture stack used to say that culture is all about who gets rewarded and who punished. So, yes, people did get punished for their failures. The key here is what behavior led to a failure? We have to analyze consider the failure and the associated human behavior together and see if there is a lesson or a punishable pattern - this is where strong leadership is needed for a just assessment and action.

There is no rating. At that time, one either their job or not. There was no bonus either, as bonus contradicted to the no-rating system.

> How did they incentivize taking risks?

Taking calculated risk was also part of the culture. Good people actually wanted to take risks. So the incentive systems is to take road blocks away.

> Were Netflix employees from then valued very highly?

I think the success of the company and the quality of the employees make them marketable. Everything else is secondary.

I don't know if anything went wrong afterwards. I left in 2014, so I only spoke for what I saw when I was there.

I joined in 2009 and left in 2014. So the timeline was anywhere in between.

Absolutely. To start with, leaders must not be afraid of their teams making mistakes. Copying from another comment: "Netflix's culture demands very strong leadership and probably works only when the company is small enough. From CEO and down, leaders in every level need to know how to set context for their teams, give enough freedom to their teams, while making sure failures are manageable and success rate are maximized. That is, Netflix assumed that people would make mistakes and fail, but the trick was to ensure that failures do not hinder overall success of the teams and the company."

I'll give an example of the leadership style of Netflix. Jordan Zimmerman, a great engineer who created Apache Curator, once asked then the director of cloud platform, Yury, if he could open source Curator. Yury just casually said: okay, do it. Jordan was puzzled and asked: "don't I have to talk to an attorney? ". Yury smiled and asked: does an attorney help you write code? Jordan said no, and Yury concluded the conversation: then you don't have to talk to an attorney.

I left Netflix years ago, so I don't know their current culture. Netflix's culture demands very strong leadership and probably works only when the company is small enough. From CEO and down, leaders in every level need to know how to set context for their teams, give enough freedom to their teams, while making sure failures are manageable and success rate are maximized. That is, Netflix assumed that people would make mistakes and fail, but the trick was to ensure that failures do not hinder overall success of the teams and the company.

I really like the approach of Netflix of 10 years ago when it was still small. They hired mature people so they could get rid of processes. Indeed, they actually tried to de-process everything. As a result, things just happened. Non-event was often mentioned and expected in Netflix at that time. Case in point, active-active regions just happened in a few months. A really easy to use deployment tool, Asgard, just happened. The VP of CDN at that time said Netflix would build its own CDN and partner with ISPs. Well, it just happened in merely 6 months with 12 people or so. Netflix said it was going to support streaming and move away from its monolithic Tomcat app, and it just happened. And the engineers there? I can't speak for others but I myself had just one meeting a week -- our team meeting where we just casually chatted with each other, to the point that the team members still stayed close to each other and regularly meet nowadays. I also learned that the managers and directors had tons of meetings to set the right context for the team so engineers could just go wild and be productive. At that time, I thought it was natural, but it turned out it was a really high bar.

> at the local cram center.

If school covers the advanced topics, we wouldn't need any cram center. Case in point, my teacher gave my class such challenging problems when I grew up, that tutoring schools were never necessary.

Also, not every tutoring school is about cramming. Instead, they are simply accelerating. The Art of Problem Solving is one of such schools. Another example would be those math camps, such as Math Path. Maths are fun, powerful, beautiful, and leads kids towards a promising future. American schools simply don't care enough about it.

Yeah, Ozone looks interesting. I was just not sure who used it at scale other than a Japanese startup. The community engagement seems much lower than other communities, though.

S3 is a truly amazing piece of technology. It offers peace of mind (well, almost), zero operations, and practically unlimited bandwidth for at least analytics workload. Indeed, it's so good that there has not been much progress in building an open-source alternative to S3. There seems not much activity in the Hadoop community. I have yet heard any company who uses RADOS on Ceph to handle PBs of data for analytics workload. MinIO made its name recently, but its license is restrictive and its community is quite small compared to that of Hadoop of its hay days.

This is such a great article. Amazon tackles this problem by emphasizing "working backwards", but it ultimately depends on the people who enforces such cultural value. I remember in one of the Uber all-hands meetings, an engineer asked the CTO whether Uber engineers should always make sure their systems can handle at least a million requests per second. To Thuan's credit, he unequivocally said no.

Maybe I'm biased, but my experience in Netflix was so good that I never felt the need for a DevOps team. Netflix engineering team did three things right:

1. Any task is an API away and everything except getting a Cassandra keyspace is self-serving. And getting Cassandra access is a matter of a 2-minute conversation.

2. There is always a UI for casual tasks, such as setting up a service cluster or CI/CD pipeline. This is crucial as engineers do not have to learn any non-transferrable shit. The UI will guide them to complete their tasks.

3. Powerful support of observability. Clients, servers, OS, networking... All kinds of metrics are just a finger snap away. In addition, adding metrics is dead easy. If we need something that is below the service level, such as SNMP, we just dropped an email to the four-person observability team, and the metrics will show up in a matter of day before I even had time to finish my own coding. Heck, we had full support of OLAP query on millions of time series with full support of boolean search by any tag (even metric name is a tag) back in 2011, when most people thought Graphite was the best thing since toilet.

God! I miss so much the Netflix days when everything was a non-event. Deploying a predictive autoscaling service to any application in production? Non-event. Getting Druid to production readiness in less than a week? Non-event. Having user-controlled fault injection via APIs? Non-event. Setting up Mesos and Zookeeper and opening up Zookeeper for the entire company to use in production within a week? Non-event. Creating a company-wide crypto service and using it to swap AWS keys automatically for all the Netflix services in production and in test? Non-event. Generating an event with a new schema without talking to anyone and see the event show up in a brand new Hive table under 15 minutes. Non-event, and it was back in 2010. Setting up eBPF traces and access all the pretty and dynamically updated charts for any server via an API? Non-event.

I don't know what more I could possibly ask for.

Wow, isn't this extortion. If this allegation were true, someone in Yelp should go to jail.

The US is a strange place. On one hand, teachers do not treat academic performance as prominently as those in hyper competitive countries, say, China. Classes in high school are easy. Exams are generally easy. Teachers normally do not spend lots of time coming up with enlightening or challenging problem sets. SAT and ACT are embarrassingly not challenging or differentiating. They have only multiple choices, for fuck's sake -- the simplest form in an exam in other countries! The message from the US education system appears to be that it's okay if you don't want to excel in school but we will give you lots of resources if you're an elite student.

On the other hand, the competition is not going away. It just gets hidden or gets distributed to families. So, kids take 10+ AP courses, if not 15+ as in many of Bay Area schools. Kids participated in hyper competitive ECs to show that they are made to change the world to appease the admission officers. In the meantime, they still need to maintain high, if not perfect, GPAs. As a result, they sacrifice their sleep and therefore their health.

It's as if there are two worlds in the US. When I read the novel Folding Beijing, I felt that I was reading about the US's education system, which is sad.

Why? More jobs are still more jobs, no? In addition, each factories may hire fewer people than 50 years ago, but I hope we have a lot more factories. Having more skillful talent does not contradict my hoping that more Americans will have chance to get a decent job.

Maybe fewer jobs per square feet, but it's the natural evolution of manufacturing. What matters is we need engineers and workers on the floor so we can innovate and develop more senior talent.

I hope this trend is real and will continue. Manufacturing is so important to a country. It creates jobs. It builds talent. It stimulates innovation. It improves the lives of millions of people. It keeps a country's defense strong and sustainable. Without a robust and comprehensive manufacturing sector, I don't know how the US can proper in the future. I pains me to see that BYD could produce millions of masks in days in spotless and fully automated factories, while GM workers labored at those manual machines that looked like the scraps from the Soviet Union. Oh, and the $20K toilet? The $200M+ fighter jets? The inability to manufacture the screen of Kindle even if we want to? The list goes on, and I truly want to see the current-day version of the US manufacturing of the roaring ages: cheap, innovative, best workers in the world, full domestic supply chain, and etc.

> Intelligent Brains Take Longer to Solve Difficult Problems

I find such summary ambiguous. Difficult for who? Isn't it very common that a difficult problem for me is a piece of cake for thee, like the PDE that Bezos couldn't solve for hours yet his classmate could solve in a matter of seconds?

In 1697 Jean Bernoulli challenged the mathematicians and scientists in Europe to solve the brachistochrone problem. Bernoulli was particularly proud of his solution after working on it for weeks. Yet Newton, who was 55 years old at that time and hadn't worked on science or math for years, worked out a brilliant solution overnight and submitted the solution anonymously. Reading the solution, Bernoulli famously said "I I recognize the lion by his paw". Not only did Newton solve the problem, he also invented the Calculus of Variations.

> I think one of the chief challenges over the last 5 years is that “tech” became a fashionable, prestige job

This is especially interesting given that Americans used to mock nerds in pop culture and in schools.

A trajectory question that always puzzles me is how fast can we really supply software engineers. Is software engineering like other ordinary white-collar jobs where supply is as elastic as it can be, or software engineering is like math that requires a peculiar personality and certain talent? I draw this question from personal observation, so it's definitely not scientific: China has been big on math education. Math whiz kids are honed like superheroes in China. Teachers and parents put great pressure on students to excel at maths. Yet in the end, only a small percentage of students can truly learn high-school math or entry-level college math. If software engineering is like math, then the supply will stay tight until the demand drops, and techies will continue enjoying good pay after the recession.

Same for the teachers in the Bay Area? I always think this is a great problem for K-12 education. If a teacher has to spend 2 hours to commute to her school, I can't imagine how the teacher would have enough energy to teach kids well. In a great scheme of things, I wonder if kids in the US succeed despite the failures of the US K-12 education. The sad truth is, the majority of the students, millions of them, do need great teachers to push and nurture them in order to grow to their full potential. I can't imagine how exhausted teachers can ever do so.

In contrast, I grew up in China when we had only public education. China did not join the WTO by then, and was quite poor in the current standard. Yet my teachers, for whom I'm forever grateful, painstakingly made sure we understood the fundamentals of science, math, reading, and writing. They designed daily exercises to push our boundaries. They gave timely feedbacks to us (we had at least two 2-hour long exams per subject every week in addition to daily homework, and most of the questions were proof and word problems. Grading them for 50 students in a day or two was not a small feat). I would never push myself as hard as my teachers did, and thanks to them, I could easily thrive in an elite university here and became a TA of CS courses in a sophomore year. I'm sure this is nothing to the elite students here, but I'm not sure if the system of the US can enable the non-elite students like me to achieve such.

If anything, I would love to see my tax dollars go to teachers, increasing their salary by at least 2X. Things like ipad in classroom, compute software this and Epic books that, are really not that essential.