In Netflix more than 10 years ago, it's more like this: a single engineer builds a deployment/management tool: 1 - 2 months. Every other engineer creates a new and fully configured cluster: minutes.
Seriously, can we please get over the fetish of using anything this DSL that YAML or whatever "specification language"? Such tools are powerful, flexible, but should not have a place for engineers who just want to provision resources. The tools violate almost every UX principles, in particular the following:
- Discoverability. Very little. One has to read tons of docs and SO posts to figure out what needs to be done. You want to pass in some environment variables? A typical answer from those who use Nomad/TFE: easy, just pass in this 200 lines of Jinja template. Really? Really? You call this ease of freaking use?
- Affordance. None.
- Constraints. If you call the errors only after you submit your 1000-line yaml scripts.
- Consistency. Maybe, but still, embedding a Jinja template to pass in variable is an insult to UX.
It's a unrepentable sin to ask me to learn your shit.
Nostalgia time! When I graduated years ago, Knuth's books were still all the rage because the central themes in the CS departments in universities were fundamentals: algorithms, compiler theory, operating systems and what not. We referenced TAOCP for implementing the priority queue using fibonacci heap for our OS projects. We studied chapters on vol 2 for some of the numerical algorithms, and etc. We read Parsing Techniques cover to cover to understand how to write a parser. We still carefully studied AI: A Modern Approach and the dragon/dinosaur/tiger book. Every once in a while a discussion on Knuth's work went to the first page of HN or Programming Reddit. Then, we got more and more mature libraries, and we rarely needed to understand 10 different algorithms that solve the same class of problems. Consequently, we got fewer discussion about Knuth's work too. Of course, it's super fun to read his meticulous discussion of combinatorial algorithms in Vol 4. It just that TAOCP does not appear as essential to as many people as a decode ago.
Many parents seem have forgotten that kids are supposed to get top education instead of getting into "top" schools. As a results, we see kids take 15+ APs in high school, even though they leetcode into a FAANG after graduation (yes, I'm talking about your kids, parents in the bay area). Or worse, the kids burn out in high school. We see kids build novel clubs to "be a world leader" in high school and only abandon the idea once getting admitted. We see parents focus so much on rankings without thinking twice what exact a highly ranked university would offer to their kids. On the other hand, we see universities game the ranking systems. We see universities inflate their GPA scores. We see universities try every shady way to lower their admission rate.
I'm still hopeful, though. Eventually good education wins. Life is a marathon, not a 100m dash. Getting into college or not is merely the beginning of one's adult life. In larger scheme of things, which college one goes to or how many APs one takes hardly matters.
It seems China had a solution, but no more. When I grew up in China, cities had dedicated bike lanes. Each lane was at least 2-car wide, and was separated from car lanes by a green belt, so bicyclists were safe. The bike lanes disappeared or shrank in width as China got more and more cars, and the green belts were removed as well, making it a lot more dangerous for bicyclists.
The challenge of "living close to the office" is that people change jobs every few years, especially in the bay area. Unfortunately we don't change our addresses as often.
> “Students fell four months behind in school, and reading and math scores sank by the sharpest margin in three decades.”
The past three years of Covid was kinda like an equity experiment in education: many families lost access to tutoring schools, schools lowered their requirement on the taught content, tests are more lenient. And then results? Families with resources didn't get affected at all. As a result, the gap in education enlarged. If the US adopted the equity policies like SFUSD, I'm afraid that the families who struggled with social economics will lose out more, instead of gaining.
Uber acquired a startup around 2015. It was that startup that was using shadow-matching and particle filtering to address the problems mentioned in the blog. The startup was founded by a bunch of UCLA PhDs who studied the problems, so I guess they had something innovative.
Is 18% or so acceptance rate really low, though? Almost 2 in 10 submissions are accepted, and I thought "the top" meant something like 2% or less.
BTW, is there any resources that catalogs which ideas in papers may work well in industry? As someone outside of academia, I find there are simply too many papers, even from top conferences, for me to consume. It's hard for me to know which paper's ideas can help me or not, and this 18% acceptance rate is not a good enough filter any more.
> Get a job at smaller to medium sized non-tech company.
I do the first part: finding smaller to medium company. Smaller companies have better visibility, more work than people can finish so everyone gets to work on big whales. There's in general a better comradeship too. Revenue per employee in good smaller companies are usually larger than that in big companies, so people don't have to fight for resources as much.
I'm not sure about "non-tech" companies, though. IT in non-tech companies are usually cost center, so there's usually insufficient investment in tech workers in those companies. Some line of work can be hard to find in non-tech companies: distributed systems, large-scale database systems, high traffic systems that require deep optimizations on throughput, latency, and etc.
Sounds we should really build walkable city with convenient mass transit in the US and Europe. It's a shame that building anything infrastructure in the US is prohibitively expensive.
Speaking of language learning, I wish Duolingo could come with leveled readers. We build our intuition on languages only when we see words and phrases used in different context. Somehow it's hard to find good Spanish readers, compared with ample readers in English. I was trying to read the reading materials for the Mexico first graders (https://libros.conaliteg.gob.mx/primaria.html), oh boy, there are just too many new words for me even though I was at Duolingo Spanish unit 5. I used to use a reader called America Today. The book 2 talked about all the natural wonders of the US, and the book 3 talked about the America society: a day of supermarket manager, a day of a big city, and etc. The difficulty of the book was so well arranged that I had no problem progressing through it, and I had no problem with English grammar or day-to-day vocabularies after finishing the series.
Not really. Use whatever technology that you know best, but hide unnecessary complexity through the right tools. I'm not sure about the OP, but I had extremely good experience when I was in Netflix. A single engineer, Joe Sondow, created the app Asgard. Using Asgard, I didn't need anyone to teach me how to configure clusters or networking topologies. The app got affordance, consistency, and constraints right, to the point that at any point what I guessed was either right or the only way to complete my tasks. Michael from the cloud team created command-line tools for managing AMIs. They exposed just what one needs to achieve while hiding anything about how things were done. As a result, I can still remember how to use these tools, years after I left Netflix. Similarly for doing service discovery, fault injection, load balancing, autoscaling, predictive autoscaling, logging events, publishing metrics, and the list can go on. Everything in Netflix, at least when I was there, followed the unspoken principle of making everything a non-event: zero unnecessary mental load, maximum flexibility, maximum freedom to engineers, and full transparency. Things just happened. Tools were just in place. The tools could be primitive and could lack of features, but the they did not get in my way whatsoever.
My point is that the author needs to make it clear what exactly DOP can do better for. OOP, whatever how that is defined, is just an example of what people discussed under the OP.
Reading the discussion here, I can't help but thinking that people are defending their own philosophies: OOP vs FP vs DOP vs etc. I wish the author had killer applications or killer examples in different categories, like can I code an operating system easier, can I code a database easier, can I create a complex streaming job easier, can I write a library as complex as Apache BEAM easier, can I write a compiler easier, can I create a web framework easier, can I write a JSON parser easier, you get the idea. Or maybe examples that contrast existing solutions: how do I use DOP to write a better RxJava? how do I use DOP to write a better SqlLite? How do I use DOP to write a better graph library? How do I use DOP to write a better tensor library? how to do use DOP to write a better Time/Date library? You know, something that's so compelling and so obvious.
I forgot who, but someone influential said more than 10 years ago that browsing internet is like having intellectual squirt, which keeps us wanting more. I guess a phone with Instagram/TikTok/Twitter just make it worse. My counter measurement is surrounding myself with all kinds of books and articles and courses. Whenever I have the urge to go to Twitter, for instance, I'd ask myself why. Usually being aware of the urge is enough for me to stop the urge. Otherwise, I'd pick a reading at the moment. If I really worry that I may spend too much time reading instead of going back to work, I'd take a Duolingo class, which takes no more than 5 minutes.
Yeah, leetcode type of test got popular because hugely successful Microsoft and Google used to use them to pick the smartest geeks for amazing technical challenges for their times. Now that leetcode is being abused by both companies and candidates, we really should find a different way to interview.
My wife went to a CC before going to a 4-year university. The challenge of studying in a CC, at least to some students, is that CC's courses are not rigorous enough. The courses are on par with those AP courses. For instance, their Calculus class focused on derivatives and integrations, while spending far less time discussing fundamentals like definition of limits, convergent functions, series, or boundary conditions in differential equations. Needless to say, they touched little on proofs, either. Their linear algebra class focuses more on matrix calculation than properties of vector space. Similarly, there were few proofs in their Discrete Maths course, little discussion on computational complexity or computability in algorithms class, no comprehensive projects on data structures like balanced trees, or graph algorithms beyond simple search.
These are not inherent problems with CC, but lacking rigorous course does impose a challenge to the students who aspire to be transferred to a 4-year college to study STEM, as they often felt insufficiently equipped to take more advanced courses in universities.
> Further, though Mr. Kail complained of problems with Sumologic (as one would see with any new startup), the product itself was “useful,” according to Ashi Sheth. (R.T. Vol. 8, p. 1670-71). As described below, at the time, Sumologic saved Netflix from paying for a far more expensive and inferior product called Splunk.
Wow! I was evaluating SumoLogic and Splunk in Netflix back then. Neither of them was suitable for our use cases. We ended up rolling out our own solutions. As far as I recall, the eng org didn't use Splunk or SumoLogic. Kail headed the IT department, though. Maybe they used SumoLogic.
I used to think that the K12 math education was really bad, but now I have a different perspective. Many public schools in China and Russia are essentially like magnet schools in the US because students need to compete for entrance. So, those schools can afford more challenging syllabus. In contrast, the US has few such magnet schools, and people of different academic capacity go into the same school. As a result, teachers can't really teach too advanced concepts or assign too hard homework. This may sound far-fetched, but you may be surprised that more than 50% of students may never be able to understand Euclidean geometry, let alone analytical geometry or set theory or probability or proof by induction, and etc. And more than 50% of STEM students in college may not truly understand calculus, let alone analysis. Case in point, roughly 50% of middle school students can't get into high school in China, even though the entrance exams are really not that hard as most of the questions are designed to examine a student's understanding of basic concepts. Therefore, we may look at the quality of the math class in the US and feel miserable, but that's only because the class is tailored to the lower 50%-ile. On the other hand, the clubs and gift programs in public schools are still of high quality -- all the more reason for not canceling such programs in the name of equity.
This got me thinking about the recent school policies for equity. Schools reduce difficulty of courses, lower standards for graduation, and cancel gift programs, all in the name of equity. I think in the long term such policy changes will hurt disadvantaged families. Family with means will simply send their kids to good tutoring schools, and the education gap among different groups of people will enlarge. It is the kids in the middle, the future backbone of our society, who will suffer most because they need strong guidance and skillful push to truly learn, yet they won't get such education because of the watered-down standards.
I was wondering if the effect of valuing education is subject to survivorship bias. Take China for example, on average 50% of the grade-9 students have to enter vocational training schools instead of high schools (grade-10 to grade 12). The odds of getting into the top 33 universities in China is less than 2%, and the odds of getting into the next best 100 universities is less than 4%. I'm sure more than 6% of parents value education to the extreme, yet large number of students may not get what they desire no matter how hard they try.
A key reason for Netflix to have an easy-to-operate infrastructure is that Netflix prioritizes productivity and scalability. They specifically did three things:
1. No fixed deadline, with a few exceptions of course, for
platform-related projects.
2. Promotion/salary negotiation was not tied directly to release of external features.
3. A single engineer could be responsible for more than one service for the entire company, with 24x7 oncall.
With Netflix establishing such incentives, engineers naturally focus on getting infrastructure right, to the point that oncall 24x7 is a non-issue.
So, yeah, culture matters, big time.
Edit: another incentive was that a service was measured by its adoption. The more people praised it, the more successful the service would be. Requiring meetings to get buy-in for a new service was considered a sign of potential failure. As a result, every single team focused on making the value proposition of their services obvious. Path of least resistance was a given instead of a debated topic.
The OP seemed to be talking about implicit bias in the framework of critical race theory, hence the quote "According to media and social media, if I deny that I'm a racist, then I'm just not aware of my implicit bias". That is, denying that you're racist is the proof that you're a racist. It's not about self-awareness, but about assertion.
Otherwise, I don't think the implicit bias is what OP said. Our HR would remind us recency bias, for instance, during a perf review. That kind of implicit bias does exist and is worth reminding.
They don't have high teacher to student ratios. Usually schools have 1:50+ teacher-to-students. Teachers do grade students' homework and exam carefully. Teacher's quality and dedication overweights the ratio, by far.
Nearly 50% of students in junior high schools will not be able to enter high schools in China. They will enter vocational high schools, in a way similar to those in Germany. And what do Chinese do? They do this: https://images.app.goo.gl/pKvyBtNtBgMhMK1T9 In the picture Chinese parents send their kids to math tutoring classes, and they stayed in the back of the classroom to study too, so they can help their kids after class.
Yeah, laugh all you want, believing that training 5 hours a day for whatever bullshit sports is superior to doing algebra a couple of hours a day. But this is how some millions of kids fight to get into college and change their life, despite limited resource and all the struggles in their life. And the government? They make sure teachers can be fired. They make sure teacher's work can be measured. They make sure teachers will have a career ladder. They make sure teachers can have decent income especially pension.
And states in the US (K12 education is the responsibility of individual states)? I'm sure they do a better job at K12 education. The US, after all, is a developed country, right?
I wonder if this has to do with the overall decline of manufacturing in the US. Supply chains have been moving out of the US so iteration will be slower and more costly. Fewer talents would like to go to manufacturing. Fewer university professors are researching traditional mechanics now, for example, but more on "cutting-edge" stuff like nanometer materials. We also get fewer and fewer highly trained workers on lathing, welding, milling and etc over the years. As a result, it gets more expensive and more risky to build something as complex as F35.
I'm not sure I get the logic of the article. Google AI Research is going to implode because we know what the current models can't do? I thought we do research precisely because we wanted to expand our boundary of out knowledge. That is, shouldn't a research institution implode if there is nothing to research?
And why is Gebru relevant in this article? Does she do quality research, and the research results have to do with the claimed implode?
I'm not sure if the calculation is correct. 2000 people winning lotteries out of at least millions of people, right? Yet in the startup word, it is thousands of people out a few hundreds of thousands? Another factor to consider is that your equity keeps giving, while lottery is a one-off deal.