g9yuayon

Is it just me or somehow programming languages and frameworks are not my focus any more? I'm still interested in programming language designs and implementations, but I'm quite neutral on what language/framework to use in my work. There are just too many other things to take priority: organizational structures, system architectures, emerging technologies, new hardware, algorithms, insights in a domain, and etc. Choice of languages barely moves the needle.

That's the point of the thread. We really don't need that much sugar to get good taste. The level of sugar in a cake made by, say Safe Way, hurts my throat.

> your sense of sweetness adjusts to it

Speaking of this, what has always been puzzling to me is that the US bakery has so much sugar in it, to the point that even the smell of the bakery screams too much sweetness. In contrast, European and Hong Kong/Taiwan bakeries use much less sugar, or at least their cakes taste so.

In addition to what people clarified in this thread, you probably will be interested in this: Neural network was not a popular research area before 2005. In fact, the AI nuclear winter in the 90s left such a bitter taste that most people thought that NN is a dead end, so much so that Hinton could not even get enough funding for his research. If it were not for Canada's (I forgot the institution's name) miraculous decision to fund Hinton, LeCunn, and Bengio with $10M for 10 years, they probably wouldn't be able to continue their research. I was a CS student in the early 2000s in U of T, a pretty informed one too, yet I did not even know about Hinton's work. At that time, most of the professors who did AI research in U of T were into symbolic reasoning. I still remember I was taking courses like Model Theory and abstract interpretation from one of such professors. Yet Hinton persevered and changed the history.

I don't think Hinton cared about fame as you imagined.

Writing an essay for someone else is cheating. It's still wrong. Just not as punishable as a crime. BTW, cheating in Keju (Chinese imperial examination) is a capitol crime, as it undermined the fairness of the system and the quality of the ruling class, and derived the chances of other poor candidates. Not that I think cheating is serious, mind you, just wanted to give perspective on how different cultures perceive cheating and the underlying rationales.

> If there was other global knowledge work available to Kenyans I’m sure they’d take it.

I'm not sure the logic stands. Following the same logic, I can also argue that of course looting should not exist, but it does. Looter could've taken a decent job if they could find one. Plagiarism shouldn't exist, but it does. The cheating students could've avoid cheating if they could get higher scores so they can find a decent job. And on and on.

What's wrong is wrong. Period.

> I kept reading news about Google's AI programs

Technology is not a product. A product that uses technology to solve a number of clearly defined real problems is a product. When Google was saying AI this AI that, I had yet to see any product.

> Mao or Stalin might have been able to rid their countries of pathological cultural factors

Mao did rid of the meritocracy-based education system, replacing it with an identity-based system. That is, if you were born with the right identity (blue-collar worker, peasant (not farmer, mind you), and soldier), then you get to go to a university without even any test. If you were born with the wrong identity (the right, the anti-revolutionary, the farmer or any family who has property, the "rich", the "bad"), then you can't go to any college. The results? Just look at China in the 70s. What a fucking hell hole.

As a for Stalin or Soviet, they never gave up their elite education. In fact, elite officers had to be elite students as well. Those who are good at STEM could even get a little leniency when they made the so-called political mistakes.

They answer to voters? If so, nothing will unless the voters think otherwise.

> "Families with resources turn to fee-required online algebra 1 courses in eighth grade, outside the public school system, or enroll their kids in private schools,"

Isn't this obvious result? The Russian School of Mathematics could manage to teach 10-year old kids basic algebra. And there's AOPS, there's Think Academy, and slew of local tutoring schools who can teach kids relatively advanced maths. Let alone many private schools. If I have the means, why would I not send my kids to such schools and therefore fuck up the funding of the public schools? Not that I want to, but it furiates me that the school administrators really hurt the kids who need public education the most in the name of equity. It also saddens me that the constituents are okay with such administrators.

A key question is what emboldened the politicians to propose such bill? It seems the constituents of the law makers have been getting more and more complacent when it comes to giving the government unchecked power, as long as there is a "good" narrative.

Ah, you are right. Gmail is amazing. I somehow thought it was released before 2003.

I don't deny that it can be hyperbolic to many people. As for the comments about people's wild dreams, I truly mean it. I used to tap into building NLP applications for enterprise customers: NERs, relation extractions, multi-round dialogs, information retrieval, and etc. We built dedicated models for each tasks, we spent millions on labeled data and annotation teams in general, we worked with customers closely to address their specific issues. It was a painful process and customers are not happy, and I overlooked the significance of OpenAI's papers on GPT and Google's paper on emerging ability out of sheer ignorance and stupidity. Then, ChatGPT came along and could have pretty intelligent multi-round conversations, could handle multiple NLP tasks easily and have better performance than my models with my canonical tests, and could be helpful to me on a daily basis. It was shocking iphone moment or a personal Sputnik moment to me. For that, I extrapolated that it was "beyond wild dreams of millions of people".

Nothing particular. I just don't remember major successful and speedy product launches from Google after 2003. In that year, Google launched Google Maps, shocking the IT world for how effective AJAX can be, even though the tech itself was invented by MS. Maybe GMaps was so impressive that I forgot what other major launches were after that.

OpenAI is like Google before 2003, dazzling the world with amazing products backed by astonishing technologies that went beyond wild dreams of millions of people. And most importantly, OpenAI kept releasing new features at blazing speed with a few hundred people, just like early days of Google and Microsoft.

This reminds me of stark contrast of Google's GSuite and Microsoft Office. Microsoft released new office every couple of years, introducing huge improvements and packing tons of features[1]. In contrast, GSuite appeared to be, hmm, quite quiet.

[1] You may argue that no one needs that many features, but I subscribe to Joel Spolsky's idea: https://www.joelonsoftware.com/2001/03/23/strategy-letter-iv...

The github repo has 5 million lines of C++ code (headers included), 1.6 million lines of C code, and even nearly 1 million lines of Scala + Java code. We'd need some serious docs to adopt this technology.

The most interesting part of YT is Cypress. I'm particularly interested in how they make their master cluster horizontally scalable.

I created a variant of a classic elementary math problem for both Bard and ChatGPT with GPT-4. The problem and the answers are below, and the differences are:

- Bard does not understand Chinese but ChatGPT does.

- Both can solve the problem, and Bard gave a more concise solution.

- Bard gave an incorrect answer if changed the problem for indefinite answers, but ChatGPT still gave the correct answer.

The question in Chinese was 今有雉兔羊马同笼，上有头一十有九，下有足六十有八，兔马羊同数。问雉兔羊马各几何.

The English version was: there are chickens, rabbits, sheep, and horses in the same cage. There are 19 heads in total, and 68 feet in total. The numbers of rabbits, horses, and sheep are the same. How many of each animal are there?

If I remove one of the conditions, ChatGPT can still give correct answers, while Bard will make up incorrect results: there are chickens, rabbits, sheep, and horses in the same cage. There are 19 heads in total, and 68 feet in total. How many of each animal are there?

> My whole life, my whole personality is architected around making things by hand for other people.

Not sure if there's a conflict here. We still use compilers, right? Unless we write in machine instructions, there's always some kind of program generations somewhere.

> I get no joy out of duct taping a few libraries together (though I respect others who do).

You still can work on stuff that requires human ingenuity . If you look at the the credits section of the GPT-4 paper, you will see many people there because they are the master of "low-level" optimization. For instance, the author of Reformer is a lead in the Long Context section, because apparently he knows how to reduce the O(N^2) of self attention to O(Nlog(N)), to say the least.

The paper does not offer enough details on how GPT-4 is implemented. And the paper also says in its Section 2 that "We plan to make further technical details available to additional third parties who can advise us on how to weigh the competitive and safety considerations above against the scientific value of further transparency". That is, no technical details to general public. If this trend continues, I'd say companies will be crazy to think that they can always rely on OpenAPI's APIs, so the arm race of building LLMs will be on, if it has not already started.

Also, the most valuable part of the paper is p15 - p18, the credits. /jk It gives me three pieces of information:

  - The credit list contains 200 people, give or take. It's going to be hard for universities to compete with OpenAI without intercollegiate collaboration. 

  - On the other hands, it's amazing that OpenAI can build a product instead of a research model with only 200 people. Big companies should really take notice. 

  - OpenAI just gave us the org structure for building a GPT competitor.

> if you want to master something, you should study the highest achievements of your field

> Thinking about this question, I wrote down a list of twenty names—von Neumann, Tolstoy, Curie, Pascal, etc

So the author advocates that we learn from geniuses how to master something? I'm not sure if this premise is true to start with. Von Neumann could solve complex PDE systems in his mind in minutes if not seconds, can we no matter how hard we try? Poincare worked only four hours day, can we? Euler wrote faster than people could digest his writings, can we? Galois invented group theory when he was 19, yet how many people could really understand the simplest college-level abstract algebra after they graduate from a STEM field? Feynman (or Einstein?) self taught himself calculus by 12 or 14, yet how many students struggled with the concept of variables when they were at the same age no matter how hard they and their teachers tried? Those are extraordinary people with extraordinary capabilities. I'm not sure how much we can learn from them to master something.

Rather, I would learn from ordinary people who struggle from time to time yet somehow manage to master something.

Instead of focusing on scalability, focus on productivity. Scaling is optimization, and optimization needs to be guided by profiling or generally instrumentation in production. Having good culture and a productive infrastructure, a company can rapidly scale its systems if needed. Without such productive environment, a scalable system could crumble any time. Reflecting upon my days in Netflix, I realized that we didn't build scalability from day 1. Instead, we evolved a scalable Netflix quite easily. Almost everything was a "non-event". When needed, things just appeared to have happened. That's the magic of productivity, not scalability.

> > where the student is being asked to produce bullshit

It depends on which subject? I have only good things to say about my college days. They are life changing. The rigorous training in computer science and maths has been paying dividend all these years. The professor in the writing class did not only teach me how to write, but also how to understand and appreciate literature. Even classes like Canadian Culture and History taught me so much about arts, literature, politics and history of Canada. I have a hard time fathoming in what classes will we write bullshit (yeah, I have some ideas, but I guess I was lucky enough that my school taught me how to love and appreciate the wonder of nature and civilization, instead of teaching me hatred to towards you know what).

The structure is different, right? "Me gusta algo" does mean something is pleasing to me, but it's not the way how one would think in Chinese or English.

My mother tongue is Chinese and I find that English is the easiest second language to learn. English has more or less the same grammatical structure as Chinese, so I didn't have to learn structures like object -> subject -> verb as in Spanish or reflexive verbs or structures like "Me pleases programming" (Me gusta programar) that means "I like programming", or the sentence structure of topic -> object -> verb with all kinds of particles in Japanese. Yes, English has a few conjugations, but boy that is so much simpler than those in Spanish. And English pronunciation is easy for a Chinese, but boy other European languages are hard, as Chinese does not have alveolar R or double R as in Spanish, and don't even get me started on the pronunciations in French and German.

Do we know the largest size of data that these object stores have handled? People seem have been moving away from HDFS, yet companies could host exabytes of data on HDFS and serve TBs of scans per second with a single team of fewer than 15 people. I was wondering how production-ready the other OSS alternatives are for such scale of data.

I also find that writing by hand led to better retention and understanding when I learned a new language, compared to writing with a keyboard. My guess is that it's because handwriting is slower, which somehow creates a better focus window.

I wish companies could go lean. I had tremendous growth when working for Netflix partly because engineers there got to build meaningful systems with huge ownership and very small teams. Case in point, my coworker and I owned four or five high-traffic systems and carried a pager 24x7. It didn't bother us at all as our we got alerts maybe once or twice a month.

That said, a lean company requires strong leadership, technical depth in management, and specific types of cultures. I don't believe that most companies are up to such challenge.

Is there any theory on why Amazon decided to spread the cut over months of time instead of planning sufficiently and then having a decisive cut?

This is amazing. So another exchange committed bank fraud, and BlockFi's answer is to block withdraw. Does it smell something very fishy or just me?

> My current company spends 4 out of 8 hours every day in meetings.

When I was in Netflix and Uber, my teams had no status meeting, at all. We came up with a context and a big enough ownership, and just let the team loose. It's something like this: we need to enable services to send their events for analytics without ever talking to the data lake team. Steven, how about you own it? Then, a month or two later, everyone in the company can simply use log4j (at that time) to log an instance of annotated Java class, and in 15 minutes or less, the logged events became available for all kinds of queries in the data lake in the right format with the right metadata. No status meeting. No alignment meeting. No customer outreach. Not unnecessary shit. Things just happened, features just introduced. It's all non-event.

In the end, it is the culture that matters. Everything else is a matter of tactics.