throwaway713

> we ask them to stop testing advanced mathematical problems on proprietary models.

Maybe I'm alone on this, but for some reason these sorts of requests strike me as akin to gatekeeping how someone should breathe air. It's math... the numbers and symbols are just out there in the platonic realm available for anyone to do as they like with them. It's patently absurd to request other people to stop.

Ensuring credit where credit is due? That's fine. If your model incorporates the efforts of many others, then it's reasonable to request acknowledgement of everyone who contributed (even indirectly). But that's not what the request states — presumably their ask subsumes any advanced ML model, including those that weren't trained on a giant corpus of text.

I've been thinking about this comment for a few days; was it intended to be serious or facetious? I wasn't criticizing Tao in any way. I just think there's a lot of interesting work that's not in his primary domain of study for which AI has some potentially non-obvious benefits.

You need to have money to do the right thing? Odd take. Plenty of people with much less have sacrificed much more throughout history.

I think the perspective from inside vs outside the buildings is important. Consider Meta’s classic campus vs their newer buildings designed by Frank Gehry. I like the exterior appearance of the classic campus much better as it feels “friendlier”, but from the inside, it flips drastically. The newer buildings are far superior in every way compared to the dimly lit, fluorescent feeling you get inside the older buildings with the small windows.

Contrarian take: I think the ability of AI to produce valid mathematical proofs (even inscrutable ones) is absolutely fantastic. Mathematics as a profession does not have a monopoly over math itself any more than professional pianists have a monopoly on who plays piano, when they play, and how.

I have sympathy for any jobs that might be affected (much as my own job has become more tenuous in software engineering). And if the field is disrupted by chaos that makes the research process unproductive, that's bad too and should of course be handled by applying better organization within the institutions that tend to perform mathematical research.

But to a large degree, the notion that "sloppy AI proofs are bad for mathematics research" seems like a total failure of the imagination to me. Attempting to find shorter proofs or more elegant proofs can be turned back in on itself via proof theory. There are proofs in Presburger arithmetic that are doubly exponential in the length of the sentence. Yet a more powerful theory like PA makes quick work of such theorems. The explainability or "subjective beauty" of a proof can be quantified and optimized against. Optimization itself can be optimized against. I really don't understand how this magical ability to know the truth of more theorems much more quickly—even via an "ugly" route—is anything but a net positive.

Am I missing something obvious? Isn’t this just a simple DB query to see the state history of the “Data Controls” → “Improve model for everyone” toggle in the settings? Just report whether that was ever on and over what time period.

> The problem I have with good faith debate is that it often falls into a fallacy of "fair-time" meaning we think we have to give the other side equal time

The Catholic church could have said exactly the same thing at one point. "Why should we even devote time to an argument as absurd as the earth not being the center of the universe?" There are darker examples along the lines of those you give, with beliefs quite opposite to those we have nowadays.

Suppose one selects an arbitrary hot-button issue [X] with two opposing sides and one side has anything less than overwhelming support. And then that person writes an article titled "Side 1 of issue [X] is true". Not "maybe" or "possibly". Just a straight-up declaration by fiat.

Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible.

That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.

Interesting. I’m going to start describing my “annualized impact” in my performance self-reviews in terms of all the things I project that I’ll do.

Bold title for something from DeepMind. I thought a crank submission slipped onto the front page somehow. I guess the next paper will be “Why AI cannot instantiate God”?

I'm a little bit mixed after reading the article. I guess I don't entirely blame companies that see a financial opportunity to help enforce laws, as ostensibly, it should be win-win for both the public and private space. Where it breaks down (as the article points out), is when the law is ineffective at achieving it's purported goal. This is where I differ somewhat from the article's conclusion, however, as I view it as the government's responsibility at that point to correct the dysfunctional law. If all the evidence is there—do something about it. There might be a moral expectation that private companies "do the right thing", but there's certainly no practical expectation that will occur.

Case in point: where I live, the interstate is often congested, and a driver "camping" in the left lane frequently leads to traffic jams that back up for miles. The cars that get backed up become frustrated and start zooming and weaving through traffic in the right lanes to get past the blockage. And while there are plenty of police, they only go after the speeders (presumably because speeding tickets are more lucrative). I don't think I've ever seen someone pulled over for squatting in the left lane, despite the fact that it's illegal where I live and despite the presence of numerous signs that say "Keep Right Except to Past".

This is what I would call an example of a dysfunctional law, as I highly suspect that if one had the capability and interest to analyze aerial footage of traffic patterns, it would be found that left lane campers are a much more significant factor in the root cause of interstate traffic accidents than speeders. But the incentives are too perverse to fix the problem, so the situation persists.

> the world outside your front door is to be treated with suspicion; that every passerby is a potential threat; that every neighbor is a potential enemy; that every human interaction must be stored and cataloged as evidence of possible crime.

Yeah, I think he summed it up better than I could have there.

> Fluid intelligence, which peaks near age 20 and declines materially across adulthood [...] while fluid intelligence may decline with age, other dimensions improve (e.g., crystallized intelligence, emotional intelligence)

As someone well past "peak" fluid intelligence at this point, I always hate reading research like this. "Crystallized intelligence" and "emotional intelligence" are the consolation prizes no one really wants.

I'd rather we instead perform research to identify how one might reverse the decline of fluid intelligence...

Sort of like the people who work in big tech and the people who post on Hacker News. You'd think the intersection is an empty set, but it's probably pretty large.

> Every time you see collaboration happening, speak up and destroy it. Say “there are too many people involved. X, you are the driver, you decide.” (This is a great way to make friends btw).

Corollary for managers: Do not say "it's your call", then once the decision has been made (and you skipped all the meetings pertaining to that decision), comment about how you would have done it differently and then retroactively request your report to go back and make changes. This is a great way to lose employees.

Ah, ChatGPT’s hidden INTP mode. We’ll finally get the right theory for ASI, but it will provide no clues on how to actually implement it in a timely manner.

I saw that Reddit post a while back. It’s interesting, but I wonder how much it really applies to all of the super wealthy. There are certainly billionaires and centimillionaires who reject that lifestyle out of hand (I know I certainly would). The average person doesn’t know their name and they prefer it that way. Even the local billionaire near where I lived for a while was pretty modest, all considered (his kids not so much). I was surprised to see him and his family sit down next to mine at a restaurant one day. Could overhear him talking about the local farmers market and commenting about the tomatoes of the season haha

That's a fair point actually, and perhaps we are only seeing these problems increase recently because "locally optimal" capitalism had historically sort of prevented the global algorithmic optimizations we're seeing now across industries. E.g., rental price fixing via algorithms.

> In my observation/bubble, people already do

Well I thought so too. I match those behaviors, and I don't even watch television. But then I worked at a tech company where I could see the actual data on consumer preferences and behaviors, and it's fairly undeniable: most people aren't like you, me, or the average commenter on Hacker News.

Do most people really want an internet without ads, flights with large seats and plenty of space, high-quality local food — or do most people just say they want that? Because when push comes to shove and these options temporarily become available for some reason (e.g., a new farmers market, a premium streaming plan that removes ads, etc.), most people don't spring for the higher quality option. The cheapest option still seems to consistently win out overall.

I'm certainly not saying "blame the consumer", but if people really don't like ads so much (to the extent that they stop clicking on them), really dislike the subpar streaming services so much (to the extent that they unsubscribe) — then why haven't they abandoned these products?

There are other countries where valuing quality seems to be more deeply embedded in the culture, and most people in these countries will reject subpar offerings altogether. I think the U.S. has had a uniquely precipitous fall in this regard — the average person just doesn't seem to care that much. Why this is the case, I'm not sure, but it's not surprising that since Silicon Valley is located in the U.S., the region simply optimizes on whatever (revealed) consumer preferences return the most. Tech companies are certainly not unique in this regard.

Anyone else feel like mathematics is sort of the endgame? I.e., once ML can do it better than humans, that’s basically it?

This concept of a blameless culture reminds me of one time when I was talking to a SWE at Facebook around 2010. I don’t know whether the story is actually true or just folklore, but apparently someone brought down the whole site on accident once, and it was pretty obvious who did it.

Zuckerberg was in the office and walked up to the guy and said something along the lines of “Just so you are aware, it would probably take a lifetime or more to recoup the revenue lost during that outage. But we don’t assign blame during these sorts of events, so let’s just consider it an expensive learning opportunity to redesign the system so it can’t happen again.”

Related to this problem is the interesting tradeoff between fairness and harm minimization. The idea of fairness is that no individual should have a higher probability of a guilty verdict or punishment than any other individual due to factors that are outside of their control. But there is an inherent conflict between fairness and our ability to reduce harm that results from crime.

For example, consider two hypothetical but identical individuals: one born into a low-income neighborhood and one born into a high-income neighborhood. If you develop a model to predict what we currently categorize as "crime" (the definition of which is its own separate issue), you will find that the income of a neighborhood is inversely correlated with the density of crime. If this is the only factor in your predictive model, then you will more effectively reduce crime by directing attention toward the low-income neighborhood. But now there is an inherent unfairness, because the additional scrutiny toward the low-income neighborhood means that individual 1 is more likely to be caught for a crime than individual 2, despite both individuals having an equal likelihood of committing a crime. This also creates a self-reinforcing situation where having more statistics on the low-income subset of the population now allows you to improve your predictive model even further by using additional variables that are only relevant to that subset of the population, meanwhile neglecting other variables that would be relevant to predicting crime in high-income neighborhoods. Repeat this process a few times and soon you have a massive amount of unfairness in society.

It's probably impossible to eliminate all unfairness while still maintaining any sort of ability to control crime, but what is the appropriate threshold for this tradeoff?

I work at one of the well-known tech companies. We have recently gone on a consultant hiring spree after previously having avoided external contractors for years.

With a team of five consultants and their manager, months of useless hour long meetings to answer simple questions that can be looked up in the documentation, and plenty of slideshows and design docs that are mostly filled with fluff, they’ve produced... a dashboard that I could have put together in about 30 minutes.

Whatever social purpose this contract fulfills is a total mystery to me, but all I can think of is what a waste of human time this is.

> understand the inner workings of an organization at, often, a senior management level, was one of the most lucrative lessons of my professional life

What is an example of something you learned about senior management?

I've always wondered why we don't see more people with $50M-$100M saved up over a lifetime. The average person can't do it, but there are plenty of doctor/lawyer or tech manager couples in the Bay Area or NYC that can save $100k-$150k per year. Max out retirement accounts, invest in index funds, assume 7% return, assume a long life of 90 years, and... that ends up being a lot of money.

Not saying that couples who are capable of saving that much per year are actually doing it, but rather, you would think it would at least be more common than it is.

I suddenly became afraid of flying when I started working for one of the companies known for having some of the best software engineers in the world and seeing how many bugs made it into prod. I can't imagine who is programming the flight computers for Boeing and what they're being paid. It's gotten bad enough that I refuse to fly on any plane that was released in the last decade. At least the older planes have some decent trip statistics to back up their safety.

I worry about this. My wife and I had an agreement before we got married that if I got a good job offer in tech, I could take it, but we would live three years on the west coast max before moving back to be closer to family.

Well, I lucked out and did get a great job, and so we moved to the Bay Area a few years ago. Right when I was trying to figure out what to do since my three years would be up, the pandemic occurred and my company announced permanent remote work. With a new baby and wildfire smoke everywhere, we decided to move back six months early. I took my job with me, and we've been in the southeast since last fall.

I absolutely hate being back here. There's no one in my field or industry, I don't fit in with the culture of this place, and I miss almost everything about the Bay Area. We currently live in a little tourist town at the moment which is sort of nice, but my wife doesn't like it and wants to move again back to her hometown (which has neither jobs nor anything interesting to do).

So I guess I'll sit in a little room by myself on Zoom all day with coworkers 2,200 miles away, make 8x the median household income, and do nothing but invest it while living in the middle of nowhere for the next 20 years and despising every minute of it. At this point I'm just waiting until my kid is 18 and then I'll buy a tiny second home somewhere that I actually like and just travel there by myself occasionally I guess.

A lot of this list just reads that they have coworkers who are jerks. Not interacting with cleaning staff? That seems like a personal problem. And I’ve seen plenty of frugal VPs that drive 15 year old cars and make their own lunch.

That said, I was also pretty horrified once when I was waiting in the lunch line and heard the person in front of me complaining that their six month $100k bonus wasn’t as high as they thought it would be right as the person making their plate of free food handed it over to them. There’s always going to be at least some rude, thoughtless people; the best thing you can do is to try to avoid hiring them.

I don’t know anything about Indian culture; could someone who is from there shed some light on how prevalent the prejudiced mindset is? E.g., do 80% of people think the caste system is stupid and outdated whereas the other 20% still hold some prejudice toward Dalits? What about Indians that have moved to the U.S.? And is this issue improving quickly or slowly?