datsci_est_2015

How something is proven is often more important than what is being proven. There are underlying systems and patterns that, when understood properly, improve our model of mathematical (or physical) reality.

With convoluted and inelegant proofs, AI may fail to uncover those systems and patterns. As a most concrete example, it may fail to recognize some problems as isomorphic to other problems. Brute force solutions are a depth-first search.

To improve human mathematical understanding, AI is probably best used as a “copilot” (lol) rather than a black box oracle, like these AI companies appear to be doing.

For an example, read about the cotton gin.

The original incident was probably an honest mistake as well.

Still getting Internal Server Error despite githubstatus claiming “All Systems Operational” and the incident as “Resolved”.

Premature declaration of Resolution is pretty bad user experience, if I’m being honest…

You’re right, I was responding more to the sentiment of the overall thread than your comment in particular. Mea culpa.

> but it's not because Russia sent malnourished troops on donkeyback and lost 27 million men that it "did more" to beat the nazis.

I mean, if we’re splitting hairs, I would argue they “sacrificed more”, if they did not “do more”.

This and other comments on the thread are pointing out what I’m assuming every History major and PhD is deeply aware of, and that is that the narrative of historical events is core to our understanding of them.

But the reason that we’ve come together to talk about this is that LLMs and the poisoning of their data corpus presents a unique challenge to historians that their predecessors did not have to face. Although, their predecessors did have to deal with other unpleasant things like libraries being burned and censorship.

Would disagree. It’s not Scott’s conclusion I’m referencing, it’s his thought experiment: “What if there was a bottomless pit of suffering that no matter how much you threw into it, there would be no meaningful progress into reducing that suffering?”

It’s a thought experiment that gets to the core of why fundamentalist utilitarianism (like you’ll often find in SV rationalist communities, especially AI fanatics) is an unbalanced approach to morals and ethics.

Utilitarians can declare that they believe that there is a proverbial universal panacea, and then work backwards to justify within their moral framework literally any action that they take that could be construed as work towards that panacea.

But what if no matter how much effort they put into achieving that panacea, they never make meaningful progress? And all the while they’ve massively increased the amount of suffering that people who they are surrounded by?

As an absurd example, imagine a utilitarian who’s convinced that if we sacrifice infants to the Gods that we’ll get to an AI singularity faster.

> There is a state of human development that the majority of people would agree approaches a utopia, which is a tangible and achievable goal.

The Star Trek vs. The Expanse meme is relevant here. There’s no guarantee that there is a continued reduction of suffering in the future, especially due to technological progress. There may be technology that reduces the suffering only for a few select elites while the rest of humanity lives in destitution, unable to meaningfully revolt.

This is some serious touch-grass territory you're treading in. Might I introduce you to the Bottomless Pit of Suffering[1] thought experiment of utilitarianism? Because it's incredibly relevant to the arguments you're presenting.

[1] https://readscottalexander.com/posts/ssc-bottomless-pits-of-...

Disclaimer: I don't generally hold Silicon Valley rationalists in high regard, but this particular piece of writing has always stuck with me quite deeply, as someone who used to be superficially interested in utilitarianism.

I was always a fan of what a former senior colleague ingrained in me early in my career:

  - a model is primarily judged by its predictive power

A model that has no predictive utility is a bad model, but every model hits its limits in terms of predictive power. Some models have a very steep cliff in accuracy at the "edges", and some gradually lose fidelity like an out-of-focus photograph.

Most anti-science criticism I see is completely ignorant of what it really means to model a system or phenomenon: the challenges, the limitations, the end goal, the criteria for success, etc. It's sort of a statistical / scientific ignorance that's deeply cross-discipline. To perform science in a lot of ways is to engage with what it means to build a model, and in some ways, interrogate the universe.

If you're both an idiot and overconfident, it's tempting to throw your hands up and not engage with something as complex as "modeling", and just decide almost out of whimsy what the universe should truly be. But at least some people who find modeling too intellectually taxing at least admit that there are things that they'll never understand.

Expanding on this thought (too late for an edit), there's a nice simile to be had:

  - technologists : software that is bad for privacy
  - nutritionists : food and drink that gives you diabetes

So, I'm no nutritionist, but I pay attention enough to pay attention when they say I shouldn't drink a liter of Coke every day. As technologists, we should carry the torch and speak loud-and-wide about the dangers of software that is bad for privacy.

> The thing is: people quickly discover that even mediocre LLM-code with a mediocre LLM-review, is still better quality than what they hand wrote and hand-reviewed.

No, I find that most teams I've been on have 5% to 20% of their most artisan engineers actually giving a shit about the code, and gently coaching the other 95% to 80% of the engineers who are usually much more junior during code review to not push unmaintainable slop. If you're on a team that doesn't still have someone giving a shit, it's not a sign that there are no teams that don't give a shit, it's that you're on a shit team.

There are also teams where the incentive is to get to pull request (or merge request) as soon as possible, so that you can move the ticket across the board and relieve downward pressure. That usually means pushing subpar code to PR that you know you'll spend another few hours fixing. EDIT: I bring this up because this can bias one's belief that humans produce poor quality code, when you're usually looking at the first draft that was incentivized to be pushed out "too early".

Anyway, if you're on a shit team, but you're still satisfying product and nothing is blowing up yet, there's a pretty good sign that your software has no moat and will be the first to be replaced by LLM-generated software on-demand. This is probably where most niche B2B back-office, front-office software sits. Stuff that sits somewhere in the realm of "We could probably do this in Excel if we wanted to."

Imagine we were talking about soda (or pop, or coke, or soft drink, depending on your dialect). Personally, I find it absolute insanity to be regularly consuming so much dissolved sugar especially during our ongoing obesity and digestive health epidemic.

But, “most consumers” don’t care about that. The beverage industry is insanely profitable, especially with recent forays into “energy drinks”.

Do people who drink so much dissolved sugar live without consequences? Absolutely not. They have a significantly lower quality of life and die much sooner than people who do not drink dissolved sugar.

I don’t see your comment as a valid dismissal. Just one more way that individual outcomes in our society are increasingly K-shaped.

The brazen disregard for safety is funny in this context, if a little frighteningly indicative about the toxic attitudes that permeate Silicon Valley at the moment.

Anyway, even if we froze training right now LLMs in their current state would have the ability to continue to provide tremendous utility. We’re not asking frontier model providers to delete their previous generations of LLMs (and harnesses), we’re trying to hold them accountable for the crimes they’re committing while training the next generation of LLMs (and harnesses).

The API surface is what’s exposed, not just what’s documented.

This is probably an even more important principle in the age of LLMs.

It’s their hardware and their web response. Especially more heinous if it’s being served as part of a subscription. I don’t think it’s functionally the same as me opening Microsoft Paint and recreating pixel-for-pixel a New Yorker artist’s signature.

Realizing now that “the banality of evil” was right there and I failed to recognize its relevance.

Frog reports water is still comfortable, though no longer lukewarm.

Something that is nearly completely harmless though inconvenient, yet also illustrative of the short-sightedness of such a ruling would have been so satisfying to witness. But the Democratic Party is clinging to the ideals of “norms” and “conventions” and “tradition” that Trumpism has set fire to.

Indeed. Highly correlated (but not strictly) with Great Man syndrome, and Elon Musk fanboyism.

This is why we need the humanities. I remember one of my application essays to my university was on the merits of a “Liberal Education”[1] and I’m very grateful that I was able to properly receive one from my institution.

[1] https://en.wikipedia.org/wiki/Liberal_education

“Seems to be innocuous” is genuinely funny. It must be true for almost all types of corruption and evil that if you zoom in enough, and rob the action of all surrounding context, that it could be described as “innocuous”. How aptly technocratic.

It’s so interesting watch the upvotes trend up and down over the course of the day. People say California is liberal but can’t help but notice how the downvotes flow for sentiments like these as the west coast wakes up and Europe goes to bed. (Maxed at +5, now sitting at +2)

I’ll add one to the pile:

“Socialize the costs, privatize the profits.”

If you think my grievances begin and end at “data centers” and “AI” you are sorely mistaken.

> And it will be good enough for 90% of use cases.

I think it’ll just raise the automation bar that much higher. In other words, it will be good enough for 90% of yesterday’s use-cases. But tomorrow’s? I’m bearish.

I imagine the amount of onerous bureaucracy when interacting with any entity is going to explode if they expect you to do it in an automated fashion, for both private companies and the government. Especially as roles are cut because they can be “automated”. Except only 80% of the workload can be reliably automated, the other 20% requires a long tail of effort.

Scott Bessent has had several foot-in-mouth gaffes through his tenure, I would not put much weight on his words in this case, especially if they contradict his handler.

> I could be convinced of your position if you can name the common elements that justify the umbrella. It becomes even stronger if those common elements are the rotten ones.

I would say our business and media cultures are most homogeneous. No matter where you are in the US, a business conference feels the same. A meeting with clients feels the same. The same toxic personalities rise to the top. I’ve never been surprised by a director or CEO.

No matter where you are in the US, you turn on the TV, and even if it’s local news, you get the same predictable messaging. We’re all obsessed with appearance and status, even within our subcultures and across the political spectrum. Superficiality.

Sure, personally I think Krugman is a bit abrasive and political when he could be a bit more refined in his delivery. But, then again he probably wouldn’t get the clicks and the traction he’s gotten over his career.

But most people I know who don’t like Krugman are generally against monetary policy, as he’s one of the biggest mainstream proponents.

Why do you think the IRS is structured the way it is? Zuck and his cohort regularly contribute to defang the IRS. Or did you miss the part where he donated to the president and then the IRS became significantly less capable of going after the largest tax avoiders?

> This is not about Zuck it's about the IRS and probable political interference.

This is not about Zuck, it's just about the probable political interference that Zuck funded.

Important context to add is that those who benefit from such tax breaks turn around and fund with their disproportionate, ill-gotten gains the very candidates who you say we should really be upset with. They also fund the media companies, both social and traditional, that are the propaganda for the status quo. They don't benefit from these laws that create loopholes in a vacuum - they're the very reason the laws continue to have loopholes in the first place!

As they say, "No war except class war", I think we are perfectly justified in being upset with the names in the headlines.