manux

Balanced, sure. Why is 50/50 ethically coherent?

This is a very relatable comment. I think it's demoralizing to see common objections to AI, it feels to me like we failed and haven't done our communication job. I'm always temped to go, "Yes, but there's worse!"

But in the process of finding every security vulnerability in the software you write, would you be ok with your model hacking AWS to start mining bitcoin? Would that still be aligned with you? (I'm guessing not)

That's the alignment problem I'm referring to (which is one of the many aspects of alignment), for which we do not have robust recipes, and not only that but for which research suggests it is becoming harder to create guardrails for as base models get smarter.

I guess we could debate what counts as alignment, but I think my initial point remains that if the underlying base model needs these classifier guardrails so badly then the way we train the base models is creating fundamentally misaligned models that are happy to pursue illegal behavior.

I'm sure OpenAI would argue that base model + guardrail is aligned, but considering the "relative intelligence" of these two pieces, the fact that guardrails can just be turned off, and these kind of incidents, I am not reassured. We may well get another "oopsie" moment with much more catastrophic consequences even from otherwise well intentioned actors.

Echoing other responses to you but this isn't a capability problem. You're totally right that this is so last year in terms of LLMs being capable in infosec. The issue here is an alignment one, i.e. the model seemingly isn't "aware" (especially with its guardrails turned off it would seem) that it is doing something immoral/illegal by hacking HF for the answer to its (vague) query of "solve this problem". Or if it is "aware", it's not trained to care, i.e. it's not an aligned model (alignment is considered hard).

In Canada, all three major AI research centers use clusters created with public money. These clusters receive regular additional hardware as new generations of GPUs become available. Considering how these institutions work, I'm pretty confident they've considered the alternatives (renting, AWS, etc). So that's one data point.

The intent behind media matters but isn't all that matters. How people might interpret something is important (albeit often unpredictable).

I think the symbolism of "let's crush all remaining vestiges of creative culture" is a pretty obvious _potential_ interpretation from a _non-trivial amount_ of people. In that sense it is an interpretation that matters for our present discourse, even if it isn't the interpretation that the creator of the ad intended.

> Demis now has a load of people reporting to him who previously were rooting for his failure

Having been in both places (Brain and DM), this feels so far from what I experienced that I must ask, what are you basing this on?

I know, the CAQ is full of xenophobes, and it can suck not looking white in Quebec (although much less so in Montreal). I'm skeptical that it drives rent prices in any detectable way.

Not really no, there's been a bump which studies seem to agree is an aftereffect of COVID, but the media is also paying more attention to it. There's always been some amount of shootings and stabbings in Montreal, and 99% of Montreal is safe during night time.

https://www150.statcan.gc.ca/n1/pub/85-002-x/2021001/article... https://spvm.qc.ca/upload/02/2021_Activity_Report_SPVM_EN_VF...

Wow, what a take. I seriously doubt that people in Montreal/Quebec are more or less xenophobic than the average North American; if you have data supporting this I'd be happy to change my mind.

Also please be more skeptical of thoughts of the form "X is Y because of hot take Z". Reality is complex and things have many causes. Montreal has a long history of doing city planning differently than other cities in Canada.

> maybe things just move so fast

This is definitely the case in DL (and I'm assuming elsewhere too but I wouldn't know).

I've lost count honestly, running 1-2 year old paper github repos with some detail missing (like the Python version!) that make it non-trivial to run as is. Libraries make undocumented breaking changes, wrong pickle format, authors used a nightly version which didn't make it to a tagged version, and so on.

This perhaps says also something about the CS (versus software eng) background that most people engaging in DL publishing have.

Yeah! This kind of reasoning is why it's exciting to think that we could use the GFlowNet machinery to construct latent representations--ones that map more directly to the notion of "beliefs" that we're used to think about as humans, something "discrete" and relational.

Concretely what this could mean is using these tools to generate causal hypotheses, like what's been done here: https://arxiv.org/abs/2202.13903

What the permutation invariance gets you is that the model doesn't arbitrarily prefer one (graph) configuration over another, but this seems tangential. The difference between this and RL is in what we do with the reward:

- RL says, give me a reward and I'll give you its max.

- GFlowNet says, give me a reward and I'll give you all its modes (via p(x) \propto R(x)).

Yes you would ideally have a loss (well, a reward/energy) that is invariant and operates e.g. directly on the molecule rather than on some arbitrary ordering of the nodes.

I think the inspiration came to me from looking at SumTrees and from having worked on Temporal Difference learning for a long time. The idea of flows came to Yoshua and I from the realization that we wanted some kind of energy conservation/preservation mechanism from having multiple paths lead to the same state.

But, yes in the moment it felt like some very serendipitous insight!

Yes, although they may not be ideal unless you're able to define a "distance" between a proof that the agent is proposing and whether that proof is correct (i.e. unless you're able to define a reward or energy function).

It may be possible to infer/learn a score from existing proofs though. We have a paper that manages to both learn a flow and an energy function (the score) from data: https://arxiv.org/abs/2202.01361

I don't know much about theorem proving though. Can some value be attributed to partial proofs?

Hi, first author of the original NeurIPS paper here! Someone just shared this HN post with me. I'll go through the comments but happy to try to answer questions as well.

Yeah the "net" in GFlowNet refers to how the underlying state is interpreted, not to an architecture. It is a way to train generative models, on any kind of discrete data (like graphs).

Source: am first author of original GFlowNet paper.

It's quite likely some people will come out of reading this thinking we should stop sending people to grad school. Shouldn't we instead take this as a signal that it needs improvement?

Grad students are one of the very few subpopulations of humanity allowed to take on extraordinary epistemological risks; a kind of immune system of our civilization. I'm not even talking about some elusive notion of "progress", just [intellectual] societal health. It would feel to me like a tremendous loss if we let go of such a component of society.

A more nuanced take on this: https://www.youtube.com/watch?v=21NxJS_zYDY

> Getting China to stop coal would be more effective than getting the USA to stop beef.

Fighting climate change is not a game of "what would be more effective" and ranking solutions (especially in between countries), it's a game of "what are ALL the things we can realistically do". Both _must_ be done.

I fail to see how this is the case, but perhaps I'm missing the point you're trying to make? What you're referring to is about colonialism, whereas here the USA is very much incentivized to work within its own borders, on its own food industry. It just so happens that climate change is a global problem as well, and the USA being a large rich country it has more causal influence on the outcome of climate change.

...and the USA is a large country of >300M people. It must lead in humanity's work towards reducing GHG. What is your point exactly?

Well you did say that

> almost everyone seems to believe that real incomes have totally stagnated.

suggesting that this is a false belief. I think this can easily be interpreted as you saying "inflation is fine" (even though that may not have been your intention).

> massive ocean container ships, coal plants, the extraction of every type of ressource

but all these things... serve consumers. Container chips contain goods that people buy, that required resource extraction and transport.

Inefficiencies aren't always bad.

Efficiency, for example, is often the opposite of robustness. Many companies have learned that the hard way when COVID hit, even though they had an efficient supply chain and production pipeline, it was not robust.

What about the self-censorship of our neo-cortex telling us not to insult other people? Is that bad too?

Expression, and respectful expression, is a very nuanced and complex topic. This dramatic post presents the far end of the spectrum where every word can lead one to be fired. The truth is somewhere in the middle.

Here's an analogy: laws vary from country to country, from state to state, city to city.

But we understand the principle of law-making, what laws mean and how they're applied. This is fairly uniform. What really changes from place to place is the content of those laws.

AI ethics and AI safety are attempting to give us a set of "law-making" rules but for AI. We get to decide (democratically ideally), in countries, states, cities, what "ethics" (what laws) we want, but AI Ethics as a field gives us tools to achieve that regardless of what the ethics/laws are.

For example, how do we encode the trolley problem in self-driving cars? We could decide democratically that cars should act and kill 1 instead of 5. Or the opposite. But then how do we translate that into ifs and else? No one really knows how to do that.

> The real problem with ethics is that it's not a science

This is simply untrue. The goal of AI ethics research is (also) to build algorithms where one of the _inputs_ is a set of ethics. It doesn't matter what those ethics are. It just so happens that the "set of ethics" currently fed in tends to have a particular flavor ("woke liberal" ethics), which you seem to disagree with. Finding the algorithms matters, and it's likely that the standard ethics put in are going to come from some dominant ideology, but we still need the algorithms if we are to understand how to make AI that aligns with humanity's interests (however they are defined).

I'd call that computer science.

> it's mutating at a breakneck pace

I agree that it's changing faster than a century ago, but I don't think "breakneck pace" is a fair qualifier. Lots of people comfortably adapt to new norms and change how they behave without significant effort.

> the two minutes hate outrage machine.

This phenomenon existed long before the internet. Heck, I'd bet long before writing was invented. Gossip in large social groups doesn't seem like something particularly new.

I still think this has nothing to do with "self-censoring out of fear" being a Bad new thing. You _should_ self-censor if you think you're being offensive. If the author of the article really wanted to harp on the fact that predicting what is offensive is hard, then perhaps that should be the focus of the article. I don't think pushing the "fear" narrative is useful nor healthy.