igorkraw

Nah, it's much simpler, the models aren't reliably able to recall the correct rule from memory - it's im the training set for sure.

This is another specialized synthetic data generation pipeline for a curriculum for one particular algorithm cluster to be encoded into the weights, not more not less. They even mention quality control still beim important

Please make sure aider and llm-cli can use this soon,kthx :-)

First point:I know people who had their houses trashed by card carrying Neonazis for being trans or migrant looking, who had their PII abused by Nazi cops (again card carrying) who shared access illegally, and I know people whose grandparents died in the resistance or camps. I've seen the Concentration camps as part of my travels when I was younger.

This isn't theoretical, this is learning from past mistakes. Only people who think themselves beyond this would be this dismissive.

2nd point: if you want to make a very specific point about coasian dynamics, freedom of business and liberal commerce, you need to be more specific, on the extremely broad strokes the new deal, china, the post war recovery in Europe, the well regulated energy markets vs unregulated monopolies etc. are all empirical counters to your vibes. I made these intentionally as broad strokes as you did, feel free to debunk them, but as an honest intellectal you'd then also find the limits to your claim and add nuance to it. Only Rand fanfiction takes the "job provider" literally

In the beginning of the PhD, to help with rent I contracted to help develop computer vision algorithms in this field, only PoCs, never got very far.

And interesting thing is that the lice apparently evolve super fast, including getting translucent and resistant against poison

While I do think gp should give you an example, I also invite you to provide a counter example, otherwise both of you are just stating your vibes.

If you are _actually_ willing to discuss, you can't just demand the other side to give, you can also set the standard by giving.

In the spirit of HN rules, actual answering instead of snark:

1. The memory of Nazis using centralised industrial might and information to kill millions of people (google dutch insurance records Nazis)

2.a much stronger history of workers rights and distrust of rich people together with a different attitude to gouvernement making it politically challenging

Can hate it or love it, but the median life expectancy in Europe is much less correlated with wealth and _I think_ is higher from memory, child hunger and food insecurity is much lower and homelessness and other corporate abuse is much less of a problem corrected for wealth and population.

Whether we manage to keep this remains to be seen, but I think it's a reasonable set of different preferences

your profile says >Hit me up if you want to collaborate on NLP research

but doesn't hint on how, check _my_ profile for hints on how :-p

A few technical questions (I had a somewhat related work with friends here https://openreview.net/forum?id=I3HCE7Ro78H although we focused on gradient multiplicity in adversarial training, not massively parallel training)

1. Do you think this is a form of variance reduction or more a form of curriculum (focus first on the bulk, then on remaining errors)? 2. Did you observe any overfitting/additional adversarial risk? 3. Did you try this on just single-node minibatches as well? How did that perform?

GNNs are useful at least in one case, when your data a set of atoms that define your datum through their interactions, specifically a set that is that is high cardinality (so you can't YOLO it with attention) with some notion of neighbourhood (i.e. geometry) within your set (defined by the interactions) which if maintained makes the data permutation equivariant, BUT you can't find a meaningful way to represent that geometry implicitly (for example because it changes between samples) => you YOLO it by just passing the neighourhood/interaction structure in as an input.

In almost every other case, you can exploit additional structure to be more efficient (can you define an order? sequence model. is it euclidean/riemanian? CNN or manifold aware models. no need to have global state? pointcloud networks. you have an explicit hierarchy? Unet version of your underlying modality. etc)

The reason why I find GNNs cool is that 1) they encode the very notion of _relations_ and 2) they have a very nice relationship to completely general discretized differential equations, which as a complex systems/dynamical systems guy is cool (but if you can specialize, there's again easier ways)

The whole point of GNNs is that they generalize to arbitrary topologies by explicitly conditioning the idea of "neighbours" on the graph specifying the topology. Graph layout has been tried here https://github.com/limbo018/DREAMPlace to great fanfare although there is recent drama about it https://www.semanticscholar.org/paper/The-False-Dawn%3A-Reev... . Graph transformations are a thing as well https://arxiv.org/abs/2012.01470 but it's a tricky problem because you implicitly need to solve the graph matching problem

can one preorder your book?

The author looks at "observables" of performance without considering whether there might be confounders such as those discussed in great nuance here https://onlinelibrary.wiley.com/doi/full/10.1111/joes.12328 .

He cites similar work by William Shockley who taught both electrical engineering and scientific racism at Stanford https://en.wikipedia.org/wiki/William_Shockley (no swipe at the author, just pointing at the biased motiviations of some of the researchers foundational to the idea of "high performers").

In general, when you see pareto structures or power laws, you should think of compound or cascade effects, which in human structures generally means some form of social mediation. Affinity for a desireable skill might be gaussian, but the selection process means that the people who _get_ to do that skill might become pareto shaped because if you aren't much better than the next guy, you wouldn't stably stay at the top. Similar logic can hold for other expressions.

In general, I wish more people would read https://blackwells.co.uk/bookshop/product/Causality-by-Judea... or at least the more accessible https://mixtape.scunning.com/ before starting to conjecture from data about social systems - the math will tell you what you can and cannot speculate on.

(fun exercise: draw the causal models of IQ in https://dagitty.net/ and ponder the results)

...by what theory of justice is CO2 per land area just? You didn't will your birthplace.

I replied to the high birthrate fallacy/dogwhistle in another comment in this thread.

...are you attacking the babyboomers parents?

And more seriously the argument is that if we on the lottery and our ancestors did the damage before we understood it, then yes, we don't get to keep the benefits from other people and have _them_ foot the bill of avoiding further damage via stifled growth. That social contract doesn't work.

Nevermind that _right now_ per capital emissions and pollution and leads to similar policy implications. It's not just our ancestors overconsuming , it's us

TFR is total fertility rate. GP is saying "the problem is not the overconsumption in the rich countries with low TFR, it's the poor countries with high TFR getting richer and consuming more and people migrating to countries where they consume more".

Which falls apart looking at the map of highest ghg emissions https://ourworldindata.org/grapher/per-capita-ghg-emissions vs the negative impact caused by climate change and the ability to mitigate https://gain-new.crc.nd.edu/ranking .

curtailing migration without massive reduction in the overconsumption by the rich countries amounts to having the poorer nations pay bills (via enduring disasters) for damage they didn't cause, which I find an injustice. Doing that and limiting economic growth and thus limiting the capability of dealing with the disasters would just be cruelty.

If you want to reduce migration without injustice or cruelty, you need to make sure other countries remain livable and thrive, hence the OPs hook.

If you accept t sne or umap as being useful because it unveils some structure and pacmap has mathematical guarantees to preserve the same or more structure, is that not enough as a motivation?

Fwiw, I use pacmap when building pipelines to get a feel whether a model is capturing signals as expected, for which it works better than the two due to the structure preserving making the conceptual mapping easier

I did my PhD on trying to use ML for EDA (de novo design/topology generation, because deepmind was doing placement and I was not gonna compete with them as a single EE grad who self taught ML/optimization theory during the PhD).

In my opinion, part of the problem i that training data is scarce (real world designs are literally called "IP" in the industry after all...), but more than that, circuit design is basically program synthesis, which means it's _hard_. Even if you try to be clever, dealing with graphs and designing discrete objects involves many APX-hard/APX-complete problems, which is _FUN_ on the one had, but also means it's tricky to just scale through, if the object you are trying to do is a design that can cost millions if there's a bug...

Full disclosure: my employers competition, but I think the results are cool.

Added "results" to the title to be clearer that this is not ongoing, hope that level of editorializing is okay, I tried to match the "blandness" level.

Context here: https://design.adaptyvbio.com/

And some cool writeups and discussion around the competitoin here:

- http://blog.booleanbiotech.com/adaptyv-binder-design-competi...

- https://x.com/pranamanam/status/1837702358995812754?s=19

Group theory is just a way to think about what properties a function needs to have under specific actions on the data. If you want to train a speech network that should be peak-amplitude invariant (to make a random example) you can normalise the amplitudes, modify your loss to be invariant to it or modify the network output to be invariant to it. These might have different numerical tradeoffs (e.g. one of the reasons why people use equivariant architectures with a final invariant aggregator is that it allows each layer to use and propagate more information, and one the reasons why graph neural networks are a thing is because we don't always have a canonical labeling).

All the stuff you mentioned is true, but thinking about it in an abstract sense, that means there's a set of universal symmetries and a set of highly context depending symmetries, and group theory is afaik our best method of thinking about them rigorously - as I say in the child comment, not the end point, but our current best starting point (in my opinion)

It's not the right one on its own, but it's the closest we have right now to talk about things like "high level equivalence" rigourously. Like, one way to talk about logic is that it's e.g. invariant under double negation - so any compression algorithm that does "semantically lossless compression" can exploit that property. Deep learning then becomes an algorithm to find features in the data which remain invariant under irrelevant transformations, and representations useful for semantic similarity could then be formulated as a very big and complex composite group, potentially conditional... there's probably a new theory of hierarchal similarity and irreducible collisions/errors required, but as of now, group theory, approximate equivariance and category theory are the bleeding edge of thinking about this rigorously while still having sota benchmarks/real world applicability to my knowledge

Happy to discuss more via email as well

This part

>What's harder to deal with from a measurement perspective is sematic equivalence. calling some kinds of errors zero-cost, but not having a great way to categorize what the loss of, exactly. But it's kinda what you want for really extreme compression: the content is equivalent at a high level, but may be a very different byte stream.

Is basically saying

> What's harder is defining a reconstruction process in terms of a "semantic group" i.e. an output encoding and associated group actions under which the loss is invariant, and having the group actions express the concept of two non-identical outputs being"equivalent for the purposes of downstream processing".

Taco Cohen is one of the pioneers of this line of research, and invariant, equivariant and approximately iv/ev architectures are a big thing in scientific and small data ml

Group theory is the abstraction you want

1. I agree that anarchists probably need to grapple with the fact that people think of this as a first association...

2. ...but terminology does not at all fail here, there is a separate term https://en.wikipedia.org/wiki/Anomie which describes a breakdown in order and social function.

Notably, anarchists would probably claim that the fact that there's a civil war of various warlord factions and wannabe-states going on https://en.wikipedia.org/wiki/Sudanese_civil_war_(2023%E2%80...

is a good example why anomie and anarchy probably _should_ be distinct concepts, as there's probably plenty of hierarchy, localized state power and centralized decision making going on in Sudan, while e.g. https://en.wikipedia.org/wiki/Zapatista_Army_of_National_Lib... appear to have done absolutely fine keeping and improving a social fabric for the last 30 yeras (until recently, gangs arrived and brought the anomie that comes with organized crime).

Again, I don't think it's a winning strategy for political anarchists to try and convince people that "acshually it's anomie, not anarchy", but I think on this page, peoples professed self-identity makes sharing out this separation of concept worth it

Just wanted to say, your ADHD wiki is part of what made me get a diagnosis in year 7 of a 4 year PhD, the meds of which then allowed me to finish it successfully.

I've since used it to show 6 fellow high functioning AuDHD people, 2 of which were getting into the depression and anxiety spirals that can come with the Systems failing.

Just in my corner of the world, you saved the dreams of at 3 people with your wiki, I would call it slightly more than moderately successful. Thank you so much.

It should be noted that

1. this is nothing that should surprise anyone who has an intuition on control theory and the evolution of unconstrained markov chains

2. there appear to be relatively easy mitigations https://news.ycombinator.com/item?id=41061085 (made a separate post because it might be of independent interest to discuss)

3. you still won't get beyond the imititation game boundary without exploration & feedback, i.e. the recursive improvement doomers are, as of now, still wrong

Title rewritten from the too long "Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data" , figured this might be of interest given https://news.ycombinator.com/item?id=41058194 and https://news.ycombinator.com/item?id=41058867

I used to do the "it's not a real nobel" take as well when I was young and edgy, and then I realized that Alfred Nobel was just another rich guy buying his way post-mortem prestige*, not even doing so during his life. Even if "most of the applications were civil", with the economic devastation it brought and the undeniable military uses, I'm not sure the "merchant of death" moniker was unearned.

So by now, I think that the Sveriges riksbanks pris might have "better pedigree" and that Alfred Nobel is honored by the dedication, especially given the stellar research that it is often recognizing

PhilosophyTube aka Abigail Thorne has a take on this in her "what's wrong with royalty" video: If people associate prestige with something they can identify with, then the prestige can "rub off" on them in their perception, they can claim to be part of it. E.g. Britishness being tied to the royal family and that being inherently more refined than more base communities.

For Musk, he's a white, rich tech bro who explicitly announces his "alpha" status https://nypost.com/2022/05/23/elon-musk-told-first-wife-at-t... (which I'll let you read between the lines yourself on) - a lot of white wanna be rich tech bros will identify with him.

And that's not even going into the whole "If I lick his boots enough senpai might notice me" dynamic

As soon as you have a form of editorial control ( which includes decisions about what to highlight and where to place it https://en.m.wikipedia.org/wiki/Editing ) you seize being a platform and become something else. One could argue that maybe twitter could become a platform by removing any form of recommendation engine (including likes) and simply acting as an aggregator of reverse chronological feeds, but then what would the business case be?

Social media has been eating up advertising by skirting in between the lines, acting as if they don't censor and are mere platforms (and thus aren't liable for falsehoods spread via them) while censoring and curating content heavily to ensure advertising cash keeps rolling in.

I think it is time for (and with recent legislation proposals in the EU, am hopeful for) changes that either severely limit the type of advertising and control these wanna he platforms can accept and exert or simply makes them legally for spreading falsehoods and endangering the public via users they choose to jot exert their editorial control over.

Ask the customer to pay collateral when enabling this for a service, then use that to pay for the data storage in the shutdown grace period, reimburse it (or count it against future use) if it was never used. You can also easily add this on all the elastic services,e.g. your price per GB simply goes up a bit for S3 in the first month, until the amount of data you have is fully in your "insurance".

Or, you know, price it in and do your own risk estimates to make this a seemless experience, but AWS got away with not offering anything, so I'm already assuming you want to be passing all the cost to the customer (heck, amazon, if you are reading this, you can even skim a little bit of the "insurance" money! )