MathYouF

I wonder how much of that is a result of the state enforced cultural isolation-

(as a means of decreasing the chances of information and ideas which they actively censure from spreading in the population, ideas which would be likely undermine the current leadership's ability to maintain unilateral power. Just as Russia has experienced with its citizens as they become more connected to others outside of the country, whose individual liberties and relatively meritocratic economic systems they've begun to push for domestically)

-rather than lacking technical merits in what they create or the speed at which they do so.

Hello! If any of the authors are reading this, what is the difference between doing this as opposed to using attention mechanisms between nodes?

This is really intersting.

> In particular, model parallelism is efficient when the amount of computation per neuron activity is high (because the neuron activity is the unit being communicated), while data parallelism is efficient when the amount of computation per weight is high (because the weight is the unit being communicated).

In this case, for fully connected layers, the amount of computation per neuron is high because it is (fully) connected to and from every other neuron in the pervious and following layer, and therefore we want more GPU's to work on those in parallel, so it isn't a time bottleneck?

What does it mean then for to have a high "computation per weight".

>• Convolutional layers cumulatively contain about 90-95% of the computation, about 5% of the parameters, and have large representations. • Fully-connected layers contain about 5-10% of the computation, about 95% of the parameters, and have small representations.

Wouldn't fc's require more computation since each parameter has more incoming and outgoing values?

I don't doubt this is all accurate and I'm just missing some obvious intuition, but for the sake of learning feel free to explain it if you wish!

Just a perspective for other readers:

I recently moved to the Bay Area. I've lived quite a few places around the world and US. Some I consider the nicest in the world (Tokyo) and some I don't consider nice at all.

I'd formed my view of SF based on comments I'd heard like these on HN, and was prepared for the worst when beginning to visit for the weekend.

Having now had several excursions into the city including places considered not that nice like the Tenderloin or Mission, I can't see how it's any worse than any other US city, and in fact consider it nicer than most other large cities in the US I've been to, like NYC and Chicago.

Maybe it's because I only visit on weekends and don't live there, but from the perspective of a frequent visitor it seems really nice to me, and I didn't go in expecting that nor am I a person particularly biased towards having rose colored views of things.

I wonder if maybe people in SF expect it to not have the same problems as other large metros like NYC. I'd be curious to hear from someone who has lived in both and thinks SF is "apocalyptic" what experiences made them think that.

I think in the Bay Area there's a large number of people who don't realize (or have chosen to forget) how inconvenient life can still be on average around the world and even the country, compared to the idyllic circumstances they've created here (especially in the valley). It's what makes it so easy for them to come across as judgemental or tone deaf when discussing problems at the national level.

Yes, you seem to understand everything correctly.

Supporters of an actual dictator for life, who are themselves capable engineers, are manufacturing weapons for a nation which regularly supports other dictators and themselves openly threaten war against democracies.

So until you, or people you know, with moral views and past political affiliations which make you more comfortable, wish to manufacture with equal competence weapons of war, we'll have to take what we can get here to preserve the cause of world democracy in so far as it requires protection from acts of violence.

Please stick to the main points (the necessary production of advanced weapons of war to defend one's chosen government) rather than partisan talking points (ones seemingly ironic political affiliations in relation to a claimed goal) if you want to advance actual mutual understanding of topics on this forum.

The weights are for the ImageNet category only, just FYI.

Which should mean the model is only trained to pair image outputs with the CLIP vector from the very short ImageNet labels, so the quality of intermediate latent space for long form text input would be limited.

If one wanted to use pretrained weights I'd recommend a model trained on LAION-400M instead.

The architecture looks cool though, seems to be a smaller model.

The first such invasion in the last millenia was that of England by the French Normans (themselves having been invaded only a generation before by Danish pagans), where French language was loosely imposed across the lands for several centuries, leading to the English we have today, majorly comprised of French loan words.

It's a combination of the cosmwasm (Rust->WebAssembly:https://github.com/CosmWasm/cosmwasm) library being popular for writing smart contracts, and the Rust specifically being chosen for its great type safety and error tooling, which can help people writing smart contracts to better guarantee the behavior of their smart contracts ahead of time. This is important because once it's deployed it's essentially a bounty program with a direct payment mechanism for the entire value of the funds that the smart contract can manage.

>This prompted the Whitehead (https://wi.mit.edu/) to hire the law firm Hinckley, Allen & Snyder to conduct an investigation

>“While we have not found any evidence that Sabatini discriminates against or fails to support females in his lab, we find that Sabatini’s propensity to praise or gravitate toward those in the lab that mirror his desired personality traits, scientific success, or view of ‘science above all else,’ creates additional obstacles for female lab members,” the report concluded.

Is this implying that the belief of the writers of the report is that "female lab members" struggle more than others to achieve scientific success or to view 'science above all else'?

What other reason would they consider a scientific lab leader favoring those who are committed to science and scientific success to be a hindrance to "female lab members". Or is that entirely relying on the vague first term: "personality traits".

It's so interesting how many people with between above average up to immense intellectual capacity can imagine so little else to do with their intellect than think about money.

I think shallow intellectual pursuits like money maxing for people who are already wealthy and with their needs well beyond met are possibly a reaction to realizing ones own inevitable death and cosmic smallness, diving deep into one of the most shallow aspects of life in order to cognitively distance oneself as much as possible from contemplations that may lead one to ponder existentially disturbing thoughts with no clear answers.

Maybe it's no coincidence this is posted on a Sunday night.

Humans are hugely responsible for the spread of violent propoganda.

Attempting to curtail people's ability to communicate what they already believe isn't a long term solution and only leads to normalising central authorities deciding what ideas can and can't be communicated.

You can't deny though that there is an incentive for some, especially hedge funds with massive crypto positions (who are known for not being above doing nefarious things with foreign government officials), to bribe legislators of countries open to the idea to adopt it, to improve their financial asset. Whether or not that is indeed something they're doing, as I mentioned, is pure speculation.

I'm personally convinced (by no evidence other than game theory, which is what trust in block validators to be honest as a group is based on as well) in cases like these, that there must be people with a huge stake in btc who are essentially bribing legislators to adopt crypto. Why else would they accept as legal tender a currency their government and people don't hold except if the people who do hold it convinced them in some way? Looking forward to the documovies in 15 years of these crypto "investors" flying to CAF and El Salvador and bribing their government officials.

> So far the ruble hasn't been decimated but this could still go south

The ruble was worth 0.14 dollars before the war and now is worth 0.12, which is just below a 0.014 reduction in its value, so it has quite literally and almost exactly been decimated, as it was done in roman tradition.

You're the first to confuse it for sarcasm on this forum. Your tone seems to be presumptuous and uncordial (possibly a learned memetic behaviour from others with similar opinions when asked to elaborate with simple explanations of their view), so I'll take the hint and stop replying.

It was a genuine question, I've not heard an intuitive or simple explanation of the concept. Your assumed combative tone is very reminiscent of others I've heard bring up the concept though (like Gary Marcus). I still haven't been able to hear a good and simple explanation though from your cohort despite my genuine curiosity.

Could you give a simple example of proof that real world since real world intelligence is much more than symbol shuffling? I'm a bit unfamiliar with the idea.

I'm a bit lost on who the 'they' are and what the 'paper' is in this case: The authors of this paper writing this paper, or the authors of SoTA papers writing SoTA papers.

A third option you may have meant which makes sense to me in context is the authors of this paper going on to write SoTA papers. I actually looked through the published works of the first two authors and didn't find any practical work, all similar very long form math notation heavy theoretic papers, which supports what I initially worried about that there's basically two diverging branches of ML papers, one of which I'm skeptical about the practical value of. I have actually not seen any examples of first authors on theory papers going on to publish SoTA results on applications, seeing such a thing I would also consider pretty convincing evidence of the utility of these papers and the symbiotic unification of the two groups.

I wonder if there's been similar concerns raised historically, like at the advent of electricity, between practical engineers focused on creating groundbreaking applications versus people still focused on theory, and in retrospect what contributions continued to be gained by the theoretical work afterwards in those cases.

I think I'd need an author of one of those (SoTA) papers to walk me through how reading part of a 50+ page theory paper helped them achieve any marginal improvement on their path towards productive contributions for me to really understand or believe it. If I could have even a single example explained to me I think I'd get it.

Since this is hacker news I'm sure there's at least a few people who will read this that are the ML research engineers who write papers on their architectures which achieve SoTA results on different benchmarks, or establish new datasets, or publish new tooling like different loss functions/optimizers (which usually are published alongside some new SoTA results achieved by integrating them into a current SoTA architecture).

I've wondered from that crowd (the productive crowd) if they get any value from papers like these, if reading them ever gives any insight. I don't seem to ever glean very much from them, whereas just reading the formerly mentioned papers and thinking for myself and experimenting seems to give me the intuition I personally need.

I'm starting to feel there's a whole secondary field within ML of people who publish 50+ page math-heavy theory only that yields no actual results nor value in achieving those results, but maybe I'm wrong. I'll ask around at the conferences this year. I'd be willing to read papers like these if it turned out it'd help me but as time is limited, I feel I'm likely better off reading more papers that show results and their architectures (which usually have a bit of light speculating/intuiting mixed in as well) than reading this.

Also, no offence meant to the writers of the paper who might read this. If you have any thoughts on this line of inquiry as well I'd be interested to hear them. Maybe there's a lot of practical value I'm missing out on by not taking the time to read and understand papers like these.

~Aside: Looking through your comment history you seem really cool and like we have quite a bit in common, feel free to hit me up over discord!

I witnessed zero crimes committed living for a year in the middle of a 30 million person city (Tokyo) . I also witnessed no yelling, no littering, no fighting in public, no speaking loudly or playing music on public transportation, no public shouting of vulgarity, and no theft. I left my Macbook in a Starbucks unattended for an hour and walked around a mall and it was still there when I came back.

All places and people and cultures are not the same. There are objective measures one can use to see this is true. Take a walk around Manhattan (~11 million) or any other city in the world and try to replicate my results.

Culture matters and has an effect on making better citizens and therefore a better quality of life for everyone. It is not simply a matter of them pretending to have these virtues, they have them and the evidence is reflected both in numbers and ones personal experience if they visit.

For anyone else reading, this does not seem like a qualified or legitimate take on Japanese life from my experience of a year of actually living in Tokyo. Maybe the OP has had equal personal experience and just percieved it differently, but there's almost no chance a Japanese person in Tokyo would dine and dash, and being raised in their culture from birth absolutely does make one less inclined to do things like that.

Post-modernist views that all cultures make people who are equally bad and good and that it has little effect simply don't hold up to even a cursory analysis of available statistics on the regional rates of crime/violence/exploitation worldwide.

At the end of the day we're all monkeys who have been biologically incentivized, since the first cell to ever exist, to survive long enough to pass on our genes.

As Dawkins covers in The Selfish Gene, sometimes this can manifest in abstract ways, like helping pass your genes by supporting your nieces age nephews (or even more abstractly, helping humans in general live on). You haven't mentioned anything in your life that is involved in helping create the next generation of humans.

In fact, you've mainly mentioned your journey has been materialistic, and even your solutions seem related to chemicals. Maybe the game you're even trying to play is what's inherently depressing.

My advice would be to a part of the world where people are suffering and try to help them, that'd likely give you the fulfilment and perspective you need.

Very interesting, thanks for clearing up my conceptions on this. I'd become quite familiar with the massive remittance use case of blockchain in Latin American countries with inflationary currencies and/or currency controls (Venezuela, Argentina) and assumed that Miami had adopted crypto in part because of how many inbound Latin Americans already used it as a medium of exchange.

I thought Miami became the hub of decentralized finance because Miami is the defacto port of entry for wealthy people from Latin America, and some (small but influential) percentage of those wealthy are trying to covertly extract their wealth from their home country into the U.S. (why they need to extract it covertly could be because they gained it through narcotic activity, or just plain old government corruption), and crypto is the defacto medium of exchange for those wishing to anonymously move illicitly gained funds worldwide.

What do you think is the relative statistical frequency of Balkan last names of American citizens relative to one's of English, Germanic, and Gaelic origin, and why?

And how do you think that ties in with the idea of a name being inherently more likely to be known by foreign groups and associated with that country?

I guess I understand your point though, that Tesla himself is very much an "American" and therefore Tesla is a household American name. Seems we've derailed very far though from the original discussion.

In case anyone reading this is interested in specific historical information the origins of the listed last names in order are English, German, English, and Gaelic

Here's a census of immigrants roughly around the time Tesla was working in the U.S. (1910's), showing the relative percentages from each region (a point of comparison which is not even accounting for the ones who came from the above regions before this time): https://www2.census.gov/library/publications/decennial/1910/...

It's good to read the facts of history rather than hear trite historical narratives. In this case, it seems inbound immigrants from the Balkans made up just 1.6% in the 1910's.

I'm pretty sure the intention of saying a last name is "all-American" is to comment on the relative statistical frequency of it in a population at any given time, since that is a name people in another country (like China) would associate and know because of their likelihood of seeing it. But again, a lot of people discuss historical narratives rather than historical facts, so the above poster is not uncommon in continuing that tradition of pushing a trendy idea rather than discussing facts and information.

Thanks so much for this! This was an exact problem my team had at a recent hackathon, and we decided to use diffusion instead. We theorised that the initial image parameter might help, but we didn't have time to test it. Documenting this was really helpful. If you have any other articles or advice on using VQGAN (or other generative models) with CLIP (or other language embeddings) I'd love to see them.

Glad you were open to seeing what I'm talking about, and why it concerns me. I also feel I have been guilty of it, and have hesitated to comment in technical conversations as often for the same reasons as you.

I've been trying to correct that in myself as well, seeking out more technical conversations where my statements can be clearly evaluated to be true or false based on technical merits, and thus valuable to anyone reading it.

Unfortunately, a part of nurturing a certain type of community, is to make it slightly unapproachable to those you don't want in it. I think the very plain css of the site already does a great job of that, but I wonder if it's still not enough to curate the community long term.