xyhopguy

Microbatch size is a hyperparameter, it can be set to 1 and work just as effectively. With gradient accumulation it's equivalent even. Large batch sizes are used to increase parallelism, and sometimes to reduce variance in the loss signal (at the cost of increased bias).

Batch size is frequently limited by compute bottlenecks well before memory.

No one wants to be around an alcoholic. But it's not to be nice, but to understand them, their struggles, and their perspectives.

Anecdote - I was at a concert with my brother, outside in the smoking area. A homeless woman came by, didn't say anything but was kinda just looking around. My brother is the only one who spoke to her, he offered his cigarette butt. Which is exactly what she was looking for.

Never would have occured to me- but I would guess it made her feel more seen and human.

cancer is more like debugging a gigantic DL model than an operating system. spaghetti of redundancies all the way down.

He wasn't the only one who saw the problem computationally. Famously, the mathematician Michael Waterman sat on the other-side of the race for the human genome.

not really. early deep learning models were run on single consumer-grade GPUs. the inflection occured _right_ when parallel computing became fast enough to do backprop in a reasonable amount of time with performance better than tree methods.

at that time all the compute resources in the world would not have been enough to train the models from even the last ~6 years or so, probably more.

3x increase in compute for a 1.5x increase in tdp is pretty good considering the underlying process had barely changed. In anycase, consumer GPUs aren't a good metric as they operate with different economic constraints.

H100 to GB200 saw a 50x increase in efficiency, for example.

> I wanted to conjure models I could play with in my hand. I wanted a museum where I could walk around inside the epithelium during an immune response. I wanted to put ideas into physical space, like on a pinboard—TLRs go here, with the other innate armament; CD4+ T cells are there, in the adaptive world—but I wanted it to be as searchable, copy-pasteable, shareable, and composable as text.

VR anyone?

Currently we deal with >= 8am sunrise for 2 months of the year. With this proposal, you get it for November and February too. yay!

this proposal will set sunrise at 9am during december in Oregon and washington. 'Wake up later' doesnt really work for most people.

I recently dealt with this. Right before my free trial expired, I went to cancel. A support rep offered another 3 months free. The first red flag was not receiving a log of our conversation. Three months go buy and sure enough, when I went to cancel, they told me I had already committed to a year. Incredible. After some persistence and vague legal threats I managed to receive a full cancelation and refund.

Definitely is technically in the northwest part of the state, but I'm not sure I'd call it close to the Washington border.

It's much closer to Salem and Bend than it is Portland, for instance.

For oregon the population center is almost exactly Detroit. There is a large population in the willamette valley south of portland (and bend is captured by this too).

Weird, it's all bulk in Salem.

I have no idea where you guys are seeing these prices. Chalice, on the absolute high end that I've seen, tops out at 250/oz.

Nectar/Green Cross are my gotos, both of them are well below the prices yall are discussing. Green cross in particular has better stuff than the other venues as well. Maybe try shopping in Salem instead of Portland? Dunno, I never go into the city for weed.

go to greencross in bend, they have the exact prices i'm referring to. Dont go to the stores that purchase from bulk retailers, shops that have their own farms typically have higher quality and lower prices.

i mean this is verifiably wrong:

try this place in salem, really nice stuff and great prices. "Reserve" is their top-of-the-plant pickings that go for $160 oz. Not a big difference between their bulk (75-100) and reserve.

https://www.greencrosscanna.com/commercial-street

its not hard to get top shelf ounces for $100-140 in oregon...

thats a dewpoint of 76F!!! dunno what its like a little more east but in the northwest that's unheard of.

Portland's dewpoint I think peaked around 65F.

At 49C you need a dewpoint of almost 70 (!!) to get 20% relative humidity. thats nasty -- during the heatwave in portland the dewpoint was between 63-68 most days. Usually its ~55. For context, the dewpoint on the east coast during the storm season usually is between 70-75.

there is never a million people on the oregon coast.

'discord professional' seems like pretty low hanging fruit and an obvious upgrade over slack + zoom

or up north... 10-11pm, awful.

many long read assemblers (good ones, at that) treat it as a hamiltonian problem.

theyre not really similar problem, genome assembly is a problem that is most effectively solved in the lab and with traditional graph algorithms. protein folding fits very nicely into structured prediction and has a very quantitative way of measuring performance. Genome assembly is more qualitative, to say the least.

given a bunch of reads of short(50-300bp) or long (1000-100,000 bp) lengths (we use different algorithms for each) you need to resolve a 1D sequence that holds all of those continguous subsequences for each chromosome. Genome assembly is often described as finding a hamiltonian cycle among the data. For short reads, we use debrujin graphs to avoid the hardness of a complete hamiltonian. for long reads, MinHash has become a popular heuristic.

But this can be tricky, for instance, in species that isn't haploid -- you actually have two possible genomes that you need to correctly assemble. Sometimes the difference is a base or two, but sometimes its much longer stretches that can appear as large 'bubbles' in the assembly graph.

The assembly problem can be harder or easier depending on the organism, for instance, Wheat is notoriously hard to assemble due to the fact that it has 3 mostly (but not completely) distinct genomes. The Norway Spruce is composed of ~80B basepairs, which will put strain on even the biggest machines. Oh and its mostly repeats.

Repeats are _everywhere_, and they can be really long (sometimes on the order of millions of nucleotides). Also -- depending on the species youre assembling, the repeat content and the kind of repeats can change. You also can get different errors from the library preparation, again related to the species of origin.

Using lab techniques is particularly helpful, as we can leverage molecular and genetic information. BACs (baceterial artificial choromosomes) can be used to break the genome into smaller continguous chunks and to act as 'anchors' for the larger assembly problem. Recombination rates of different SNPs can be used to infer spatially local segments of DNA, and optical tags can be added and imaged to provide physical anchors for certain sequences. Some organisms can be bred into pure lines, with limited heterozygosity. Expressed RNA transcripts can be used in a similar manner, as they are the concatenation of ordered exons. Combining these methods is typically the best way to get a good genome assembly.

Some of the bubble resolution heuristics could probably be improved with deep learning, but getting the right data for that is probably more effective with traditional graph algorithms. Also -- you really only need a good genome assembly once, and sometimes it doesnt even need to be all that good.

Genome assembly is a _really_ fascinating area of research, and honestly, the right direction is probably to represent genomes as something other than 1D sequence that more accurately reflects biology. Deep learning is still looking to make its mark on genomics, and unfortunately its not well suited for much of the discovery efforts such as genome assembly.

non-determinism can also be modeled as a state-machine (and in fact, you can prove non-deterministic state machines can always be represented by deterministic state machines).

You make excellent points -- the point related to culture is the most common I've heard from people outside Portland (particularly non-white people).

I think the only things you omitted are the differences in weather and the size of the cities.

- Portland is a fairly small city (especially inner portland) with a long history of anti-growth sentiment. The result is very rural agricultural areas _just_ outside of Portland.

- The weather patterns bring different wildlife and kinds of beauty. While the forests outside SF are quite pretty, they are also very different than the wet douglas fir forests up north. Beauty is in the eye of the beholder -- Portland also extends into the hills and provides famous views of Mt. Hood from the SW hills. The portland zoo for instance, is like going from a city into a rain forest.

The oregon coast is cold and rainy 95% of the year, not the most fun place to visit.

agreed -- portland is right there with very similar recreational activities and literally 25% the rent. why rent a 1b1b in SF when you can rent a mansion right on washington park for the same price?

right but thats not the same as selecting for IQ.

thats not even getting into the fact that genetics << environmental effects for predicting IQ..

Sun in winter sounds nice

Because sunrise is already at like 8 in the dead of winter. Pretty hard to wake up after that if you work at 9