ckrapu

This is an awesome idea.

In my first digital design class, my physics professor (who I took 6 classes with and respect very much) took a look at my breadboard computer with no consistent color coding for groups of wires and overall bad layout. He said it was bad and that I should start over from scratch in front of the whole class.

Awesome link and one of the best intros I’ve seen for really digging into compute graphs of statistical models to see what’s under the hood.

“non-teaching staff” includes all the research scientists, postdoc, and techs that help make the research happen. These universities are not especially good at teaching compared to SLACs. They are exceptionally good at research.

For models with >100 parameters, there are theoretical reasons for why JAGS can fail badly. It has to do with the mixing time of Gibbs samplers versus Hamiltonian Monte Carlo.

In case anyone wants to do this with factually grounded content, check out my site: https://www.wikistorytime.com/

Just be nice with the usage because the billing is going on my personal OpenAI API key!

Definitely Hamiltonian Monte Carlo / NUTS for the last few years. From both a practical standpoint and theoretical perspective, I see no reason why it doesn't scale to millions of parameters if you have enough GPUs. Stuff like Gibbs samplers and Metropolis-Hastings tends to run into issues once you get beyond ~100 parameters, sometimes more with clever graph structure.

I am also really interested to see if CHEES HMC (https://proceedings.mlr.press/v130/hoffman21a.html) makes it as a viable in-the-wild inference technique since it gets around the nastiness of writing NUTS as a recursive algorithm on hardware (GPU / TPU) generally not friendly to that type of control flow.

Also do you have any functionality for recognizing graph structure (I.e. marginalizing out variables, identifying conjugacy, automatic reparameterization)?

I would love to build a massive global macroeconomic model (vector autoregressions for exchange rates/ securities, spatiotemporal processes for raw materials like crop yields, etc) using something like this. It would just take way too long with Monte Carlo.

I would use one of those for scenarios when I need proper measures of uncertainty like credible regions.

The methods listed in this project (EM algorithm, VI) tend not to provide this as well but usually produce something that looks like a good posterior mean (global optimum under mean square loss) in my experience.

I’m a diehard MCMC advocate but love that the is system is built for real-time applications. Hope I get a chance to try this out sometime!

Awesome. In wind mode, I see a blue line of low velocity in the middle of the North Atlantic. It’s flanked by high speed zones. Is that a real thing?

First person I’ve seen on HN also from ND! And yes, TR National Park is a good one but a little hard to conveniently travel to unless you have connections to the state already.

A better comparison would be ND and Ukraine in terms of climate.

This definition might be true in some places but the vernacular usage of “pesticide” by users is slanted towards targeting animal life and not plants.

Cool work and I hope to be able to use it some day.

Most importantly though, does anyone know how they made the animation with the data sources? I feel like that came from something lightweight and convenient and I'd like to know what it was.

Yes, and my only commentary here is that the threshold for academic decision science is a lot higher than the one I have for managing my day-to-day life.

These are excellent but I think #8 (statistics of small numbers) is one that, with some nuance, you can get more personal mileage out of if you're willing to tolerate being wrong some of the time.

For example, I've noticed that of the managers I've had, the good ones and bad ones are cleanly separated by one trait or property. This is a sample size of maybe 7 or smaller.

Depending on your priors, my P(Hypothesis | Data) is maybe 0.6-0.7 which is terrible from a research and scientific point of view.

I'll take that level of certainty for day-to-day decision making anytime, though. Make enough of these weakly-backed hypotheses and you can avoid a lot of trouble.

Potentially, yes! The scenario I am imagining is that context A as context for a query yields results 1, 2, and 3. Sometimes, finding neighbors of 1 (ie not necessarily in the top K w.r.t A) instead of going to results 4, 5, 6 might be better.

I’m really interested in content explaining how to navigate graphs of embedded items for Q/A. Any resources on how to do this or arguments for why it’s a bad approach?

For example, if my top K docs aren’t answering the question but each are linked to neighbors, I’d want to know some folk wisdom or tricks for structuring the neighbor graph to cheaply expand the set of useful results.

An assumption here is that attractive men both pair with women at a higher rate and also have more children. I don't see any reason that's the case - the very largest families I am aware of in my extended social circle belong to men who are not necessarily the greatest catches.

There’s a few companies making inroads here. Upcodes is one, IIRC. I’ve seen some work from Stantec that is perhaps more conventional ML and less LLMs. There’s also civilis.ai but I’m not sure how serious they are.

I’m broadly interested in this field and have done some prototypes for NLU as applied to development regulations.

This is a bit of an embarrassment for that country in that they aren’t trusting their own people to actually build cars properly and have to build up from just reassembling cars. I do not expect them to hit their target of fully manufacturing EVs by mid-decade.

I think I can see what's going on here - I used a shapefile with the boundaries of the world's countries (and their coastlines) which had some geometric simplification applied to it. This file was used to mask out any water (for which the embedding model won't do much), but I think that the simplification process snapped the coastline too far inland, leaving some points on land which were masked out erroneously.

You pretty much hit the nail on the head. The application areas you mentioned are the same as the ones that I had in mind when developing this.

Yup, and while the similarity search is perhaps the most visually appealing way to work with it, the real use (in my opinion) is in providing generic sets of geospatial features which are reusable across applications. I've built out versions of H3-referenced feature sets at each of the jobs I've had over the last 10 years.

Cool! Glad you got to see it working and that presentation was a nice reason to make sure everything was cleaned up.

Thanks! And you are giving it too much credit here - it's just trained on one-hot encoded land cover (24 classes) from Copernicus. Using imagery directly would be # 2 on my list of to-dos after including elevation in the input data.

I intentionally avoided using lots of ocean areas - this way I cut down the number of required sites for inference from ~100 million (at resolution 7 in the H3 system) to around 25 million.

Sorry! The presentation could be better. I'll work on the FAQ.