ogrisel

> advances in nuclear fusion or other technologies that lower the price of CO2 neutral electricity

We are still far from the point where anybody serious could reasonably affirm that advances in nuclear fusion would be expected to lower the price of CO2 anywhere in the foreseeable future.

That does not mean we should slow down R&D in nuclear fusion in any way but rather bet on the fast deployment of renewables if the objective is to lower the price of C02 neutral electricity in the coming decade(s).

There is a similar problem with the YouTube recsys on cheaply generated videos (e.g. a physics + 3d rendering model to generate colorful videos of Spiderman and Batman driving colliding cars).

See this older piece by Natasha Lomas on Tech Crunch:

I watched 1,000 hours of YouTube Kids’ content and this is what happened… http://tcrn.ch/2iPXpIA

There is also this very interesting 2017 paper:

Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes

Lei Wu, Zhanxing Zhu, Weinan E

https://arxiv.org/abs/1706.10239

I think it was the first paper to study the volume of the basins of attraction of good global minima and used the poisoning scheme to highlight the frequency of bad global minima that are typically not reachable found via SGD on the original dataset without poisoning.

Actually it has explicit support for async task based reporting:

https://github.com/gaogaotiantian/viztracer#async-support

viztracer can create traces for python programs. Not sure about async-io awareness though.

And here is another one:

https://oxisenergy.com/

but this one was set "in administration" on 19 May 2021...

Apple engineers recently started to contribute bug reports, root cause analysis and even fixes to solve compatibility issues with their hardware and software platforms via a shared github account: https://github.com/Developer-Ecosystem-Engineering

For instance for NumPy:

https://github.com/numpy/numpy/pull/19926

Although it's weird to interact with such an anonymous account in github discussions, it is very welcomed by the maintainers of those projects because debugging without assistance those low-level platform specific issues was a significant maintenance burden.

As @jjerphan commented above, there is already ongoing effort to get an improved portable brute-force implementation in vanilla scikit-learn, see:

https://news.ycombinator.com/item?id=29069760

> Intel are focused on data-parallel C++ for delivering high performance, rightly or wrongly.

They also invest efforts in making it possible to write high performance kernels in Python using an extension to the numba Python compiler:

https://github.com/IntelPython/numba-dppy

plantnet works very well. I have also used Google Lens (the magic button on Google Photos on Android) but I have the subjective feeling that plantnet works more reliably for plant identification.

Nesting is actually not such a problem in itself, it just hides the fact that your seemingly simple object on the surface might have a reference to a large collection of small subojects that will be slow to pickle.

If you plan to mint digital art and sell or buy NFTs, please consider low-carbon platforms such as:

https://www.hicetnunc.xyz/

that is based on Tezos and therefore relies on Proof-of-Stake instead of Proof-of-Work to reduce the environmental impacts.

pickle.dump/load is only slow if your main objects has references to many small nested objects: e.g. a large python dicts with million of key values that are small Python str or int objects for instance.

If your main object has only references to a few large sub-objects (e.g. a bunch of multi-MB or GB numpy arrays to store the numerical parameters of a machine learning moodel), then it can be very fast, basically IO-bottlenecked by writing or reading the bytes to/from the disk.

This kind of matches the performance I recorded when numpy is linked to vecLib for large matrix matrix multiplication in float32:

https://gist.github.com/ogrisel/87dcf2c3ab8a304ededf75934b11...

Note however there is currently no way to build and link numpy and scipy against vecLib to get correct results when calling LAPACK routines (to get Singular Value Decomposition for instance). It might be related to low level fortran ABI problems but I am not an expert so I don't know for sure.

It's possible to get a fully working numpy / scipy stack with OpenBLAS and gfortran by using the conda-forge distribution:

https://github.com/conda-forge/miniforge#download

The performance is not as good as with vecLib (see the linked benchmark) but it's already very good (e.g. compared to a similarly priced Intel or AMD laptop with OpenBLAS and maybe even MKL).

Hopefully Little Bobby Tables did not participate.

But do you see any practical scenario where you have to keep 3TB of parameters in accelerator's RAM at inference time?

The accuracy would have to be significantly higher that any alternative to justify monopolizing that many hardware resources.

Actually now I realize that the title and the intro paragraph are contradicting each other...

As a nitpicking person, I really like to read a post that does not confuse GB/s for GiB/s :)

https://en.wikipedia.org/wiki/Byte#Multiple-byte_units

A MacBook Air with M1 does not have a fan and I find it faster than my Dell XPS 13 from 2015 when running the scikit-learn test suite which is quite CPU intensive.

Running those tests on the XPS 13 would typically cause the fan to turn on after a minute or two if I remember correctly.

Isn't it possible to play under linux directly using chrome or chromium?

One could expect that docker on macOS with M1 will be able to run aarch64 linux container images and therefore make it more convenient for developers to better maintain open source libraries for this platform (without suffering from long build time on an under-powered external ARM box, or the complexity to setup a cross-compilation toolchain or the slowness of a qemu layer).

I don't know about Mother Nature, but silicate rich (or carbonate rich) rocks weather by sucking C02 out of the atmosphere (or oceans) and it can apparently be accelerated by spreading crushed rocks on fields or beaches.

- https://en.wikipedia.org/wiki/Enhanced_weathering

- https://www.theguardian.com/environment/2020/jul/08/spreadin...

- https://projectvesta.org/

The debate is low carbon vs high carbon (if we all agree that climate change is the number #1 pbm).

Here are some numbers to back this statement:

https://www.energy-charts.de/energy.htm?source=all-sources&p...

It's true that in the first 3 years after the decision to speed-up the closure of nuclear in Germany in 2011, coal use increased. But more recently it went down drastically and was replaced mostly by wind and solar on a yearly basis.

Here is similar data but not just Germany (using the data from the 2020 BP statistical review):

https://nbviewer.jupyter.org/github/ogrisel/energy_charts/bl...

and github.

Data augmentation will help prevent your model from overfitting a bit but the amount of useful information you get from naively augmented data will reach diminishing returns at some point.

Data augmentation alone (e.g. rotations / shift / crops / color perturbations / cutout... of a single photo of an husky dog) will never yield the added information that is contained in new pictures showing subtle variations of the phenomenom your are trying to model (e.g. a new photo of a Dalmatian dog if you have no Dalmatian dogs in your original training set).

The reason is that you do not need to finely understand the structure of individual sentences to group documents by similar topics. Word order does not matter much for this task. Hence the success of methods that use Bag of Words (eg TFIDF) as their input representation.

Big transformers neural network are probably overkill for topic modeling. More traditional methods implemented in Gensim or scikit learn such as tfidf vectors followed by SVD (aka LSI) or LDA or NMF are probably just fine to extract topics (soft clustering).

Better planning of power prices and demand response for power grids with a high fraction of weather dependent renewable energy production.

A simpler alternative to get carbon negative buildings is to use Cross-Laminated-Timber (CLT) to build both the frame and the walls themselves.