The title of Tencent's paper [0] as well as their homepage for the model [1] each use the term "Open-Source" in the title, so I think they are making the claim.
[0] https://arxiv.org/pdf/2411.02265 [1] https://llm.hunyuan.tencent.com/
227 karma · joined July 25, 2018
[0] https://arxiv.org/pdf/2411.02265 [1] https://llm.hunyuan.tencent.com/
Also common in robotics applications is the Particle Filter, which uses a Monte Carlo approximation of the uncertainty in the state, rather than enforcing a (Gaussian) distribution, as in the traditional Kalman filter. This can be useful when the mechanics are highly nonlinear and/or your measurement uncertainties are, well, very non-Gaussian. Sebastian Thrun (a CMU robotics professor in the DARPA "Grand Challenge" days of self-driving cars) made an early Udacity course on Particle Filters.
In the words of Brandt, "well, Dude, we just don't know."
Unlike the OP article, it does make use of the math formalism for Kalman filters, but it is a relatively gentle introduction that does a very good job visualizing and explaining the intuition of each term. I have gotten positive feedback (no pun intended!) from interns or junior hires using this resource to familiarize themselves with the topic.
If you are making a deeper study and are ready to dive into a textbook that more thoroughly explores theory and application, there is a book by Gibbs[1] that I have used in the past and is well-regarded in some segments of industry that rely on these techniques for state estimation and GNC.
[1] https://onlinelibrary.wiley.com/doi/book/10.1002/97804708900...
From the Paella paper[2]: "Our proposal builds on the two-stage paradigm introduced by Esser et al. and consists of a Vector-quantized Generative Adversarial Network (VQGAN) for projecting the high dimensional images into a lower-dimensional latent space... [w]e use a pretrained VQGAN with an f=4 compression and a base resolution of 256×256×3, mapping the image to a latent resolution of 64×64indices." After training, in describing their token predictor architecture: "Our architecture consists of a U-Net-style encoder-decoder structure based on residual blocks,employing convolutional[sic] and attention in both, the encoder and decoder pathways."
U-Net, of course, is a convolutional neural network architecture. [3]. The "down" and "up" encoder/decoder blocks in the Paella code are batch-normed CNN layers. [4]
[1] https://arxiv.org/pdf/2012.09841.pdf [2] https://arxiv.org/pdf/2211.07292.pdf [3] https://arxiv.org/abs/1505.04597 [4] https://github.com/dome272/Paella/blob/main/src/modules.py#L...
They also get tons of use in results-oriented modeling of lots of other statistics questions in structured data (home prices, resource allocation, voter turnouts, etc.) but in this luddite's opinion, these sorts of applications tend to be pretty fraught if they short-change the convenience of the model training paradigm for a deeper understanding of the data phenomenology.
I get the feeling that my visual system and the language I use are respectively pretty bad at processing and conveying precise information from a plot, (beyond simple descriptors like "A is larger than B" or "f(x) has a maximum"). I guess I would find it mildly surprising if any Vision-Language model were able to perform those tasks very well, because the representations in question seem pretty poorly suited.
I get that popular diffusion models for image generation are doing a bad job composing concepts in a scene and keeping relationships constant over the image--even if Stable Diffusion could write in human script, it's a bad bet that the contents of a legend would match a pie chart that it drew. But other Vision-Language models, designed for image captioning or visual question answering, rather than generating diverse, stylistic images, are pretty good at that compositional information (up to, again, the "simple descriptions" level of granularity I mentioned before.)
>Importantly, this diffractive camera is not based on a standard point-spread function-based pixel-to-pixel mapping between the input and output FOVs, and therefore, it does not automatically result in signals within the output FOV for the transmitting input pixels that statistically overlap with the objects from the target data class. For example, the handwritten digits ‘3’ and ‘8’ in Fig. 2c were completely erased at the output FOV, regardless of the considerable amount of common (transmitting) pixels that they statistically share with the handwritten digit ‘2’. Instead of developing a spatially-invariant point-spread function, our designed diffractive camera statistically learned the characteristic optical modes possessed by different training examples, to converge as an optical mode filter, where the main modes that represent the target class of objects can pass through with minimum distortion of their relative phase and amplitude profiles, whereas the spatial information carried by the characteristic optical modes of the other data classes were scattered out.
It seems like that may not be so possible.
Later on in the article:
>It is important to emphasize that the presented diffractive camera system does not possess a traditional, spatially-invariant point-spread function. A trained diffractive camera system performs a learned, complex-valued linear transformation between the input and output fields that statistically represents the coherent imaging of the input objects from the target data class.
Note here that the learned transformations are linear, and the Fourier Transform is linear, but you cannot invert from output to input because the sensor measures real-valued intensities of complex-valued fields. All the phase information is lost.
This means the parameters of the trained model fit in something like 7GB (decoder only, half-precision floats) to 24GB (full model, full-precision). To actually run the model, you will need to store those parameters, as well as the activations for each parameter on each image you are running, in (video) memory. To run the full model on device at inference time (rather than r/w to host between each stage of the model) you would probably want an enterprise cloud/data-center GPU like an NVIDIA A100, especially if running batches of more than one image.
The training set size is ~97TB of imagery. I don't think they've shared exactly how long the model trained for, but the original CLIP dataset announcement used some benchmark GPU training tasks that were 16 GPU-days each. If I were to WAG the training time for their commercial DALL-E 2 model, it'd probably be a couple of weeks of training distributed across a couple hundred GPUs. For better insight into what it takes to train (the different stages/components of) a comparable model, you can look through an open-source effort to replicate DALL-E 2.[2]
[0] https://cdn.openai.com/papers/dall-e-2.pdf [1] https://openai.com/blog/clip/ [2] https://github.com/lucidrains/dalle2-pytorch
They're popular for street-level advertisements for (concerts, events, clubs, jobs) to provide a link to more information without the consumer having to type a url into the browser. This is more important if you don't want to pay for a pithy high-level domain as a landing page for the thing you are advertising.
I have also seen them for easily entering WiFi credentials to your device on a new network.
They can be used in museums to link to self-guided tour information on the artwork/exhibits you are looking at. This used to be done more with rentable/loanable walkman+headset kind of devices, but makes more sense to do on one's own mobile device now.
The sun is a very hot (~5500K) blackbody that emits radiation in a broad spectrum, but that spectrum peaks in the visible. Some of that light is incident on Earth and warms it up. Earth also emits its own blackbody radiation, but it's much cooler (~300K), so it emits much less power over all and its spectrum peaks somewhere in the long infrared. The system is in equilibrium when the sun has heated the Earth enough that the total energy radiated away from Earth is equal to the fraction of the sun's radiation that is absorbed by Earth.
Earth's atmosphere can change this equilibrium temperature by changing the fraction of incident energy that is absorbed by Earth, or changing its emissivity. Moreover, these changes can be wavelength dependent. Greenhouse gasses are gasses that are transparent in the visible spectrum but are reflective in the infrared, which allow in most of the sun's energy, but "trap" infrared energy that is being emitted by earth. Smoke and soot, by contrast, are very reflective in visible light (we can see them!) and so "block" much of the sun's energy from heating the Earth.
[0]https://en.wikipedia.org/wiki/Nuclear_winter#Criticism_and_d...