Zacharias030

Not my area of expertise, but here you go.

I don’t know how much tool use there is these days of the llm „just“ calling image generation models, with a bunch of prompt reformulation for the text-to-image model which is most likely a „steerable“ diffusion model (really nice talks by Stefano Ermon on youtube!).

Actually multimodal models usually have a vision encoder submodel that translates image patches into tokens and then the pretrained llm and vision model are jointly finetuned. I think reading the reports about gemma or kimi VL will give a good idea here.

What is „market manipulation“ actually vs non-manipulative buying and selling to make a profit?

100% agree with you both and I love the litmus tests mentioned!

Looks like big and small tech are mostly doing things wrong then?

Those must have been about other things than LLMs though. Meta has huge inference loads for other types of models.

The crypto mind cannot comprehend

This was a hot topic in 2019-2020 and I just thought I‘ve read an article like this back then from NYU and another from the National University of Singapore, because I and many others were at the time independently working on graph problems using Transformers [0,1].

Turns out this article _is_ the one from NYU, just on arxiv now:

https://graphdeeplearning.github.io/post/transformers-are-gn...

[0] e.g. https://arxiv.org/abs/2003.10536, [1] ProteinMPNN, Dauparas et al.

This is changing rapidly.

Google announced they are serving 500T tokens per month. State of the art models are currently trained with less than 30T tokens. Even if training tokens are more costly to run (eg, a factor of 3x for forward, backward, and weight updates, and take another factor of 2x for missing quantization), you end up in a situation where inference compute dominates training after a very short time of amortization.

I like how the smallest Eurocentrism is greeted with the wagging finger to be inclusive on hackernews, while the expectation is that 50 state acronyms are well understood by any reader from Lazio, Lorraine, or Thuringia ;)

HN at its best! What do you do for a living, Sir,

I‘ve only seen it done by hedge funds so far. What were you referring to?

I believe that if the reader is familiar with PPO, they will immediately understand GRPO as well.

I‘ve heard people say that GRPO giving a zero gradient in cases where neither the current sample nor the group scores give any reward is advantageous for optimization. It avoids killing your base model with low signal to noise updates, that can be a problem in PPO where the critic usually causes a non-zero gradient even for samples where one would rather be like „problem too hard for now, skip.“

I‘d be curious to hear you lay out your thoughts though!

Actually no! If we take their paper at face value, the crucial innovation to get a strong model with efficiency is their much reduced KV cache and their MoE approach: - where a standard model needs to store two large vectors for each token at inference time (and load/store those over and over from memory) deepseek v3/R1 only stores one smaller vector C that is a „compression“ from which the large k,v vectors can be decoded on the fly. - They use a fairly standard Mixture of Expert (MoE) approach, which works well in training with their tricks, but whose inference time advantages are immediate and equal to all other MoE techniques, which is to say that from ~85% of the 600B+ params that are inside the MoE layers, the model at each token inference step will only pick a small fraction to use. This reduces FLOPs and memory io by a large factor in comparison to a so-called dense model where all weights are used for every token (cf Llama 3 405B)

do you?

If you don’t like the title, wait till you see this acronym: „… we introduce the Tensor ProducT ATTenTion Transformer (T6), a new model architecture…“

I‘m not sure I believe that when just yesterday, I ran a bunch of data analysis, simulation, and visualization based on a single csv and produced 5-10 decent matplotlib plots in a 90 minute back and forth between OpenAI canvas, vscode, python and jupyter. I didn’t believe some of my results and then discovered some problems of the dataset itself, so there was some „real work“ done in those 90mins.

I can say with certainty that I wouldn’t know how to wield matplotlib and pandas with such fluency in an hour, even though I am perfectly able to read the implementation and query for some relevant intermediate results to check my mental model.

Granted this is not the world’s most complex problem, but that is a good example of the domains where these tools are incredibly useful and productive already (I didn’t have to consult the docs even once). So in a way I think of LLMs as very good interfaces to the docs :)

I often feel that the UI aspect of new technologies is underappreciated. All our computers (even grep) are turing complete. This means software engineering is fundamentally a discipline of building better user interfaces that allow us to do whatever we want more easily.

I am always curious how other people experience these things as so useless :)

My iOS Safari has it. I turned it on for the NYT, because I wanted a dark theme and then turned it off again because I realized that I like what they do with their pages (still have an ad blocker turned on though, because subscribers still see tons of ads).

What was the service called? Never heard of it.

Love this answer.

I know from personal experience that it is tiring for the knees.

What makes you think that it is „bad“?

I find this a very interesting observation! (Don’t have a dog) Note though, that the paper describes the different speeds that are possible when walking with equal efficiency to be quite narrow in comparison to running.

There is this related saying that „the fastest marathoners burn just as much calories as the slowest to complete 42km, they just do it in a much shorter time“. This kind of range is not available to walking.

What kind of „AI“ / machine learning was used?

This problem was treated with a bit more sophistication by Harvards Lieberman in „A story of the human body“ and the corresponding nature article [0] establishing persistence hunting as an ancestoral hunting technique of homo sapien. See figure 2b for the plot you were looking for. Running faster is less efficient but only slightly so and walking is a U-shaped curve like most mammal gaites.

It shows that in contrast to most animals, the efficiency curve of humans for running speeds is extremely flat, ie, we are about equally efficient at many different speeds, while the kind of game that we hunted was not.

The discrepancy allowed us to find a speed where we could exhaust the animal after 10-30km (as I understand) provided we were also excellent trackers.

https://www.nature.com/articles/nature03052.epdf?sharing_tok...

Could you point me to some references to learn about his?

Note that this is a „small o“, so o(1) captures terms that „divided by 1“ go to zero as m to infinity.

https://de.m.wikipedia.org/wiki/Landau-Symbole

This sounds like a detailed account of burnout. Take some time away from it all and find your thunder and curiosity again?

Consider that climate change doesn’t care for percentages, but total CO2equivalents. Coal is still the third-fastest GROWING energy source in TWh (from the article), so the question is really 90% of what?

this.

John Carmack, https://en.m.wikipedia.org/wiki/John_Carmack

Interesting! Can you elaborate?