omneity

Do you have any evals on how good LLMs are at generating Glyphlang?

I’m curious if you optimized for the ability to generate functioning code or just tokenization compression rate, which LLMs you tokenized for, and what was your optimization process like.

Sounds like most of this is simply taking shortcuts instead of properly parsing[0].

0: https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-va...

Thinking from first principles, a large part of the content on stack overflow comes from the practical experience and battle scars worn by developers sharing them with others and cross-curating approaches.

Privacy concerns notwithstanding, one could argue having LLMs with us every step of the way - coding agents, debugging, devops tools etc. It will be this shared interlocutor with vast swaths of experiential knowledge collected and redistributed at an even larger scale than SO and forum-style platforms allow for.

It does remove the human touch so it's quite a different dynamic and the amount of data to collect is staggering and challenging from a legal point of view, but I suspect a lot of the knowledge used to train LLMs in the next ten years will come from large-scale telemetry and millions of hours in RL self-play where LLMs learn to scale and debug code from fizzbuzz to facebook and twitter-like distributed system.

It's likely to either be an approach like this [0] or something even less involved.

0: https://github.com/apple/ml-clara

Related, I heard about curriculum learning for LLMs quite often but I couldn’t find a library to order training data by an arbitrary measure like difficulty, so I made one[0].

What you get is an iterator over the dataset that samples based on how far you are in the training.

0: https://github.com/omarkamali/curriculus

I tried to implement something similar to optimize sampling semi-random documents from (very) large datasets on Huggingface, unfortunately their API doesn't support range requests well.

Given its age, I thought this language had a chance to predate the eponymous computer mouse, but nope. It was created a good decade after the first documented reference to computer mice [0].

0: https://en.wikipedia.org/wiki/Computer_mouse

> With a typical LLM SDK

What is a typical LLM SDK?

Thanks, this was helpful! Reading the seminal paper[0] on Universal Transformers also gave some insights:

> UTs combine the parallelizability and global receptive field of feed-forward sequence models like the Transformer with the recurrent inductive bias of RNNs.

Very interesting, it seems to be an “old” architecture that is only now being leveraged to a promising extent. Curious what made it an active area (with the works of Samsung and Sapient and now this one), perhaps diminishing returns on regular transformers?

0: https://arxiv.org/abs/1807.03819

It tells you about their ambitions..

Isn’t this in a sense an RNN built out of a slice of an LLM? Which if true means it might have the same drawbacks, namely slowness to train but also benefits such as an endless context window (in theory)

If performant FPGAs were more accessible we’d be able to download models directly into custom silicon, locally, and unlock innovation in inference hardware optimizations. The highest grade FPGAs also have HBM memory and are competitive (on paper) to GPUs. To my understanding this would be a rough hobbyist version of what Cerebras and Groq are doing with their LPUs.

Unlikely this will ever happen but one can always dream.

Not until it gets tensor parallelism.

You're right. Let me correct myself: a hobbyist-friendly hardware solution. Dolphin's PCIe switches cost more than 8 RTX 3090 on a Threadripper machine.

I wish for a hardware + software solution to enable direct PCIe interconnect using lanes independent from the chipset/CPU. A PCIe mesh of sorts.

With the right software support from say pytorch this could suddenly make old GPUs and underpowered PCs like in TFA into very attractive and competitive solutions for training and inference.

To my knowledge it's not as much hurting the parent domain as having two separate "worlds". Your docs which are likely to receive higher traffic will stop contributing any SEO juice to your main website.

The way I understood OP’s point is that because LLMs have been trained on the entirety of humanity’s knowledge (exemplified by the internet), then surely they know as much as the entirety of humanity. A cursory use of an LLM shows this is obviously not true, but I am also raising the point that LLMs are only summoning a limited subset of that knowledge at a time when answering any given prompt, bringing them closer to a human polymath than an omniscient entity, and larger LLMs only seem to improve on the “depth” of that polymath knowledge rather than the breadth of it.

Again just my impression from exposure to many LLMs at various states of training (my last sentence was not an appeal to expertise)

While an LLM is trained on trillions of tokens to acquire its capabilities, it does not actively retain or recall the vast majority of it, and often enough is not able to make deductive reasoning either (e.g. X owns Y does not necessarily translate to Y belongs to X).

The acquired knowledge is a lot less uniform than you’re proposing and in fact is full of gaps a human would never make. And more critically, it is not able to peer into all of its vast knowledge at once, so with every prompt what you get is closer to an “instance of a human” than “all of humanity” as you might think of LLMs.

(I train and dissect LLMs for a living and for fun)

Nemotron now works on LM Studio if you update the runtime (from the settings > Runtime screen).

The default chat template is incorrect though and will fail but I published a corrected one you can replace it with: https://gist.github.com/omarkamali/a594b6cb07347f501babed489...

What a strange naming choice, mixing two things (vLLM and LoRA) while being related to neither..

Mistral Large 3 is reportedly using Deepseek V3.2 architecture with larger experts and fewer of them, and a 2B params vision module.

You might actually get that desired behavior through reasoning, or if the model was reinforced for coding workflows involving COM, or at least enough stack diversity for the model to encounter the need to develop this capability.

In the case of LLMs with reasoning, they might pull this off because reasoning is in fact a search in the direction of extra considerations that improve its performance on the task. This is measured by the verifier during reasoning training, which the LLM learns to emulate during inference hence improved performance.

As for RL coding training, the difference can be slightly blurry since reasoning is also done with RL, but for coding models specifically they also discover additional considerations, or even recipes, through self play against a code execution environment. If that environment includes COM and the training data has COM-related tasks, then the process has a chance to discover the behavior you described and reinforce it during training increasing its likelihood during actual coding.

LLMs are not really just autocomplete engines. Perhaps the first few layers or for base models can be seen as such, but as you introduce instruct and reinforcement tuning LLMs build progressively higher levels of conceptual abstractions from words to sentences to tasks like CNNs learn basic geometric features then composing those into face parts and so on.

While not solving everything (such as a single bill), I implemented borgllm[0,1] as a way to get some of that convenience and development speed with no lock-in or even infra.

Perhaps you’ll find it interesting.

0: https://github.com/omarkamali/borgllm

1: https://borgllm.com

Slightly related, I made a monitor[0] for AMD gpus with a nifty chart. I had many issues with nvtop, it is a bit too strict for some situations and ends up crashing too often.

0: https://github.com/omarkamali/picomon

Hello HN! I've been trying out some AMD gpus for Gen AI workloads lately and found the state of monitoring quite unsatisfying.

The best option out there, `nvtop`, has some very strict correctness asserts crashing it too often. With `picomon` I tradeoff some of that accuracy for increased reliability.

If your amd-smi version is incompatible, please open an issue and I'll be happy to fix that. Happy hacking!

This strongly reminds me of the Orthogonality Thesis.

https://www.lesswrong.com/w/orthogonality-thesis

Feels like most resources on the map aren’t mined yet.

Perhaps not a CEO but I figure an AI COO could be quite valuable for founder-types like me.

It was hard for me to believe as well but streaming games wirelessly on a Quest 2 was totally possible and surprisingly latency-free once I upgraded to wifi 6 (few years ago)

It works a lot better than you’d expect at face value.