carterschonwald

at the very least i have tools that let me easily hit better perf for fancy dense memory layouts, and the same tooling lets me experiment with frankly wildly wacky sparse and structured memory formats. the performance claims at least on the dense side are solid so far!

the sparsity angle is because i want magic in the world. like anyone with a really chunky computer like any of those mac mini pros or serious workstation / server tier compute should be able to train from scratch their one 31b equivalent model in a week or so tops is the goal post i have in mind

like i can do all sorts of memory layout of tensors/matrices etc tricks that if you dont have the abstractions for it would just never happen. so i can optimize the kernel flops

i genuinely think these models should be like 0.1 percent sparse for same capabilities we associate with them today, but theres no sane way to do that with extent tools. i built the right core tech for that in 2014 when there wasnt a market, but now there is and the experimentation velocity is wild.

amusingly llms really have a hard time using my simple apis because its not in distribution array programs. but i literally stood up cpu custom memory format and micro kernel for dense causal attention in less than 24-36 hours and outperforms the equivalent fused ggml/llama cpp fast oath by like 20-25 percent

its a technological tragedy because it was the only facility im aware of globally that could actually manufacture steel based carbide alternatives at commercial volumes. idk if the relevant equipment is being operated by anyone post bankruptcy. powder steel equipment is a bit less destroyed when turned off, but i think the key blocker is that heat cycling a 3k centigrade furnace will age the material and cause cracking thatmakes it hard to resume the powederization flows

good for the founders. also explains why my resume got dropped on the floor as a desk reject :p

use that to install assuming you have whatevers needed to set up.

ive a much fancier next gen thing i hope to make available as a sass mid to late summer, but if you have any feedback or questions on mine do reach out

mine def has to be the initial impl of my python to my personal fave compiler ir. i got claude chat to write it in two sentences. 4 turns helping it rmeemver where it put shit becsuse of transcript issues

saying i'm "furious" has lower entropy that incredibly implausible abuse. In some first party harnesses it just results in doom loops, but thats usually because the COT is hidden after the immediate turn in those setups. COT persistence helps with a lotta stuff