alexgartrell

https://thinkingmachines.ai/tinker/ https://github.com/thinking-machines-lab/tinker-cookbook

(As a TML person, I'm obviously biased, but I couldn't resist because of "tinkering").

TBF it's hard to imagine a real architecture change that wouldn't require a ton of compute, but you could certainly fine tune and play with different recipes, loss functions, etc. And Claude can carry you a lot of the way through doing this.

One fun task is to invent a tool and then train a small model to use it. You could export that small model and run it locally for free forever to do your thing. I think this is what a lot of Software Engineering will look like later.

There are a lot of other high level abstractions here to look at. Prime Intellect has one.

The other thing to play with is self-hosting small models, but IMO most of the interesting stuff is actually related to multi-gpu or multi-node inference so there's not necessarily a ton to learn here.

The linked article says they decided to do CD in 2016 fwiw so that's not inconsistent with what I said.

You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.

I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.

For the peanut gallery more: I worked with both of these guys at Meta on this.

The "servers are only on for a few hours" thing was like never true so I have no idea where that claim is coming from. The web performance test took more than a few hours to run alone and we had way more aggressive soaks for other workloads.

My recollection was that "write zeroes" just became a cheaper operation between '12 and '14.

A fun fact to distract from the awkwardness: a lot of the kernel work done in the early days was exceedingly scrappy. The port mapping stuff for memcached UDP before SO_REUSEPORT for example. FB binaries couldn't even run on vanilla linux a lot of the time. Over the next several years we put a TON of effort in getting as close to mainline as possible and now Meta is one of the biggest drivers of Linux development.

I did something similar a long time ago https://github.com/facebookresearch/py2bpf

It was definitely a toy, I transliterated from python bytecode (a stack based vm) into bpf. I also wrote the full code gen stack myself (bpf was simpler back then)

But using llvm and not marrying things to cpython implementation makes this approach way better

Not sure it’s relevant in the cloths these guys take

The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.

I’d imagine that these clouds are probably being incentivized to participate

I don’t think pivot_root is necessary for something like this, but a new mount namespace will definitely help avoid creating a mess on accident

More low effort posts please!

Sharing a queue itself is not new https://www.kernel.org/doc/html/v5.8/networking/packet_mmap.... and https://docs.kernel.org/next/userspace-api/perf_ring_buffer.... are two examples.

Issues with io_uring security mostly stemmed from an old architecture and just the fact that there's a ton of surface area.

The thing that we need in order for your dream to become a reality is excellent user space frameworks, so I encourage you (and anyone else) to go build one or (better) find one you like and contribute.

> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads.

AIO has existed for a long time. A lot longer than io_uring.

I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach.

The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.

Yeah that’s it. Probably just wasn’t supported in the rust age library when I used it. Will double check.

Age is great. I used the rust crate to write an ftp server that encrypts the files before they hit disk (specific use case is having a drop box for my network scanner) and I love the simplicity and composability it provides.

One feature request: it would be awesome to have paraphrase encryption for age private keys.

The core framework (sched_ext) was written for general workloads and can quite beneficial. It lowers the costs of creating and iterating on schedulers quite a bit.

To be honest, when they started working on it I don’t think any of us expected for it to be a source of collaboration with gaming companies :)

(I’m a middle manager at Meta)

Latency and cache coherency are the other things that make this hard. Cache coherency can theoretically be resolved by CXL, so maybe we’ll get there that way.

Or perhaps more accurately, throughput is not the only goal.

A professor actually told us that freeing prior to exit was harmful, because you may spend all of that time resurrecting swapped pages for no real benefit.

Counterpoint is that debugging leaks is ~hopeless unless you have the ability to prune “intentional leaks” at exit

As a Meta employee, I consider this a victory against NIH :)

OpenAI's recruiting pitch was 5-10+ million/year in the form of equity. The structure of the grants is super weird by traditional big-company standards, but it was plausible enough that you could squint and call it the same. I'd posit that many of the people jumping to OpenAI are doing it for the cash and not the mission.

https://the-decoder.com/openai-lures-googles-top-ai-research....

Even if the laws are stupid, the killer feature you are describing is called crime.

IME, familiarity with /etc/nsswitch.conf and the rest of the nss stuff is very highly correlated with people who have seen some shit.

I still remember when I came to you with my “attachments shouldn’t live in the mssql database” plan and you said “yeah, probably, but doing it any other way would be a million times harder to maintain.” You were 100% right and I think of it often when I encounter someone who is about to do a similar dumb thing for “the right reasons.”

It was a joy to work with you as well :)

It's a bit harsh but I always feel like Fog Creek might be the cautionary tale in "what happens if you over hire for capability vs. your requirements?" I think that a less capable team would have never landed on the "let's maintain our own programming language" approach w.r.t. Wasabi.

As an aside, I do think that targeting Mono was the right thing to do for the universe, as it butterfly-effected tedu into writing weird and wonderful technical blog posts for the next ten years :p

I had an internship at Fog Creek and would add that it was probably the most friendly and harmonious place I worked, which made it very reality-show-incompatible (and very 21-year-old-me incompatible, I wasn't asked back lol). Certainly the representation of you as an asshole was ridiculous IMO.

(Since you're answering arbitrary Fog Creek questions) In retrospect, do you think it was a mistake to make kiln hg-centric at first?

> Mark Zuckerberg justified the move in a Facebook post: “Open source drives innovation because it enables many more developers to build with new technology. It also improves safety and security because when software is open, more people can scrutinize it to identify and fix potential issues.” [0]

I don't work in the space so I can't comment much, but my team (Server Operating Systems and similar) has been working with Linux, Systemd, Centos, etc for years for essentially the same reason. Working with the community forces you to build better stuff, because, in a healthy community, no one cares that a "Director from Meta wants it" which is the kind of thing that kills proprietary projects in the cradle all the time.

[0] https://www.vox.com/future-perfect/23817060/meta-open-source...

IMO this is equivalently scary to installing an arbitrary rpm.

The person to whom you are replying clearly meant (IMO) that you shouldn’t ask for more compensation or you will make people act defensive. Frankly, your reply reads a little defensive so maybe that’s not awful advice?

It also seems like this was a spontaneous initial conversation and not part of the process, so I’m not sure why you are suggesting that they made it up.

I actually disagree that this is the reason for the aversion. Instead I think it comes down to a couple of things:

1. Containers are commodity. For commodities, price wins first, then marketing. Docker is equally free (as in beer) and is far better marketed 2. There are a very low number of people working on systemd-nspawn and a very high number of people working on docker (and the ecosystem). 3. Dockerfiles and images are ubiquitous. They're easy to support from other run times, but if you're already in the ecosystem, what's the incentive to change?

It's also worth noting that io_uring has had at most 10-15 engineer-years worth of performance tuning vs. the many (?) hundreds of years that epoll has received. I work with Jens, Pavel, and others and can confidently say that low-queue-depth perf parity with epoll is an important goal to the effort

As an aside, it's great to see high praise from an spdk maintainer. One of the big reasons for doing io_uring in the first place was that it was impossible to compete in terms of performance with total bypass unless you changed the syscall approach.