samsartor

SketchUp uses stencil shadows! Although we have more modern options, it is part of the look.

100,000 Einsteins in your basement would be amazing. You'd have major breakthroughs in many fields. But at some point the gains will be marginal. All the problems solvable by shear intellectual labor will run dry, and you'll be blocked on everything else.

> We humans have the ability to internalize the world and conduct "what if's" in our heads; we can solve many problems thousands of times faster than natural selection.

I don't know for sure whether superintellegence will happen, but as for the singularity, this is the underlying assumption I have the most issue with. Smart isn't the limiting factor of progress, often it's building consensus, getting funding, waiting for results, waiting for parts to ship, waiting for the right opportunity to come along. We do _experiments_ faster than natural selection, but we still have to do them in the real world. Solving problems happens on the lab bench, not just in our heads.

Even if exponentially more intelligent machines get built, what's to stop the next problem on the road to progress being exponentially harder? Complexity cuts both ways.

I think this may actually be true in a sense. A lot of good science gets published but never wildly noticed, just because good science is rarely sexy.

Sure! I'm just disagreeing that "C is anarchy because anyone at any point can do anything". You can't do anything, Linus will eventually yell at you in an email. To me, the only philosophical difference is that Rust wants to automate Linus

It is a good video, but I'm not at all convinced that a philosophical "authoritarian" vs "anarchy" difference between Rust and C actually exists. C programmers work through all sorts of constraints and rules on how to correctly use a system, same as anyone else. Heck, there are hundreds of pages of kernel documentation explaining how developers are expected to use the various locking subsystems. Does that make C authoritarian? I don't think so... that's just a fact of programming. The details matter.

IMO the only real cultural difference in Rust is that you are expected to explain those constraints through the language of the type system, not just in English. That's a lot of work up front but it also gives you way more automation down the line (eg checking that you used a mutex correctly via rustc rather than through emails to Linus Torvalds). Some people definitely take it too far, and blow up their code with endless incomprehensible traits. But the islands aren't incompatible, it just takes work and skill to bridge them.

This was exactly my experience too

Overall you guys do an amazing job! I'm just griping XD

And actually you are right, it didn't throw at all, just logged a nice warning! Despite me being annoyed that I had to go fix my mistake, the API has caused me 10x fewer issues than any other python lib I've touched

I've been using rerun for the past year or so instead of tensorboard/wandb/etc for logging my ML training runs. I really like being able to just throw enormous amounts of arbitrary data back to my own laptop without worrying about cloud storage!

I will say the (not so recent) API change to "components" and "archetypes" hasn't clicked for me yet. Obviously I could sit down and figure it out. But it would be a lot nicer if the API could just ingest whatever damn types I throw at it, rather than raising an error 20-minutes in because I wasn't polite enough. The old API felt basically magic.

The community has been very supportive of the gccrs (https://github.com/Rust-GCC/gccrs) project, which is the main project to write a Rust compiler written in C.

I have run into a lot of similar problems writing a state management framework for Rust (wip at https://gitlab.com/samsartor/hornpipe) and at one point despaired that I would have to abandon Rust and make a whole new reactive language. But the last couple years I've got it working really nicely with weak references and software transactional memory. Every reference is `Copy + Send + Sync + 'static`. And you can mutate objects by having a transaction make a local bitwise copy, which will get atomically merged and swapped on commit. The old copy gets kept around for undo/redo purposes. I've still got a boatload of algorithmic puzzles to solve to provide all the MVP features, which will take a while because it isn't my full-time job. But the details seem technically sound.

I did write a blog post about some of my design thoughts, although it doesn't dig into the technical guts: https://samsartor.com/guis-3/

My previous laptop was a Chromebook running Linux+Coreboot. Unfortunately the usual Tianocore UEFI BIOS people use had some bugs in the nvme and keyboard drivers, which I gave up fixing or working around (at the time). Obviously Linux had working drivers because that's all ChromeOS is, so we setup a minimal Linux install as the Coreboot payload in the firmware flash, and I wrote a little Rust TUI to mount all visible partitions and kexec anything that looked like a kernel image. It worked like a charm and had all kinds of cool features, like wifi and a proper terminal for debugging in the BIOS! Based on that experience I don't see any reason why we don't just use Linux direct instead for everything. Why duplicate all the drivers?

The code is here although it hasn't been touched it years: https://gitlab.com/samsartor/alamode-boot

https://github.com/mcabbott/Tullio.jl is my favorite idea for extending einsum notation to more stuff. Really hope numpy/torch get something comparable!

My favorite has always been "I can't log in when I stand up" https://www.reddit.com/r/talesfromtechsupport/comments/3v52p...

Well that is the whole trick. ML models ideally generalize from the training inputs to whatever new inputs show up during inference. For example, a vision model should recognize an image of a dog as a dog even if that exact image was not trained on. But that generalization always has limits. Usually score will decrease substantially the further "out-of-domain" the inputs are. So this model works fine when running a randomly generated dungeon it has never seen, but not when running a set of game rules it has never seen.

They do participate pretty heavily in ML research from what I've seen. To continue your metaphor, they try to invent as many gold digging techniques as possible which exclusively work with their own shovels and buckets.

As a person midway through a PhD, I personally like these sorts of competitions. My dissertation is so all consuming that I have to be able to make fun of it a little bit! Life would just be too depressing otherwise.

> Intel and AMD don't use x86 cores but [...] instead translate the x86 code to RISC code on the fly

I used to say this all the time and I've been informed that it's something of a misunderstanding. For example, most RISC-V processors also decompose instructions into multiple μops: https://docs.boom-core.org/en/latest/sections/execution-stag...

So it isn't like there is a literal RISC processor inside the x86 processor with a tiny little compiler sitting in the middle. It's just that the out-of-order execution model requires instructions to be broken up into subtasks which can separately queue at the core's various execution units. Even pipelining instructions still wastes a lot of silicon (while you're doing an integer add the floating point ALU is just sitting there, bored) so breaking things up this way greatly improves parallelism. As I understand it, modern μop-based processor cores can actually have dozens of ALUs, multiple load/store units, virtual->physical address translation units, etc all working together asynchronously to chug through the incoming instructions.

In terms of well-written fun I would rank "Murderbot Diaries" very highly, alongside any of the John Scalzi books I've read. For similar levels of ridiculous (and trying-not-to-be-but-kinda-still sexist) look no further than "Off to Be the Wizard". That one is best if you recruit a few programmer buddies to also read it, so you can all compare each other's resulting self-insert fan fiction. I'll also occasionally reread the "Synchronicity Wars" series if I'm feeling ill and want something sorta kooky.

Yah I was thinking of Diaspora a bit when I included the word "few" lol. Although even in that case Egan is willing to hand-wave away technology in one place if it is the most narratively convenient way to move the story around to some other technological or philisophical conundrum he's after in particular (like how the computer technology of a Polis is basically just magic) or if it makes the story more accessible to a human audience (like how the "gestalt" sense is usually written as straightforward human-style vision even if it is ostensibly some distant derivative there of). If we do start uploading our brains into computers I don't think I'm going to give Greg Egan credit for "predicting" it, but he did take good advantage of that trope to explore some really cool ideas.

IML the point of sci-fi is to ask interesting questions. Very rarely is that question "what is the absolutly most plausible thing that could happen next century?”.

I do think predictions are an unavoidable side-effect of writing science fiction, but those predictions are almost always in service to those purposes you listed. Stephenson must think nanotech book AI is more plausible as a form idealized childhood education than say, magic fairies. But in the end he relies on our suspension of disbelief to get on past that to the big questions of nurture vs nature and the quality of human intelligence.

I haven't thought too much about Hiro potentially being an unreliable narrator, and it definitely puts a different spin on things! In that case I certainly think the specific design of the technologies in Snow Crash is motivated by that narrative need to make Hiro's mundane life into something exciting, rather than the literally mundane motivations behind technology in the real world. Assuming anyway that we aren't careening into a future were all technology is driven primarily by hype... that's a scary thought.

Yah "mostly" might have been a bit strong on my part. I think Neil Stevenson is capable of both expressing interesting ideas and shoehorning in great excuses for action set pieces at the same time. Certainly any good science fiction can inspire different interpretations by different people!

I think the framing around these sorts of discussions always annoys me because few sci-fi books (and certainly no Neil Stephenson books) are primarily written as predictions. They are written as stories, with technology serving narrative above all else.

Snow Crash in particular uses the metaverse mostly as an excuse to include sword fights and motorcycle/monorail chase scenes. Fantastically fun in my opinion! But that motivates all kinds of choices (making the Internet a literal place with a street grid and real estate, where people can get chopped up by swords) that real XR tech has no particular technical use for. And I would implore any engineers using Snow Crash as an inspiration to consider how absurd it would be to take all the world's most sophisticated technology and dedicate it to gratifying personal power fantasies. It starts with the almost pornographic display of advanced weaponry and logistics deployed to deliver a pizza, and just gets more gloriously ridiculous from there. The main character is named "Hero Protagonist". The main antagonist has a nuke strapped to his motorcycle. Take a hint!

Anyway, I am happy Diamond Age gets a call-out because it is by far my favorite of Stephenson's novels. And I think the Young Ladies Illustrated Primer is one of the all-time most interesting technological plot contrivances (the Imago machine, game/civilization of Azad, and Shrike all providing strong competition). But the technical constraints/capabilities of the Primer have almost nothing to do with realistic limitations/advantages of AI technology, and everything to do with getting the right characters into the right places at the right times. We need a Miranda to provide the Primer's voice so that Nell can have some kind of human connection in the end, and we need Miranda to be paid anonymously so that Nell won't get that human connection too soon. The Primer is a language model not a robot so that Nell will have to solve problems on her own. Yet she can learn Kung Fu from a language model because we need a few action scenes. I think really the interesting question posed is "Can a person grow up to be influential given no resources except a perfect education?" not so much "Can a language model provide a perfect education?". Many characters in Diamond Age seem to agree with the former notion, but in the end it (SPOILER) gets shot down when Nell needs control of an literal army to come out on top.

This is still pretty idealized. In my experience the pytorch abstraction leaks _constantly_. In particular, any interesting ML project probably pulls in at least a few dependencies with custom pytorch extensions somewhere.

Reflecting a little bit more I don't think it was replit's fault, per-say. But that change should have been made together with a larger adjustment to the program. Like adding a class/unit in the style of [the missing semester](https://missing.csail.mit.edu/) to make sure people came away with a good range of intuitions.

At my old university one of the intro classes moved all their students to Replit. Which might have been fine in isolation, but I think it created a year of students who struggled endlessly to use gcc, Makefiles, debuggers, or local tooling in general. Starting your CS journey with the local command line seems to put you into the perfect headspace right off the bat, even if it is a little slower to get going.

Looks more like 4 s/iter which is actually quite good for CPU inference in Automatic111. It is an apples-to-oranges comparison to the LCM models used by fastsdcpu (since those can get away with fewer sampling steps) or to the quantized models favored by apple (since they use totally different underlying compute capabilities). Currently automatic111 and other popular GUIs are pretty focused on users with beefy GPUs that can sample images in seconds, so they haven't yet implemented some of these really significant CPU-friendly speedups. But we'll get there!

This is a bit of a spoiler, but if you cheat your way through the videos it does go all the way to a trillion.

I helped make a really cursed RISC-V version of this for a class project last year! The idea was to first compile each program to WASM using clang, and lower the WASM back to C but this time with all opcodes implemented in terms of the RISC-V vector intrinsics. That was a hack to be sure, but a surprisingly elegant one since 1. WASM's structured control flow maps really well to lane masking 2. Stack and local values easily use "structure of arrays" layout 3. Heap values easily use "array of structures" layout

It never went anywhere but the code is still online if anyone wants to stare directly at the madness: https://gitlab.com/samsartor/wasm2simt