osmarks

I was briefly looking into using SMT for Minecraft autocrafting, but it turns out you can do integer linear programming and the mapping is easier.

This is sort of true currently, but extrapolate the trend.

Also, I don't use ChatGPT to rewrite blog posts and don't like people who do. Its style is annoying and if ChatGPT is doing content I might as well ask it whatever you asked it myself directly. For code I do not care much so long as it works.

Artists correctly realized the threat to their future economic viability and made up reasons it was morally bad. Programmers are currently stuck in an earlier stage, insistent that it can never replace them because [various things].

Common Crawl is petabytes. Anna's Archive is about a petabyte, but it includes PDFs with images.

You could just run a local LLM over every document and ask it "is this related to this query". I don't think you actually want to wait a week (and holding all the documents you might ever want to search would run to petabytes).

(the reasonable way is embedding search, which runs much faster with some precomputation, but you still have to store things)

There is at least one organization doing actual embedding-based search (Exa). I wrote about this a bit: https://docs.osmarks.net/hypha/osmarks.net_web_search_plan_%....

Most of these are just an EPYC server platform, some cursed risers and multiple PSUs (though cryptominer server PSU adapters are probably better). See https://nonint.com/2022/05/30/my-deep-learning-rig/ and https://www.mov-axbx.com/wopr/wopr_concept.html.

They couldn't have built it on anything but UDP because the world is now filled with poorly designed firewall/NAT middleboxes which will not route things other than TCP, UDP and optimistically ICMP.

The C specification limits programs to addressing a finite amount of memory, though it can be made arbitrarily large by an implementation. The Python specifications do not imply this though real interpreters do.

Yes. C is not Turing-complete even in theory. Other languages are. It doesn't especially matter.

You can't implement a Python interpreter with access to infinite memory in C as specified. That is the point.

CommonMark mostly fixes this.

Preserving the semantic content is helpful if you think you might want to switch the rendering later.

I solve this for my usecases with custom Markdown rendering which accepts a few new block elements (via a markdown-it plugin). https://github.com/osmarks/website/blob/master/src/index.js

Python-the-language can be Turing-complete even if Python-as-actually-implemented is not.

C is indeed not Turing-complete for more or less this reason.

Crowdstrike should have higher testing standards, not every random back-office process.

> When I first read about Darwin Machines, I looked up "evolutionary algorithms in AI", thought to myself "Oh hell ya, these CS folks are on it" and then was shocked to learn that "evolutionary algorithms" seemed to be based on an old school conception of evolution.

I think a lot of the genetic algorithms people do implement recombination-like things. Most of the things operated on aren't really structured like genomes so it makes less sense there.

> But intelligence like you or I's operates in an unconstrained problem space. I don't think you can apply gradient descent because, how the heck could you possibly score a behavior?

> This is where evolution excels as an algorithm. It can take an infinite problem space and consistently come up with "valid" solutions to it.

Evolutionary search also relies on scoring. Genetic algorithms on computers hardcode a "fitness function" to determine what solutions are good and should be propagated and biological evolutionary processes are implicitly selecting on "inclusive genetic fitness" or something. You can't apply gradient-based optimizers directly to all of these, though, because they are not (guaranteed to be) differentiable. There are lots of ways to optimize against nondifferentiable functions in smarter ways than evolutionary search, and these come under "reinforcement learning", which does work but is generally more annoying than (self-)supervised algorithms.

> I think Darwin Machines might be able to explain "animal intelligence". But human intelligence is a whole other deal. There's some incredible research on it that is (as far as I can tell) largely undiscovered by AI engineers that I can share if you're interested.

As far as I know human brains are more or less a straight scaleup of smaller primate brains.

> A Darwin Machine uses evolution to produce intelligence. It relies on the same insight that produced biology: That evolution is the best algorithm for predicting valid "solutions" within a near infinite problem space.

It seems to be suggesting that neuron firing patterns (or something like that?) are selected by internal evolutionary processes.

I don't think this is true as stated. Evolutionary algorithms are not the most efficient way to do most things because they, handwavily, search randomly in all directions. Gradient descent and other gradient-based optimizers are way way faster where we can apply them: the brain probably can't do proper backprop for architectural reasons but I am confident it uses something much smarter than blind evolutionary search.

Mistral and Meta release "instruct" (RLHF) and not-instruct models. The non-instruct ones are in fact non-RLHF, pretraining-only ones (though they probably have ChatGPT-ish text in the dataset nowadays, and Meta might have done some extra training on evals...).

The Google research was based on OpenAI research from 2021, though.

(https://arxiv.org/pdf/2209.06794.pdf page 20.)

I think the SigLIP models' dataset (WebLi) includes OCRed things too, so they have very good text understanding. I tested a bunch of things for my own meme search engine.

People are perfectly happy to give other people specific gifts, which are even more constraining than gift cards. This doesn't seem that surprising.

AX201s have always worked perfectly for me.

https://github.com/facebookresearch/faiss/wiki/Vector-codecs has some good things available too.

I use the Mistral-7B base model for a few things via fewshot prompts, but not really for anything production, just some fun toys.