jdonaldson

Are you saying that all we have to do is hire a few boring Indians to cancel out one DEI hire? This is huge news!

That is a pretty badly overloaded word for it, and I didn't even pay much attention to the notion of "rank" anyways. I'm mainly interested in how the text is represented with bit vectors. It's very reminiscent of how vector math plays out in other ML domains, but I would bet that many people working with text have never heard of it.

I think what's changed since this was posted in 2011 is the emergence of embeddings and the need to take advantage of its higher dimensional space. While embeddings expose more underlying structure that can be used for tensor math, ranking systems often are still good ol' trees. This project to me points at a new major "hinge" of information architecture.

I remember the concept of the virtual dom had to be explained very carefully so people understood why it led to more efficiency when rendering (e.g.) scrolling content.

Is there a technical trick that one takes that can beat a virtual dom use case? What is it?

Macros give their own kind of power, and it's a tough call to give that up for runtime hot-reloading. Languages like Haxe have macros, but also have hot reloading capabilities that typically are supported in certain game frameworks. You probably don't want to mix them together, but it's also a good development process to have simpler compilation targets that enable more rapid R&D, and then save macros for larger/more comprehensive builds.

https://haxe.org/manual/macro.html

https://github.com/RblSb/KhaHotReload

It's true, but one has to ask "who made woke-ism into a mind virus?". I think alt-right media say that BLM/Liberals have, but in reality it went viral when it became a favored perjorative and performative act by the alt right, a sort of gradual straw man argument that became true by its own belief. That's the textbook definition of a virus if I've ever seen one.

I would have to refute the notion that wokeness is a mind virus. "Stay Woke" has a much deeper origin in African American culture, and it refers to the fact that one needs to stay vigilant about another's intentions.

The implicit message is that the "us" cannot trust the "they", and writers like Paul Graham show the reason why: Any attempt at social change can easily be labeled a virus by capitalists if it does not produce greater prosperity. It's the same prosperity that has poisoned the earth, so I hope they have answers there too.

I’d say it is, although it’s not really a document but a stream of tokens. The intuition is that the model is able to build abstractions from that stream. Some of the abstractions are related to goal states of the training metric. However, it’s easy to imagine how some of the goal states pertain to the model’s survival in the “wild”, and part of the abstraction that the model captures for this goal is a reflection of itself.

Just wanted to point out that the notion of a "document" is also an illusion to the LLM. It's processing a sequence of low dimensional spaces into another sequence of low dimensional spaces. The input spaces preserve aspects of content similarity based on co-occurrence. The model learns to transform these spaces into higher order spaces based on the outcome of training.

You couldn't say that the model has a singular sense of self, but it certainly has been trained on data that allows it to mimic it in short spurts, and mimicry is what humans do to learn more complex/abstract tasks. The training goal is not to learn how to "be", but rather to learn how to "do" the parts necessary to continue existing.

"Fake it till you make it" is really all that's required to exist in the world.

If that’s the case, why was it a bang instead of a slow leak?

"Convincing you is more important than informing you"

Always a pass from me, gets things off on the wrong foot right away.

One must imagine Atlas shrugging.

Are you saying that they cant explain the contents of machine code in human readable format? Are you saying that they can’t be used in a system that iteratively evaluates combinations of inputs and check their results?

It’s kind of surprising that these haven’t just been reverse engineered yet by language models.

Curves always flatten when faced with a constraint and/or risk. In this case it's surface area and climate variability.

Australia's electricity consumption in 2021 was approximately 265 TWh (terawatt-hours) per year. Napkin math works out to:

Annual energy production of one 2 MW reactor: 2 MW * 8760 hours/year = 17,520 MWh/year = 17.52 GWh/year.

Number of reactors required: 265,000 GWh/year / 17.52 GWh/year = ~15,130 reactors.

This is of course a silly computation for scaling a research grade molten salt reactor. 300+ MW reactors knocks things down to a few dozen.

Now, how about if we apply this same requirement to solar power? How many solar panels will we need to get the same power gain?

Total energy required: 265,000 GWh/year = 265,000,000 MWh/year = 265,000,000,000 kWh/year. Number of panels needed: 265,000,000,000 kWh/year / 584 kWh/year per panel ≈ 453,767,123 panels.

I know Chinese panels are cheap and all, but the notion that Australia will maintain a network of 450M panels is silly, and this is just to replace what we have now, not what is needed in the future.

Dropping a link/endorsement for quarto : https://quarto.org/

It's a static site/document generator that supports bibliographies, jupyter notebooks, and good old fashioned markdown. It works well for complex academic resumes and CV's, as well as blogs and library documentation.

I use a template for my website : https://jjd.io/

github repo : https://github.com/jdonaldson/jjd.io

Also, if anyone has any positive/negative comments on my site, let er rip! I'm still working on it.

These work great if you have a seldom used door for your furnace and/or ac : https://a.co/d/0h8uGmLQ

Wind/solar can’t make the order of magnitude jump we need in power.

Western Australia and Queensland have a good deal of Monazite sands that one can extract Thorium from. This would greatly reduce the need for more complex safety and disposal regulations necessary with larger fission reactors.

A small college in Texas was able to build and run a Thorium reactor, so it can't be that far out of reach for a small country : https://youtu.be/kCrrrmh-eMo?si=NChlWACDXajKm9hr

PCRE2 is specified through its implementation, which has so many edge cases and special flags that most people can't reason about what kinds of problems it could cause.

I really wish more people used PEG parsing. I wrote a library for it in Haxe that was surprisingly fast despite being interpreted : https://www.youtube.com/watch?v=CtNQvjyioGQ

Most of the money there is in hospitals. I don't think it does the economy any good to tax those. Love to hear arguments the other way.

It's hard watching venerable institutions rot into "just avoid administerial short term blame" death loops. You have to have skin in the game, not just hire a temporary manager for it.

Emotions let us reason about ourselves, which rationality alone can't do for us. Because, in a purely rational sense, the "self" doesn't really exist.

I’m finding smaller vector databases can be almost ephemeral if you avoid parsing : https://hushh-labs.github.io/hushh-labs-blog/posts/you_dont_... I can retrieve a query, encode the embedding, load the vector store, calculate KNN, and return rendered results in under 50 milliseconds. It’s even faster if you simply cache the vector store. Really interested in this space and hoping to hear more ideas.

If you can’t undermine someone’s argument, undermine their nationality. American tech culture doesn’t do this as much as it should, perhaps because we know eventually those folks wake up.

Can you put the colon on the blockchain? I might know a guy.

Strongly suggest you check out Haxe. Been my favorite open source community to be part of.

"Uncertainty is an intrinsic aspect of life, a constant element that humans have tirelessly sought to navigate and comprehend."

I can appreciate people doing things from the heart, but in that case, it better be more poetic than a BBC documentary.

This paper is a far better argument for psycho-acoustic components of sound : https://www.nature.com/articles/nature04485/