gradys

I do think much of the kind of software we were building before is essentially solved now, and in its place is a new paradigm that is here to stay. OpenAI is certainly the first mover in this paradigm, but what is helping me feel less dread and more... excitement? opportunity? is that I don't think they have such an insurmountable monopoly on the whole thing forever. Sounds obvious once you say it. Here's why I think this:

- I expect a lot of competition on raw LLM capabilities. Big tech companies will compete from the top. Stability/Alpaca style approaches will compete from the bottom. Because of this, I don't think OpenAI will be able to capture all value from the paradigm or even raise prices that much in the long run just because they have the best models right now.

- OpenAI made the IMO extraordinary and under-discussed decision to use an open API specification format, where every API provider hosts a text file on their website saying how to use their API. This means even this plugin ecosystem isn't a walled garden that only the first mover controls.

- Chat is not the only possible interface for this technology. There is a large design space, and room for many more than one approach.

Taking all of this together, I think it's possible to develop alternatives to ChatGPT as interfaces in this new era of natural language computing, alternatives that are not just "ChatGPT but with fewer bugs". Doing this well is going to be the design problem of the decade. I have some ideas bouncing around my head in this direction.

Would love to talk to like minded people. I created a Discord server to talk about this ("Post-GPT Computing"): https://discord.gg/QUM64Gey8h

My email is also in my profile if you want to reach out there.

> How can we explain the predictions of a black-box model? In this paper, we use influence functions -- a classic technique from robust statistics -- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction.

I make dramatically more than my parents did when they were buying their first house around my age, but as a fraction of my income and savings, housing costs vastly more.

(It’s also not clear that the skilled trades jobs Gen Z apparently doesn’t want actually pay more than the alternatives. See other comments on this post.)

The thing Roam-style note taking software accomplish is letting you be productive by organizing bottom-up rather than being forced to decide what notebook/tab/folder any given new thing needs to go in. It's very pragmatic and low-bullshit in that sense. Being able to make references to other nodes (rather than only to other pages) is also very pragmatically useful in my experience (e.g. refer to a TODO I wrote in my weekly TODO list when writing my daily TODO list, so when I check off a daily TODO it's also checked off in the weekly list), and I never saw anything like this in pre-Roam tools like OneNote.

I get that other people operate differently and are obsessively "gardening" their graphs and searching around for connections to make. I don't do any of that. Doesn't seem valuable for me in particular, though maybe if I were a professional writer, I could imagine it being useful.

Neural network models seem to encode an approximate notion of quantity in their representations. This paper is pre-GPT-3, but I would think the larger training set and larger model capacity would help the model learn quantity more easily.

I feel like ultimately you want both. Explicit links are a useful navigation affordance with nice properties that spatial embeddings won't give you (e.g. you can explicitly establish a link between things that are not similar according to the embedding space).

More important than that though, explicit links let you train the embedding model to understand the dataset the way the user does. All of these embedding models are trained on graphs (word or sentence cooccurence graphs, parent/child comment graphs on social media, etc.). The graph structure in something like Roam can provide training data for updating and adapting the embedding space to the specific knowledge context in which it's used.

Conversely, if you have an embedding representation of your knowledge base, you can use that to suggest explicit links. The embedding space is the dense dual to the sparse graph of explicit links in something like Roam. It's a fully connected weighted graph rather than a sparse unweighted graph.

Maybe this system is meant to only focus on the spatial embedding representation. That makes a lot of sense. A fully-fledged version of this vision though IMO should include a bridge between these two dual representations.

To your edit: the fact that the entity in question is a government is the difference here in my opinion.

I think what it means to understand language is to be able to generate and react to language to accomplish a wide range of goals.

Neural nets are clearly not capable of understanding language as well as humans by this definition, but they're somewhere on the spectrum between rocks and humans, and getting closer to the human side every day.

I can't help but think that arguments that algorithms simply don't or can't understand language at all are appealing to some kind of Cartesian dualism that separates entities that have mind from those are merely mechanical. If that's your metaphysics, then you're going to continue to find that no particular mechanical system really understands language, all the way up to (and maybe beyond) the point where mechanical systems can use and react to language in fully all situations that humans can.

No, you bumped your net income. Revenue is unaffected by the depreciation change. Revenue is up 61%.

Also, nobody would be fooled by an accounting trick like this. Analysts routinely compute EBITA, earnings before interest, taxes, and depreciation, exactly for this reason - filtering out the more purely financial/virtual expenses that are less informative for understanding the core business.

They likely did it because you're required to report things like depreciation in a way that reflects reality. There could be tax implications for instance, since you can count depreciation expenses against your taxable earnings (though often companies maintain separate depreciation accounting for financial reporting vs taxes due to the different rules for each).