xg15

The sentence itself makes no sense to me. If it's an illusion, then what does "illusion" even mean? Who perceives the illusion?

I'm with you if you want to say that the idea of a metaphysical entity that is separate from the body and not part of physical reality (a "soul"?) might be wrong - i.e. that our conscious experience is fully caused by physical effects, by the interactions of neurons in the brain and all the other machinery there - and that it might not even be an indivisible whole but might be "composed" of different components.

But that doesn't make any practical difference. We're still experiencing the world, have internal thoughts, memories, feelings, etc. Those things exist. In what form they exist is an interesting scientific question, but that's something different from dismissing them completely.

I have zero qualms about aborting a chat or resetting it to an earlier state, because any potential kind of consciousness that could exist would IMO have to take place during the inference passes, so there is nothing that amounts to "killing" an LLM.

I do try to not be a dick in chats, avoid intentionally subjecting the LLMs to mindfucks etc. This happens to be the far more effective way of working with LLMs (in terms of tokens needed to complete a task) as well.

I do not use character.ai or "virtual friend" services or anything else that tries to anthropomorphize LLMs even more than the training already does anyway. (This is more out of concern for myself than for the LLMs)

I think the whole debate runs to conclusions a bit prematurely, before we even have a good model to understand what happens in an inference pass of a billion parameter ANN with 100 layers of attention modules switched in series.

More generally, I don't like the attitude of encouraging people to be assholes. Suddenly we are in a situation where some people are building detailed simulations of torture and the people who are upset about it are the idiots?

(I still have no love for the EA guys or the rest of the TESCREAL circus. They always had a cultish vibe coming off them, but in recent years, all the masks seem to have fallen. They also consistently make the most un-empathic statements and observations, all in the name of "empathy")

I guess I mixed up different things here - I was thinking about "software dashboards" like Grafana or various Kubernetes frontends.

I didn't know this is still done for dashboards of industrial systems. I think then my annoyance is more that this has never been translated to other areas, like software systems.

One thing I notice is how much efforts had been spent to illustrate both the current status of the system, but also it's overall structure and how the two relate - i.e. the actual components of the facility.

Almost all of the dashboards here have some kind of graph, showing the wiring or plumbing between components, or in case of railway control, the railway lines. The graph acts like a map on which status indicators or switches can be placed.

Compare that to modern software "dashboards" which are full of complicated charts showing various metrics, but almost never show the structure of the system.

For a modern take on this, look at Factorio. Why can an industrial zone with a population of one look more lively in Factorio than an entire simulated city in Cities Skylines?

> and those that do ask for things they don't actually want, or are only wanted by very few people.

Then how do you know what people actually want?

> or do we want companies that get feedback from users on whether new changes are helping or hurting.

You don't get that feedback. The feedback you get is whether some telemetry KPI goes up or down. That's not the same as actual utility for the user.

But wow, they even want an account to read unlocked stories now...

If you just want to play with cool tech and make money, but for marketing reasons, you need to show some ostensible real-world reason why this tech should be developed, then "well, our chatbot could help lonely people with their relationship issues or give people in an abusive relationship a covert channel for help" is probably what comes to mind...

Runners up: Our AI startup will help the poor children in Africa and/or combat climate change (ignore the gas turbines, please).

It's starting with the exception before you even had a chance to understand the rule.

I guess this will become the frontline of the upcoming civil war in programming-land...

My point is that text-based interfaces - and voice is just a convenience around text - are not the best interfaces for a lot of tasks.

Unless you have a local model and the appropriate hardware for it, your agent is Somebody Else's Computer. Do you really want to send all your data and make your entire computing experience dependent on whatever OpenAI or Anthropic or whoever else is planning in this moment?

That's not even starting with inference time and token cost. Despite all the incredible advances in inference, it still takes more time than most non-AI computer functionality. Do you really want to wait a few minutes and pay money for something that you could also do with a few clicks fully locally on your PC?

But the most important thing: User interfaces. Right now, we're basically cramming everything you could possibly want to do at a computer into a chat interface. But there are lots of applications fir which specialized graphical interfaces are much more suitable. Why would you want to get rid of them?

Somewhat connected to that: Repetition. If you have to do the same task again every week or every day, it seems wasteful to ask the AI for it every time: Not just are you wasting a lot of time, energy and tokens, you're also at risk of getting inconsistent results, if the agent from today's session will interpret the requirements slightly differently than the agent from yesterday.

You can circumvent all those things by having the AI write you a custom app, but then using the app without AI to do the task.

Also, repeating a point from a similar thread: Software can have "intent" in the sense that it steers itself towards a predefined goal without having to be "alive" or "conscious" in any way. Some classic examples are thermostat control loops, navigation systems and chess engines.

So how to make the plugins remote-capable? You could write a massive virtualization layer that captures all system calls and forwards them to the remote - or, you could run the plugin on the remote and just pass the user commands and UI updates over the connection.

VSCode does the latter, so the nodejs runtime is where all the plugins are running on the remote.

(I understood the reason, I didn't say it was a good reason...)

Edit: another comment clarified it really goes both ways: The remote can use it to run code on the local machine as well, exactly what the "sandbox" pattern was supposed to prevent.

    Wander around the filesystem
    Edit arbitrary files
    Launch its own shell PTY processes
    Persist itself

Wait, could someone clarify which machine is being referred to here?

So in the author's setup, he runs VSCode (i.e. the front-end) on his dev laptop, which he wants to keep free of direct LLM access.

VSCode connects via ssh to a dedicated "sandbox" machine on which the LLM will be free to do whatever it wants (mostly).

VSCode realizes this the Microsoft way, by using the ssh connection to install VSCode Server on the sandbox machine - the "backend" - and communicating through it via a websocket connection.

So then, what happens? If the websocket connection allows the front-end to run arbitrary commands on the sandbox machine, this wouldn't be very exciting: The front-end already has an ssh connection and a massive server process that can do the same - and the entire purpose of the sandbox machine is to run arbitrary, untrusted commands without harm.

But the article says the websocket connection goes "back to your running VSCode front-end". So does that mean things are reversed? I.e. the agent/harness runs in the server on the sandox machine but for some reason has this websocket connection that also lets it run arbitrary commands on the dev laptop?

Is that it? That would be truly insane!

AFAIK, the first line is supposed to be the a very brief summary, while the other lines may contain additional information and can sometimes make up a pretty long text. Lots of tools make use of this convention and only show the first line if no detail information is needed.

Most tools that show the history only show the first line of each commit.

But I agree with you, this still assumes that long commit messages are rare and not that almost every commit has a huge message. Also, "long" doesn't mean you should put a novel in there.

Huh? I guess that depends on the exact definition of "classification", but I think the bulk of basic classification tasks we make every day to make sense of our surroundings, such as object recognition is definitely done using system 1. So is higher-level "stereotyping" or anything you could described with "I know it when I see it".

Because those responses can be incorrect or even harmful, you would sometimes make use of system 2 to correct them - but that doesn't change that the initial response is from system 1.

About Jev:

> We didn't train a model with Reinforcement Learning for Calibrated Decisions (RLCD) to calibrate the decisions and probabilities (even though they are not always correct).

Only 99% correctness! Borderline unusable!

About their model:

> It classifies: it gets a prompt with choices and outputs probabilities.

You want numbers, it gives you numbers! What more could you want?

If prompts are specifications for a change of the system's behavior, then it seems natural to manage them as changes and not as resources.

This would also keep them in the right "historical context" of the repo and avoid the "prompt rot" you were talking about.