TuringTest

However there are two fundamental problems to computational predictions. The first one obviously is accuracy. A model is a compressed memorization of everything observed so far; a prediction with it is just projecting into the future the observed patterns. In a chaotic system, that goes only so far; the most regular, predictable patterns are obvious to everybody and give less return, and the chaotic system states where prediction would be more valuable are the less reliable. You cannot build a perfect oracle that would fix that.

The second problem is more insidious. Even if you were able to build a perfect oracle, acting on its predictions would become part of the system itself. That would change the outcomes, making the system behave in a different way as it was trained, and thus less reliable. If several people do it at the same time, there's no way to retrain the model to take into account the new behaviour.

There's the possibility (but not a guarantee) to reach a fixed point, that a Nash equilibrium would appear where such system becomes into a stable cycle, but that's not likely in a changing environment where everybody tries to outdo everyone else.

For some reason he doesn't like doing mathematical demonstrations so he shuns the practice of doing them, and invented a new word to describe that way of using formal systems.

https://en.wikipedia.org/wiki/Formal_system

I.e. by making sure that they're doing the right thing.

So, absolutely necessary and essential?

In order to get the machine out of trouble when the unavoidable strange situation happens that didn't appear during training, and requires some judgement based on ethics or logical reasoning. For that case, you need a human in charge.

Also Wikipedia has developed an editorial line of its own, so it's normal that edits that go against the line will be put in question; if that happens to you, you're expected to collaborate in the talk pages to express your intent for the changes, and possibly get recommendations on how to tweak it so that it sticks.

It also happens that most of contributions by first timers are indistinguishable from vandalism or spam; those are so obvious that an automated bot is able to recognize them and revert them without human supervision, with a very high success rate.

However if those first contributions are genuinely useful to the encyclopedia, such as adding high quality references for an unverified claim, correcting typos, or removing obvious vandalism that slipped through the cracks, it's much more likely that the edits will stay; go ahead and try that experiment and tell us how it went.

And that's a good thing, 'cause it means they're living to their standards.

It's the Eternal September of our generation, and it's not recognised enough as such. Before that, the internet was a different place.

https://en.wikipedia.org/wiki/Gamergate

No, I'm saying than 'alignment' is a concept that doesn't help to solve the problems that will appear when the machine ultimately breaks; and in fact makes them worse because it doesn't account for when it'll happen, as there's no way to predict that moment.

Following your metaphor of criminals: you can control humans to behave following the law through social pressure, having others watching your behaviour and influencing it. And if someone nevertheless breaks the law, you have the police to stop them from doing it again.

None of this applies to an "aligned" AI. It has no social pressure, its behaviours depend only on its own trained weights. So you would need to create a police for robots, that monitors the AI and stops it from doing harm. And it had better be a humane police force, or it will suffer the same alignment problems. Thus, alignment alone is not enough, and it's a problem if people depend only on it to trust the AI to work ethically.

Correction: if your training data and the input prompts are sufficiently moral. Under malicious queries, or given the randomness introduced by sufficiently long chains of input/output, it's relatively easy to extract content from the model that the designers didn't want their users to get.

In any case, the elephant in the room is that the models have not been trained with "sufficiently moral" content, whatever that means. Large Language Models need to be trained on humongous amounts of text, which means that the builders need to use a lot of different, very large corpuses of content. It's impossible to filter all that diverse content to ensure that only 'moral content' is used; yet if it was possible, the model would be extremely less useful for the general case, as it would have large gaps of knowledge.

Yeah, I agree that alignment is a desirable property. The problem is that it can't really be achieved by changing the trained weights; alleviated yes, eliminated no.

> we can greatly reduce the probabilities they will show it

You can change the a priori probabilities, which means that the undesired problem will not be commonly found.

The thing is, then the concept provides a false sense of security. Even if the immoral behaviours are not common, they will eventually appear if you run chains of though long enough, or if many people use the model approaching it from different angles or situations.

It's the same as with hallucinations. The problem is not that they are more or less frequent; the most severe problem is that their appearance is unpredictable, so the model needs to be supervised constantly; you have to vet every single one of its content generations, as none of them can be trusted by default. Under these conditions, the concept of alignment is severely less helpful than expected.

The model stores all the content on which it is trained in a compressed form. You can change the weights to make it more likely to show the content you ethically prefer; but all the immoral content is also there, and it can resurface with inputs that change the conditional probabilities.

That's why people can make commercial models to circumvent copyright, give instructions for creating drugs or weapons, encourage suicide... The model does not have anything resembling morals; for it all the text is the same, strings of characters that appear when following the generation process.

Again, you're talking from a moral standpoint, but it's not practical. Who's going to stop Google or other corporations from tracking DMCAs the current way?

> Why does scale matter?

Because of resources. Any defined process needs resources to be implemented; law enforcement is no different.

Google provides services at scale by means of automating the shit of them. The only way to identify legit from fake claims at that level is to also create an automated resolution process, with the results we see.

You may want to limit Google size by forcing them to perform human reviews for all their customer service interactions; but again, how are you going to force them into compliance? You'd need a US judiciary system the size of Google to do it.

For that, having elegant and simple software is not needed; getting features fast to try out how they work is the basis of the pleasure, so having to write every detail by hand reduces the fun.

I explicitly recognized that the practice of accounting as a discipline keeps using traditional concepts that are culturally adequate for its purpose. What my posts are pointing out is that the original reason for double-entry records, which was having redundant data checks, is no longer a technical need in order to guarantee consistency because computers are already doing it automatically. From the pure data management perspective I'm analysing, that's undeniable.

The most obvious consequence of this analysis is that traditional bookkeeping is no longer the only viable way of traking accountability; new tools open the possibility of exploring alternative methods.

Compare it to music notation; people keep proposing new ways to write music scores, some of them computer assisted; and though none of them is going to replace the traditional way any time soon, there are places where alternative methods may prove useful; such as guitar tablature, or piano-roll sheets for digital samplers. The same could be true for accounting (and in fact some people in this thread have pointed out at different ways to build accounting software such as the Resources, Events, Agents model.)

https://en.wikipedia.org/wiki/Resources,_Events,_Agents

In the past this was problematic because you missed error-checking by redundance, but nowadays you can trust a computer to do all the checking with database transactions (which are more complex checks than double entry, though they don't need to be exposed into the business domain). Any tracking that you'd want to do with double-entry accounting could be done by creating single-entry accounts with similar meaning, and registering each transaction once, if you know that you can trust the transaction to be recorded correctly.

Double-entry bookkeeping was from its inception an error-correction code that could be calculated by hand.

Modern databases contain much more powerful error correction methods in the form of transactional commits, so from a pure technical point, double-entry bookkeeping is no longer needed at all; that's why programmers have a hard time understanding why it's there: for us, when we store a value in the DB, we can trust that it's been recorded correctly and forever as soon as the transaction ends.

The thing is, cultural accounting still relies on the concepts derived from double entry bookkeeping as described in the article; all those assets and debts and equity are still used by the finances people to make sense of the corporate world, so there's no chance that they'll fall out of use anytime, at least 'in the context of a company' as you out it.

Now would it be possible to create a new accounting system from scratch that relied on DB transactions and didn't depend on double entry? Sure it can, in fact crypto coins is exactly what happens when computer engineers design a money system unrestricted from tradition. But in practical terms it still needs to relate to the traditional categories in order to be understood and used.

They can, but the probability of two opposite errors of exactly the same magnitude is much lower than of any individual random error.

It's the same as with any other error-correction encoding. You don't have a guarantee that all errors will be caught by it, but most of them can be, so it's useful overall.

https://en.wikipedia.org/wiki/Error_correction_code

If they start becoming relevant, it's usually a sign that you're using the language in a domain where a duck-typed bytecode scripting-glue language is not well-suited.

It was never the point that it's possible to go online without recieving such ubiquitous abuse, which is what you suggest women should do. You have still not acknowledged that such high level of abuse can be dangerous, not merely for the mental health problems but for the real possibility of being stalked in the real world. That way, you're just reinforcing the point you tried to deny, which is that it's hard for men to understand such abuse without having experienced it first hand.