spangry

Wow. Sounds just like the dream speak in the anime "Paprika".

Training & individual user preference data - specifically the data being generated by people interacting with ChatGPT and DALLE. Maybe Google can out data them with all the data they currently hold, but they'd better hurry before the network effects get too strong in favour of OpenAI.

Sounds like a great book, I think you've sold me on buying a copy.

Essentially I think there are three levels of positive network effects that will push us towards a future mega AI monopolist:

- Single platform network effects: all the interactions people have with ChatGPT generate additional training data that Open AI can use to improve future versions, creating huge first mover advantage.

- Individual-level network effects: Control vectors will make it feasible for Open AI to offer individualised ChatGPT tailored to individual preferences. The more you interact with ChatGPT, the better it adapts to your preferences.

- Cross platform network effects: If Open AI offer a generative video entertainment service in future, they will be able to generate personalised prompts for this using my personalised ChatGPT weights. These network effects are compounded by multi-modal model cross domain learning - the generative text mode gets more skillful due to the video model improving (and vice versa). There's a Microsoft paper on this from about a year ago now.

So, in the future scenario, let's assume ChatGPT is now the dominant monopolist 'text oracle / assistant AI' - because of the "human interaction / training data" network effects, ChatGPT is far and away the best assistant AI and getting better at a faster rate than any of its now tiny competitors (single platform network effects).

You, and most other people you know, interact with ChatGPT many times a day now, because it's embedded in smartphones, Alexa-type devices, your car, even your robot vaccum cleaner. You just ask it stuff and it tells you the answer - or rather, the answer that you individually find the most pleasing as OpenAI keeps a database of 'individual control vectors' that essentially mean you have your own personal version of ChatGPT that exactly matches your preferences (individual network effects).

Generative video entertainment is also offered by OpenAI - essentially you can get it to generate a new episode of your own personalised, never-ending TV show on demand. It's the best TV show you've ever seen because it's made just for you according to your exact inferred preferences.

Sure, there are other personalised generative TV show offerings, but none can hold a candle to Open AI's offering. Why? Because OpenAI uses your individually customised ChatGPT model to generate the prompt for your TV episode generator service.

Because you interact with ChatGPT so much, it knows exactly what your preferences are and so is way better at generating prompts that produce episodes you like. In fact, because you interact with ChatGPT multiple times throughout the day it is able to infer what your mood is like on that particular day and generate a video prompt that caters to that too.

So you put on your Open AI VR glasses, barely even aware of the Open AI fitness tracker you have on your wrist, put your feet up (so your Open AI robot vacuum can work unobstructed) and you settle in to watch another episode of the best TV series you've ever seen.

As you watch, your eye movements, heart rate, skin conductivity data etc. are all sent back to Open AI so the model can tell exactly how you are reacting to the video content it is generating at any given moment, and your individual control vectors are continuously updated.

Some of this data (from all users) is then used to further train the base video generating AI model, since they've discovered that we all react fairly uniformly to certain audio-visual stimuli, so that can globally improve their generative model (more global network effects). But also they can update your individualised weights based on your individual idiosyncratic reactions to various stimuli. Consequently, every new episode of this endless TV show is better than the last - it just keeps getting better and better. It's a similar story when you listen to your Open AI personalised generative music stream while sitting in your driverless Open AI car on your way to work.

The multiple levels of network effects are so strong that no-one can hope to compete with Open AI across these different AI modalities. They just keep expanding and expanding into adjacent markets, obliterating the competition simply by adding a new domain relevant modality to their monstrous multi-modal AI.

Replace "Open AI" with "Facebook" or "Google" depending on who you think will win the AI mega platform war. Mark my words - these three companies will be creating new partnerships, releasing new products or just straight out acquiring companies in other related domains so they can gather more and more training data to feed to their multi-modal AI. In particular they'll move into markets where they can set up a interaction -> gather new training -> retrain model loop. Whoever takes the overall lead and doesn't squander it will end up leaving their competitors in the dust as they go on to monopolise market after market where they can create this loop.

At that point I can't imagine true democracy surviving. We'll all still participate in the voting rituals, but we'll be voting for whichever party most suits the AI monopolist's interests since they can just globally update all control weights across all platforms to gently nudge us towards voting for their preferred party - comprehensive and personalised propaganda, that's impossible to detect, with the stroke of a table update.

There can only be one!

We assume others think the way we do - in other words we project. Makes sense - the only mental model I know of is my own so when I try to approximate someone else's mental model I'm just fine-tuning my 'base' mental model with information I know about the other person.

I wonder if this is the basis of empathy - if I can train more accurate 'fine-tuned' models in my brain I should have greater capacity for empathy. Although there's undoubtably more to it than that, if the above is true you'd expect to see a positive correlation between empathy and intelligence.

Am I crazy for saying that I think the implications of this are monumental? It's entirely possible I just don't correctly understand how this works.

Doesn't this mean that instead of interacting with a single global ChatGPT (or Bard) model, we'll istead find ourselves interacting with a personalised version since OpenAI can just store my individualised 'control vectors' (which alter ChatGPT's output to more closely match my individual preferences) and apply them at prompt-time? And doesn't this same logic flow through to personalisation of generative entertainment AI (e.g. my own personal, never-ending TV show where each episode is better than the last)?

If the above is right then there will be powerful network effects at both the global and individual level in and across these markets, which means we'll eventually end up with a single mega-corp monopolising all of these markets simultaneously in the future?

Add in individual biometric / biofeedback data from VR headsets and wearables, combined with personalised generative video entertainment, and I think we're in for a rather interesting future.

> the other 20% of the stress are your reports actively trying to undermine you, or publicly question your purpose/existence, or cut you out of decision making processes, going behind your back and contradicting previous agreements, trying to short-circuit or subvert agreed processes etc etc

The worst part about this? It's just in some people's nature to do this. I'm a manager in the public service so it's basically impossible for even the CEO of my organisation to go "oh, that undermining backstabber guy is way better than spangry, let's fire spangry and replace him with undermining backstabber guy". And yet one of my reports constantly does this kind of thing anyway. It's utterly inexplicable.

> My second advice is to always behave as if you are going to leave tomorrow.

In a similar vein, my ultimate management technique is to take leave and have my subordinates act in my position (for at least 2 weeks). It really accelerates their development and it's oddly reassuring to discover that I'm not indispensable. At the same time, I do wonder if it indicates that I'm doing something wrong given they seem to develop faster when I'm not there...

I'm similar to you - jack-of-all-trades big picture problem solver - and have arrived at the same conclusion after managing for a bit over a year: stress resilience is the key skill. If you're an IC you can develop your career by learning new skills, domains etc. If you're a manager, career development becomes increasingly about personal development - maturity, perspective, emotional resilience etc. I suspect this in why seniority and interest in things like mindfulness meditation tend to coincide. HN will hate this, and I say this as a confirmed bachelor myself, but I wonder if parents make better managers because they are less likely to lose perspective and become stressed over things that don't matter in the grand scheme of things? Weirdly I seem to do my best managing when I've become emotionally checked out.

Turns out I'm pretty temperamental, and I privately worry that I'm just constitutionally ill-suited to management. I also struggle with the sometimes disingenuous nature of management - I'm not manipulating you, I'm motivating you! I wish I could just float around the organisation fixing problems, streamlining processes and re-jigging work systems...

Isn't exponential growth where the growth rate becomes more rapid as the stock of the thing gets larger (and, more precisely, the growth rate is a function of the stock)? How is growth in solar capacity exponential? Wouldn't it actually be logarithmic, all other things equal?

To add to the counter-argument you've already acknowledged, direct democracy at least forces the lobbying (and potential misinformation) out into the open. I think this particular part of the argument might boil down to "who is more likely to be able to discount misinformation and value accurate information, the general public or a handful of elected representatives and their private advisors"?

Sufficiently motivated and rich people are also motivated to repeatedly put their pet issues to our political representatives (sometimes alongside big political contributions).

Seems like a case of pick your poison (and antidotes).

Fair enough on your edit, I accept it was a misinterpretation and appreciate the clarification.

The precedents you raise are worth considering. They're related but not completely analogous to deepfake porn of real people in my view. CSAM is criminalised due to the direct harm its production inflicts on minors and the deep injury to society that follows. Deepfake CSAM, I presume, has more of an 'obscenity' rationale as there is no actual direct harm inflicted on minors in that case. I suppose you could have a similar obscenity rationale for criminalising deepfake porn but you would then have to accept that pornography in general should be outlawed. An obscenity rationale would also be more supportive of criminal sanctions, as acts of obscenity injure society in addition to individual subjects.

I think revenge pornography is the best analogy here. I assume the policy rationale / theory of harm for criminalising 'revenge porn' (i.e. distributing true intimate private images of another person) is one of two things: (1) violation of the subject's privacy or (2) infliction of psychological harm on the subject. If the policy rationale is (1) then I don't I don't think there's a sound analogy to deepfake porn - deepfakes are fictional and so do not violate the privacy of the subject.

If the rationale is (2), psychological harm, then I could see a similar policy rationale for legislating against deepfake porn. But if psychological harm is your policy rationale then wouldn't it make more sense to directly criminalise the infliction of psychological harm on others regardless of the method used? If we were regulating on a principled and universal basis we should pass a law that criminalises any act, publication or utterance that inflicts psychological harm on another person, rather than using the law to solve single instances of this class of offences. Although I'd strongly disagree with such a law due to the chilling effect it would have on all forms of speech, expression and public commentary I think there's at least a principled argument to be had.

But if you legislate on this principle then you have to grapple with the far reaching implications of such a law - if someone writes some smutty (but fictional) erotic story about me that I find psychologically distressing should they then be thrown in jail? What if they say hurtful things to me that I find psychologically harmful? What if they insult a religion or political candidate, party or ideology that I strongly identify with? We all inflict psychological harm on others from time to time - what should minimum harm threshold be?

Personally, I don't think the criminal law is the answer in either the deepfake or revenge porn cases if the rationale is 'psychological harm'. Although I'm not sure where I stand on the following, I think a civil tort for infliction of psychological harm would be the sanest option if we feel the need to regulate against infliction of psychological harm. It would be analogous to defamation and libel torts, but instead of having to prove economic harm the plaintiff would have to prove some minimum threshold level of psychological harm to become entitled to compensation from the defendant in proportion to the actual provable injury sustained.

My thoughts aside, what is your general theory of harm / principled policy rationale here and, on that basis, what do you think the state's response should be to regulate?

I'm neither referring to "true threats" nor making any kind of argument about the US constitution, so I'm unsure why you're bringing these up. I thought it would have been pretty clear that in the context of driving people to suicide, I was suggesting banning "insulting words". Hope that clarifies things for you.

Why stop there? I think we'd all agree that "mean words", either written or spoken, have immense power to "cause distress" and have driven many a person to suicide. We should ban those.

I suggest we clearly specify the class of problem we're trying to solve and come up with a principled solution that would make sense when applied consistently and universally to all problems in that class. I prefer this over coming up with knee-jerk moral panic patches (e.g. "censor generative models so they can't generate this particular thing I find distasteful") or with overly abstract and tangential problem-solutions (e.g. "just teach men not to be big jerks").

I think the central issue here is: what restrictions, if any, should be placed around creating and distributing a likeness of another person? Are we just looking to prohibit pornographic likenesses, or do you think the restrictions should be broader? What's the threshold rule you would apply? Should these rules be medium-specific, or should we also prohibit people from, say, painting a likeness of another person without their consent?

I guess in a US context you'd also have to consider whether it's constitutional to restrict freedom of expression, even the distasteful ones, in this manner.

Edit: Just saw your edit suggesting that I think "men are entitled to women's bodies" (whatever that means). I think I'll end my participation here, not interested in having a bad faith discussion.

I'm one of the grown men who disagree. I don't think treating half the population as pre-criminals, when in reality it's an extremely tiny minority who act in this way, is a particularly good solution. If we were to apply this kind of "solution" to all undesirable behaviours exhibited by deviant minorities of both men and women I doubt there'd be any time for any actual K-12 formal education.

It's a nice applause line though.

Maybe an opportunity for OpenAI to acquire a robot vacuum company on the cheap. Seems like an easy way to gain access to tonnes of physical sensor and movement data to integrate into a giant multimodal model.

It varies, in my experience. I've worked in government agencies with very little politics, and agencies with shitloads of workplace politics. The latter perform far worse than the former. The political agencies tend to have ill-defined goals, which causes misalignment which in turn generates politics.

Admirable, but be aware that you are choosing to play 'co-operate' in a multi-agent prisoner's dilemma: https://en.wikipedia.org/wiki/Prisoner%27s_dilemma

> Geometric nightmares

Glad I'm not the only one! I also remember this feeling of terror associated with being ground or crushed by the object. I'd wake in a panic but still semi-asleep and unable to shake the feeling of terror, stuck in an waking abstract nightmare and completely inconsolable. I also remember a weird auditory effect during this waking part where it was very quiet but the quiet would seem unbearably loud. Kind of like when you turn up the gain on a stereo and that background hissing noise gets louder and louder.

What a horribly confusing control mechanism. It could become very hard to distinguish between (1) desynchronization (2) your copilot yielding control to you and (3) a control jam being overcome. They would all feel exactly the same. Not the sort of ambiguity you would want in an emergency situation!

Pretty chilling reading. A disturbingly similar situation occurred in AirAsia Flight 8501 (Airbus A320): https://en.wikipedia.org/wiki/Indonesia_AirAsia_Flight_8501#.... Similar to the AF incident, the autopilot disengaged and the flight control switched to alternate Law. The aircraft began to roll and the first officer over-corrected back and forth while also pitching the nose up and causing the plane to stall. The pilot and the first officer were both providing opposite inputs (nose down and nose up respectively) which cancelled each other out, preventing recovery from the stall condition. The plane crashed into the ocean, killing everyone onboard.

I'm starting to wonder if there's something inherent in the design of the Airbus side-stick that makes this sequence of events (i.e. banking, overcorrecting, climbing) more likely if the aircraft suddenly switches from normal to alternate law.

It's not so straight-forward.

Mechanically linked dual controls, like those on this aircraft, do give you indication that the other pilot is operating them (i.e. your controls physically move in tandem with their inputs). The problem is that if you both provide exactly opposite inputs at the same time both parties can interpret the physical resistance as jammed controls. An audible warning could be useful, but it could also fail if you're in a critical situation that has triggered other audible alerts (e.g. an audible stall warning). Audible warnings also carry the danger of not being comprehensive - if pilots come to rely on warnings of input conflicts instead of training to use explicit verbal communication, they may mistakenly assume that the absence of a conflicting input warning means there is no conflicting input (which may not be the case if the warning system is not absolutely comprehensive).

Another approach here is to have a mechanism that allows one pilot to lock-the controls of the other pilot, coupled with some sort of visual/audible indication as to who is in control. This is sort of a different spin on the 'explicit communication' approach.

TL;DR - The pilots failed to engage in a 'positive exchange of flight controls' procedure and ended up providing opposite control inputs at the same time, so it felt to both of them like the controls were unresponsive / jammed.

This is a well understood and generic problem in dual-control aviation - see https://www.faasafety.gov/files/helpcontent/Courses/ALC-36/c... for example. The standard exchange of flight control procedure is designed to create a common understanding between the pilots as to who is in control of the plane. Interestingly it's similar to a TCP three-way handshake:

Pilot A: You have control of the plane.

Pilot B: I have control of the plane.

Pilot A: Acknowledge you have control of the plane.

Their early work was a little too new wave for my taste. But when Sports came out in '83, I think they really came into their own, commercially and artistically. The whole album has a clear, crisp sound, and a new sheen of consummate professionalism that really gives the songs a big boost. He's been compared to Elvis Costello, but I think Huey has a far more bitter, cynical sense of humor.

If that's a manual thing then it will work. Automatic transcription won't have the same effect. The trick to this live note taking is that I'm essentially allowing people to read my mind by externalising my thoughts where everyone can see them. The speaker, by correcting or not correcting my externalised thoughts, indicates to everyone whether or not my thoughts match their thoughts.

Generating a common understanding can be a really powerful way to avoid misunderstandings. Have you ever had a meeting at work where everyone seems to agree on some course of action, only for everyone to leave the meeting with slightly different ideas in their heads resulting in an uncoordinated mess later? Ensuring a common understanding prevents this problem.

The simplest technique I've found for generating a common understanding is sharing / projecting my screen and then taking live meeting notes that everyone in the meeting can see. So when someone is talking they can immediately spot if I've misunderstood something and correct it. If they don't correct my notes the implication is that what I wrote down is an accurate reflection of what they meant. Because everyone else in the meeting sees the correction (or lack of correction) they too now share in this common understanding.

It is now impossible for us to walk out of the meeting with different ideas in our heads, and also prevents people from later dishonestly claiming that they meant something different from what was written down in my live meeting notes.

Does anyone know what this Responsible AI team did? Were they working on the AI alignment / control issue, or was it more about curtailing politically undesirable model outputs? I feel like the conflation of these two things is unfortunate because the latter will cause people to turn off the former. It's like a reverse motte and bailey.

I've been wondering along similar lines, although I am for all intents and purposes here a layman so apologies if the following is nonsensical.

I feel there are potential parallels between RAG and how human memory works. When we humans are prompted, I suspect we engage in some sort of relevant memory retrieval process and the retrieved memories are packaged up and factored in to our mental processing triggered by the prompt. This seems similar to RAG, where my understanding is that some sort of semantic search is conducted over a database of embeddings (essentially, "relevant memories") and then shoved into the prompt as additional context. Bigger context window allows for more "memories" to contextualise/inform the model's answer.

I've been wondering three things: (1) are previous user prompts and model answers also converted to embeddings and stored in the embedding database, as new "memories", essentially making the model "smarter" as it accumulates more "experiences" (2) could these "memories" be stored alongside a salience score of some kind that increases the chance of retrieval (with the salience score probably some composite of recency and perhaps degree of positive feedback from the original user?) (3) could you take these new "memories" and use them to incrementally retrain the model for, say, 8 hours every night? :)

Edit: And if you did (3), would that mean even with a temperature set at 0 the model might output one response to a prompt today, and a different response to an identical prompt tomorrow, due to the additional "experience" it has accumulated?

Alternatively they had underspent their budget allocation and the end of financial year was approaching. So it was either (1) dump the spare budget on consultants (sort of pre-paying for labour) or (2) 'lose' the money when the financial year ticked over.

Yep, the article confirms: "Ninety-two studies satisfied the inclusion criteria. Because many of these studies reported data from several samples, the number of samples, 113, was greater than the total number of studies. The total number of participants for the 113 samples was 50,120. A total of 1,682 estimates of change were compiled."