maxutility

This is a very intriguing explanation of observed “bad” behavior by Bing/Sydney, the effectiveness of GPT jailbreaks, and a somewhat compelling argument that it is hard-to-impossible to truly align a LLM and maintain that alignment through extended chats.

In short, it says that a LLM simultaneously simulates all the possible narratives that explain the text and characters observed thus far, and then predicts the most likely next word/token consistent with those simulated narratives. Since so many of the narratives it was trained on include characters that hide their true identity initially but eventually reveal that identity, it almost always maintains one possible simulation path in which the simulated persona is actually the opposite of what was revealed thus far (eg evil, sad, whatnot). And as soon as it picks a single token that is most consistent with this opposite-persona/“waluigi” scenario, it makes the probability of all the non-Waluigi scenarios (eg where Bing really is friendly and polite and happy as originally presented) go to near-zero, such that the Waluigi persona acts as an attractor that can happen at any point in a long chat thread but which is almost impossible to escape from once invoked.

> This approach, which we call DetectGPT, does not require training a separate classifier, collecting a dataset of real or generated passages, or explicitly watermarking generated text. It uses only log probabilities computed by the model of interest and random perturbations of the passage from another generic pre-trained language model (e.g, T5).

Very interesting, though the passage above makes me wonder how robust it is to different models or even finetuned variations on models, as even GPT-3(.5) has evolved quite a bit over recent releases since its initial introduction, and there is likely to only be a greater and greater proliferation of models over time.

Craig Mazin, the screenwriter of HBO's Chernobyl and the Last of Us series, once gave an awesome 45 minute lecture on his ScriptNotes podcast titled “How to write a movie,” in which he walks through how he thinks of storytelling and story structure. If you have any interest in storytelling or screenwriting, it’s a fabulous listen. (You could argue that anything that smacks of formulaic storytelling should be avoided at all costs, but I think it’s still critically important to understand the basic tenets of strong storytelling structure. And as someone who enjoys both of Craig's HBO series, I would say he’s proven he knows how to tell a good story.)

Here’s a YouTube link: https://youtube.com/watch?v=vSX-DROZuzY&feature=shares

You can also find it by searching for episode 403 of scriptnotes in whatever podcast app you use.

I found this article on “transformers from scratch”[0] to be a perfect (for me) middle ground between high level hand-wavy explanations and overly technical in-the-weeds academic or code treatments.

[0] https://e2eml.school/transformers.html

Not mentioned in the OP: if your only Apple Device is an iPhone 7 or earlier, you’re out of luck, since iOS 16 requires iPhone 8 or newer.

Here’s the Dropbox paper[0] and presentation(YouTube)[1] on zxcvbn from the 2016 USENIX security conference:

[0] https://www.usenix.org/system/files/conference/usenixsecurit...

[1] https://youtube.com/watch?v=vf37jh3dV2I&feature=shares

BitWarden has a web tool that lets users securely use zxcvbn to test password strength: https://bitwarden.com/password-strength/

I also enjoyed their follow up walkthrough of Stable Diffusion’s execution code: https://youtube.com/watch?v=-lz30by8-sU&feature=shares

This is cool. I just used it to find a list of link blogs and "cool/interesting things" sites that could be considered adjacent to kottke.org: https://explore2.marginalia.nu/search?domain=kottke.org

This is cool. I just used it to find a list of link blogs and "cool/interesting things" sites that could be considered adjacent to kottke.org: https://explore2.marginalia.nu/search?domain=kottke.org

Chapter five, postscript, and critical Amazon review. This is the final (third of three) post outlining the ChatGPT session.

—— Prompt: Chapter six of six - Inspection and Testing. —— Chapter 6: Inspection and Testing 1. Conduct a final inspection of the new wiring installation to ensure that it meets all necessary safety standards and codes. This may require consulting the building's electrical plan or consulting with a qualified inspector. 2. Test the new wiring to ensure that it is functioning properly. Use a voltage tester to confirm that power is flowing through the new wires and that all connections are secure. 3. Make any necessary adjustments or repairs to the new wiring installation. This may include tightening loose connections or adding additional support for the wires. 4. Provide the property owner or manager with a final report on the new wiring installation, including any recommendations for future maintenance or upgrades. Conducting a final inspection and testing of the new wiring installation is an important step in the process. It helps to ensure the safety and integrity of the building's electrical system, and provides peace of mind for the building owner and occupants. —— Prompt: Postscript: —— In conclusion, replacing faulty knob and tube wiring in a 100 year old house requires careful preparation, attention to detail, and a thorough understanding of the building's electrical system. By following the steps outlined in this manual, electricians can safely and effectively replace the old wiring and ensure the continued safety and functionality of the building's electrical system. It is important to always follow proper safety procedures and consult with a qualified inspector or other professional when dealing with electrical work. —— Prompt: Now, pretend we are reading about the above electricians manual on Amazon, and the review voted most helpful is very critical of the manual. The review might read as follows: —— I recently purchased this manual and was very disappointed in its content. As a professional electrician with over 20 years of experience, I found the information to be overly simplified and lacking in detail. The step-by-step instructions are vague and do not adequately address the complexities and potential hazards of replacing knob and tube wiring. I would not recommend this manual to other electricians. —— Prompt: What type of inadequately addressed complexities and hazards might the Amazon reviewer be referring to? —— The Amazon reviewer may be referring to the potential complexities and hazards that can arise when replacing knob and tube wiring in a 100 year old house. These may include issues such as working with outdated or corroded wiring, dealing with hazardous materials such as asbestos or lead, and navigating the intricacies of an old building's electrical system. Additionally, the reviewer may be concerned about the vagueness of the instructions provided in the manual, which may not adequately address these complexities and hazards. It is important for electricians to be thoroughly prepared and knowledgeable when dealing with these types of projects, in order to ensure their own safety and the success of the project.

Chapters three through five (second of three posts outlining the whole ChatGPT session). Refer to the prior warning.

Chapter 3: Installing New Wiring 1. Consult the building's electrical plan or speak with the property owner or manager to determine the appropriate type and size of wire to use for the new installation. Make sure to use wire that is rated for the voltage and amperage of the circuit you will be working on. 2. Measure and cut the new wire to the appropriate length. Leave some extra length at both ends to allow for connections to be made later. 3. Strip the insulation from the ends of the new wire, using a wire stripper or knife. Be careful not to damage the wire itself during this process. 4. Connect the new wire to the existing circuit. If the new wire is being installed in a parallel circuit, simply connect the new wire to the same points where the old wire was connected. If the new wire is being installed in a series circuit, it will need to be connected to the circuit at both ends. 5. Secure the new wire in place using electrical tape or other appropriate materials. Be sure to leave enough slack in the wire to allow for movement and expansion, but not so much that it becomes a tripping hazard. 6. Test the new circuit to ensure that it is functioning properly. Use a voltage tester to confirm that power is flowing through the new wire and that all connections are secure. By following these steps, electricians can safely and effectively install new wiring to replace faulty knob and tube wiring in a building. In the next chapter, we will discuss the importance of properly labeling and documenting the new wiring installation. —— Prompt: Chapter four of six - labeling and documentation. —— Chapter 4: Labeling and Documentation 1. Label the new wiring using a permanent marker or adhesive labels. This will help identify the new wiring as separate from the old, and make it easier to locate and troubleshoot in the future. 2. Document the new wiring installation in the building's electrical plan or maintenance log. This will provide a record of the work that was done, and help ensure that the new wiring is properly maintained and serviced in the future. 3. Provide the property owner or manager with any necessary documentation, such as receipts for materials and permits for the work that was done. This will help ensure that the new wiring installation is up to code and meets all necessary safety standards. Proper labeling and documentation of the new wiring installation is an important step in the process. It helps to ensure the safety and integrity of the building's electrical system, and provides a record of the work that was done. In the next chapter, we will discuss the importance of properly disposing of the old knob and tube wiring. —— Prompt: Chapter five of six - Disposal. —— Chapter 5: Disposal 1. Properly dispose of the old knob and tube wiring that was removed from the building. This may require contacting a licensed disposal company or following the guidelines set by your local government or waste management agency. 2. Do not simply throw the old wiring in the trash. Knob and tube wiring may contain hazardous materials, such as asbestos or lead, which can be dangerous if not handled properly. 3. Follow all necessary safety precautions when handling and disposing of the old wiring. This may include wearing personal protective equipment, such as gloves and a face mask, and following any specific guidelines set by the disposal company or waste management agency. Proper disposal of the old knob and tube wiring is an important step in the replacement process. It helps to ensure the safety of both the electrician and the building occupants, as well as protect the environment from hazardous materials. In the final chapter, we will discuss the importance of conducting a final inspection and testing of the new wiring installation.

Here are the first prompts and responses in the ChatGPT session. As noted above, the information here should not be relied upon in any way for performing electrical wiring, and is included only as a demonstration of the capabilities and limitations of ChatGPT.

Prompt: Pretend you are writing a manual for professional electricians on how to replace faulty knob and tube wiring in a 100 year old house. Chapter one of six - Preparation: —— Chapter 1: Preparation Before beginning the process of replacing faulty knob and tube wiring, it is important to properly prepare for the task at hand. This will help ensure the safety of both the electrician and the building occupants, as well as ensure the success of the project. 1. Familiarize yourself with the building's electrical system. This may require consulting the original blueprints or speaking with the building owner or property manager. It is important to know the location of all electrical components, such as circuit breaker panels and junction boxes, as well as the layout of the wiring itself. 2. Turn off the power to the area where you will be working. This can be done by switching off the appropriate circuit breaker or removing the fuse from the fuse box. Use a voltage tester to confirm that the power is off before proceeding. 3. Remove any insulation or other materials that may be covering the knob and tube wiring. This may require the use of a knife or other cutting tool. Be careful not to damage the wiring itself during this process. 4. Gather all necessary tools and materials. This may include wire strippers, wire cutters, wire nuts, and electrical tape. Make sure you have enough of each item to complete the project without having to interrupt your work to search for supplies. 5. Put on personal protective equipment, including gloves, safety glasses, and a face mask. This will help protect you from electrical shock and other hazards that may be present during the project. By properly preparing for the task at hand, electricians can ensure their own safety and the success of their project. In the next chapter, we will discuss the process of removing the faulty knob and tube wiring. —— Prompt: Chapter two of six - —— Chapter 2: Removing the Faulty Knob and Tube Wiring 1. Locate the faulty knob and tube wiring. This may be visible in areas such as attics, crawl spaces, and basements. The wiring will typically be marked with a label, such as "K&T," to indicate its type. 2. Carefully cut the electrical tape or other materials that are holding the wiring in place. Use a knife or wire stripper to carefully expose the wires. 3. Use a wire cutter to carefully cut the wires at both ends of the faulty section. Make sure to cut as close to the knob or tube as possible, without damaging the surrounding wiring. 4. Disconnect the wires from any electrical devices or fixtures that they may be connected to. This may require the use of a screwdriver or other tool. Be careful not to damage the devices or fixtures during this process. 5. Carefully remove the faulty section of wiring from the building. Use caution to avoid damaging any surrounding wiring or other electrical components. By following these steps, electricians can safely and efficiently remove faulty knob and tube wiring from a building. In the next chapter, we will discuss the process of installing new wiring to replace the old.

I struggle to think of another example of an organization of this scale placing such a massive companywide bet on a new direction so far away from their core business or any existing mature market. Any Facebookers - ahem - Meta employees want to share their perspective on how this is all being perceived on the inside?

There was a really interesting NYTimes piece [0] about Gen X comedy icon Janeane Garafolo earlier this week that touched on similar themes of “not selling out.” I have often wondered over the years what happened to her. It turns out that she really walked the walk of not selling out and the obscurity that comes with avoiding publicity and promotion.

Personally my feelings on the subject are conflicted. I think that some degree of promotion is important so that others can discover great art and contributions, and so that artists and creators can make a comfortable living off of their work, but that “selling out” becomes bad when the pursuit of commerce overtakes and reduces the art.

[0] https://www.nytimes.com/2022/07/14/movies/janeane-garofalo.h...

There’s insufficient information in this article to draw any meaningful conclusions. For example, was the Tesla in autopilot mode or was this purely human error?

I’ve updated the original post to remove the restaurant name.

After a little more digging, I’m more confused than ever.

Yelp and other sites list the website I used as the official website, the picture on the website matches the interior of the restaurant, and I don’t see scores of online users claiming their bank accounts were hacked after ordering.

I removed the restaurant name out of caution since the best I can tell, the website I used was the official website.

But I still for the life of me can’t think of any legitimate reason why the verification code on the order form would prompt me to receive a banking 2FA code unless it were a phishing attempt.

Does anyone have any theories on what was going on, and what if anything I should do?

Cool - I probably shouldn’t be surprised that the smart minds in machine learning and natural language processing are well ahead of me, an interested lay person!

Any good links or HN comment threads that you’d recommend?

Not directly related to this API, but I’ve begun wondering if the next revolution in speech synthesis is to integrate with a natural language model like gpt-3 in order to gain semantic awareness, and use that context to produce emotional expressiveness and inflection that is attuned to the meaning and tone of the text.

I mentioned this in the iPadOS thread, but it bears repeating here:

Of all the features mentioned, the one feature that I’m really excited for is the ability to mark text messages as unread.

On the other hand, did it really have to take 15 years to get here?

Of all the features mentioned, the one feature that I’m really excited for is the ability to mark text messages as unread.

On the other hand, did it really have to take 15 years to get here?

If the window you’re looking at is just below your camera, when you look at the window it almost appears that you’re looking into the camera.

> Used this way, the thin-film loudspeaker could provide active noise cancellation in clamorous environments, such as an airplane cockpit, by generating sound of the same amplitude but opposite phase; the two sounds cancel each other out.

As someone who suffers from “eardrum suck” [0] when using noise canceling headphones, I’d be concerned if the idea of active noise canceling environments caught on.

[0] https://www.nytimes.com/wirecutter/blog/how-do-noise-cancell...

Are there specific advancements in statistics that you’re thinking of? An explanation or link would be great!

This post gave me a bit of an existential crisis the first time I read it. I’m used to the universe making me seem infinitesimally small. I’m not used to large numbers doing the same to the universe itself (and again ad-infinitim)

Top Career (from the article): AI Safety Technical Research > “To mitigate the risks posed by the development of artificial intelligence, it’s imperative to research how to solve technical challenges and design problems to ensure that powerful AI systems do what we want — and are beneficial — without any catastrophic unintended consequences.”

As the person who originally posted this to HN, after reading some of the refutations of Matt Y’s piece (the OP), I’ve come to believe the evidence for zoonotic origin is stronger than Matt Y gives credit for.

One short rebuttal I found particularly understandable to a layperson comes from a prominent economist (not a virologist), pointing out a few considerations that Matt’s piece appears to give insufficient credence to:

https://www.themoneyillusion.com/case-closed-2/

There’s also the twitter thread by one of the original research article coauthors, but I actually found that harder to evaluate as a layperson.

In particular: 1. Matts piece may overweight how much of a coincidence the virus emerging in Wuhan was if it were zoonotic 2. Matt’s piece may underweight the coincidence of emerging at a live animal market if it were a lab leak 3. Matt’s piece underweights the finding that the market hosted both the original A and B variants

The article (by Matt Yglesias) is more nuanced than the title may suggest. Here’s a quote that gets at the gist of the piece:

“The evidence really does show pretty clearly that there were one or more superspreader events at the Huanan Seafood Wholesale Market. I just don’t think that’s really what the lab leak debate is about; it’s about why there was a superspreader event at the market. Did an infected animal pass it to people there or did a person who got infected at the lab pass it to people there?”