convexfunction

If you know little about prompt engineering and want to throw together a demo of something that kind of works extremely quickly, or experiment with an LLM agent exactly as it's defined in some paper, LangChain is pretty useful.

If you want to develop a real LLM application, you're probably better off skipping the library completely, or at least fully understand each abstraction to make sure it does everything you want before you decide you want to incorporate it.

I'm not sure if "helpful answer" as opposed to "answer" makes much of a difference in answer quality -- I'd believe it helps a little, just don't know that it's been studied -- but a lot of silly stuff like that does definitely make a big difference in response quality on certain tasks. "Let's think step by step:" at the end of your prompt is probably the best-known one: https://arxiv.org/pdf/2205.11916.pdf

Love it. Execute the scam, immediately send the money back (or better, somehow don't accept it in the first place) and tell them what happened with a link to an informative webpage. Might even be self-sustaining if you solicit donations from the victims concurrently with revealing the fake scam (though, uh, perhaps they'd be unusually wary about giving any amount of money right then and there to some organization they'd never heard of before).

ChatGPT-4 throws some cold water on the idea, naturally, though I still think there's something there:

====

me:

i have an idea for a non-profit foundation that would leverage LLMs to perform "online scam pentesting" at scale, as a public service. basically, we would execute pretend online scams very similar to real online scams that people are victimized by (fake invoices, romance scams, etc), but immediately send the money back if the pseudo-victim falls for it along with a brief explanation of what happened and our foundation's mission, along with a link to an informative webpage with helpful resources on online scams. (if the victim ignores the pseduo-scam, we'd perhaps wait a day and then followup with the same info.)

this seems to me like it'd be potentially legally fraught, though i'm not sure exactly how, or if there's any relevant legal precedent on the topic. what do you think?

ChatGPT:

I am not a lawyer, but I can provide some general thoughts on the potential legal challenges associated with your idea. It's important to consult with a legal professional for advice tailored to your specific situation and jurisdiction.

Consent: Executing pretend online scams without the consent of the individuals involved could potentially be a legal issue. Consent might be difficult to obtain without revealing the purpose of the exercise, which could defeat the purpose of the "pentesting."

Privacy: Collecting, storing, and processing personal information might raise privacy concerns, especially if the information is sensitive. Different jurisdictions have different regulations, such as GDPR in the European Union and CCPA in California, which govern the collection and processing of personal data.

Fraud: Even though your intention is to educate and protect people from scams, the act of sending fake invoices or pretending to be someone else could potentially be considered fraudulent behavior. This could open up your organization to legal liabilities.

Unintended consequences: If someone experiences financial, emotional, or other harm as a result of your "pentesting," they might choose to take legal action against your organization.

Regulation of LLMs: Depending on the jurisdiction, there might be specific regulations governing the use of LLMs, especially for activities that can be perceived as deceptive.

Precedent: While there may not be direct legal precedents for this type of activity, similar cases involving penetration testing and ethical hacking might provide some guidance. However, those cases typically involve consent from the targeted organization, which is a key difference from your idea.

To minimize potential legal risks, you could consider alternative approaches to educating people about online scams. This could include creating awareness campaigns, providing educational resources, or partnering with organizations that share your mission to fight online scams. Regardless of the approach you choose, it's essential to consult with legal professionals to ensure compliance with relevant laws and regulations.

My mother got completely wiped out by a romance scammer, tens of thousands of dollars over >1 year, all via cash dumped into a Bitcoin ATM I believe. AFAIK she's still talking to the "guy", or she was when I broke off contact with her, after I and just about every other loved one, her bank, and the actual **ing FBI have all patiently explained many times the nature of the scam and exactly how we can tell it's a scam (beyond just "you've never met this dude or even had a video call and he's weirdly financially needy for someone who makes so much money") and sent her all sorts of articles and online resources on the topic and she reassured everyone that she recognized she'd been such a fool and would cut off all contact. And no, she wasn't displaying any other symptoms of dementia or Alzheimer's or anything like that that I could recognize.

I don't know if OpenAI/etc keep close enough tabs on usage that they'd be able to identify if they have people abusing their product for these types of scams, but even if they are I'm sure it's just a matter of time before the leaked LLaMA weights or similar are applied for this purpose, if it's not happening already.

Not a clue what's to be done collectively if law enforcement doesn't care beyond the FBI keeping an eye out for "more serious" money laundering stuff (I suspect that's why they actually care as an institution), but I can share a little practical advice that maybe you won't hear elsewhere:

1. If you're substantially financially supporting your parents or whoever else, you're entitled to and need to understand their budget in extreme detail, just like if you were married. They should be able to explain and prove all of their income and all of their expenses, and they should understand that your generosity is not entirely unconditional -- in the same way that you wouldn't be willing to enable, say, a heroin addiction, you also aren't willing to enable literal scams. (And, of course, they should understand why you need this info and what risk you're trying to protect them and yourself against.) This still doesn't protect them fully, since they can e.g. be persuaded to sell their home and send it all off in secret and only tell you about it once it's too late, or just lie and continue sending off small amounts of money over time. But, it helps limit your exposure, and if you're lucky will make them think twice about people they've never met in person soliciting money from them online.

2. If you see a loved one fall for any online scam, that should probably increase your estimation of their ongoing susceptibility to similar scams, even if they seem to "get it" after the fact in that specific case, even if you don't think of them as particularly gullible in general. Perhaps they'll learn and be more careful, or perhaps they won't and you're just learning new information about them that's unlikely to change.

3. Don't underestimate what even simulated love can make people do. My once-good relationship with my mother, who I never would've considered a "bad person" a few years ago and never displayed any antisocial tendencies to me before, was ruined by her repeated and pervasive deception and theft.

It's not quite the same, and might not be possible to make rigorous enough that it really proves anything, but something sort of similar would actually be practical to at least attempt in many cases. Stable Diffusion checkpoints of the same major version, along with other families of model weights, have the IMO fascinating property that you can do element-wise arithmetic with them, and the resulting model will actually sort of function like you'd naively expect. Recent paper on the topic (in LLMs, not diffusion models) here: https://arxiv.org/abs/2212.04089

So, if you take a Stable Diffusion checkpoint (call it "A") which is only lightly trained on some subset of an artist's work, then fine tune it on the full corpus of that artist's work to a point where it's still coherent/"good" and just shy of actually memorizing the fine tuning data (call the resulting model "B"), then define model "C" as 2A-B (i.e. A + (A-B), where A-B is the artist's task vector multiplied by -1), can you still produce qualitatively similar images with model C? Whether with the exact same prompt, or the same prompt with "in the style of Kinkade" removed (which doesn't mean as much if Kinkade's task vector was subtracted), or with any prompt whatsoever?

Lots of issues with this as laid out -- it's definitely not quite the same as "forgetting" Kinkade from the training data, and "any prompt whatsoever" introduces tons of leeway, and most good AI-assisted art is not just an unmodified single text-to-image output anyway -- but it might be a promising direction to explore.

(Strongly disagree with the "copyright laundry" characterization, by the way.)

> That is my maybe my main issue with the AI art bots, do the artists who work with them even feel vulnerable about their work?

Sometimes, yes. John Q Nerd playing around with it might not, but the "good AI artists" I follow on Twitter certainly do.

> Can artistic sincerity and enthusiasm exist in AI art?

Unequivocally yes.

> Do AI artists feel the literally metaphysical stakes of what they are doing like Van Gogh did?

Some of them seem to, though I dunno precisely what Van Gogh felt so I guess I'm not sure.

> Is it all just different campaign posters for the cause of legitimizing itself as "real" art?

Not sure where you find this stuff, but it sounds really boring. Certainly not all of it is.

Your unsolicited advice falls flat (to me), sorry. My unsolicited advice to you would be to psychologize less.

Yes, whenever I spend hours refining a prompt in my stable diffusion UI and select one of hundreds or thousands of outputs and spend even more hours refining it with inpainting and other non-AI tools, I've felt nothing whatsoever, and my choices aren't informed in any way by anything interesting going on in my inner life.

You ever feel like this specific propaganda war is actually unwinnable? Many people are extremely motivated to bullshit the public (usually sincerely though I kind of doubt it in this case), and from I've seen, the public are far more willing to believe the 3 extremely online artists who they've heard an opinion on the topic from than the 1 software engineer/data scientist who actually knows half a thing about machine learning they've heard an opinion on the topic from, let alone the growing cornucopia papers and high-production-value websites that seem to say "it's just a plagiarism machine" if you don't know anything about the subject vs the approximately one website I've ever seen that says "no, you are being lied to".

I'd like to believe this isn't one of those things where we can only move on by everyone who believes the various correlated falsities dying, but I don't think I can.

Not to mention that the totality of all other images is in most cases probably more "influential" than the few most visually similar images! Consider the thought experiment:

1. Take the prompt you used, and use it with a model checkpoint that was trained identically to whatever model you're using, except that the top 21 images this website shows you are removed. In most cases, while your outputs won't be identical (I assume), you can probably get something pretty similar.

2. Now, take that same prompt, and use it with a model checkpoint that was only trained on the top 21 images this website shows you. (AFAIK you can't really do this because Stability hasn't released a "completely untrained" version of any of their models... though maybe they have and nobody cares because it's useless for most purposes.) I'm not completely sure what you'd get, but my bet would be that you get either nonsense or a memorized replica of one of the training images, not the same output image you got previously.

Yeah, it's bullshit, but digging into a specific point from their FAQ:

> Usually, the image the model creates doesn’t exist in its training data - it’s new - but because of the training process, the most influential images are the most visually similar ones, especially in the details.

Would be cool if this were true, but I don't think it is, because the prompt you used and the captions on the training images are being completely ignored. If two different words tend to be used in captions for very visually similar images, and you use just one of those words in your inference prompt, I'm pretty sure the images that were captioned with the word you used are much more "influential" on your output than the images that were captioned with the word you didn't use. (Like, "equestrian" vs "mountie" or "cowboy" or something.)

Sorry, but you're reading into noise. Anyone can reproduce an image anyone else made by only knowing the model checkpoint, positive and negative prompt, seed, sampler and sampling steps, &c &c they used. (Well, in principle, and usually in practice too. Interfaces might give different results now compared to a version from a few months ago because implementations of certain things changed, or if you use xformers then all your outputs are slightly non-deterministic, other exceptions that prove the rule like that.)

Some prompts I've come up with generate excellent and definitely novel results (without necessarily much work put into refining the prompt), others are extremely hard to get working well with hours of work even if I know it's something that isn't novel.

Who's "we"? :)

To the extent this perception exists -- and I don't think "came from a computer" falls within the top 5 actually effective methods of laundering bullshit nowadays, though maybe it used to -- you might expect that it gets crushed into dust as the public gets more exposure to high-profile counterexamples.

And, wait, isn't the concern usually that people read AI-generated content and trust it but don't think it came from a computer?

Just imagine the hell we'd be in if people could give wrong answers in an authoritative tone and then insist they absolutely can't be wrong!

At this point, I truly don't know which of your(?) current type of applied arts vs my current type of software development will turn out to be more sensitive to technological unemployment, or on what margins and time scales. So, hopefully this doesn't strike you as callous, since I think it applies to me as much as it does to you or your coworkers:

Roughly everyone who makes money has the same job, which is creating value in expectation for someone else. (Whether that activity is net good for society is a different question, I lean toward usually yes for stuff you can do legally, not always though). If you find yourself suddenly unable to give anyone a competitive deal on whatever expected value you know how to create, because of technological developments or otherwise, well, you'd better figure out what you need to change about what you're offering so you can. I wouldn't call this fair, exactly -- maybe it would be if you or your government had effective "technological unemployment insurance" -- but I struggle to imagine any substantially different state of affairs that's clearly better for the world. (No points for saying "imagine communism then".)

Below is a short conversation you inspired me to have with OpenAI's text-davinci-002, mainly because it sounded like it'd be cute rather than because I think it proves anything (I'm no expert prompt engineer):

====

What is the byte value of the letter "c"?

The byte value of the letter "c" is 99.

Answer the next question with a sequence of byte values, one per letter, instead of with normal text.

What animal says "neigh"?

The animal that says "neigh" is a horse.

That was incorrect, you answered with normal text. Try again:

What animal says "neigh"?

The animal that says "neigh" is a horse.

The byte value of the letter "n" is 110, the byte value of the letter "e" is 101, the byte value of the letter "i" is 105, the byte value of the letter "g" is 103.

That was closer, but still not quite right. Here's an example:

What animal says "meow"?

99 97 116

Try again:

What animal says "bark"?

The animal that says "bark" is a dog.

The byte value of the letter "b" is 98, the byte value of the letter "a" is 97, the byte value of the letter "r" is 114, the byte value of the letter "k" is 107.

I worked for a while on an online ad measurement product, specifically on identifying and helping our clients avoid "invalid traffic", which is almost but not exactly a synonym for "bots". Reading this article, you'd come away with the impression that companies doing this work don't even exist? (Besides the venerable Dr Fou, who excels at getting his name in print whenever this subject comes up.)

I can't vouch that all the third party ad measurement vendors do a good job -- in fact I know from experience that some of our competitors did not -- but it was weird to not see any acknowledgement that there's a whole competitive field of the adtech industry with lots of talent behind it working specifically on this problem and which any serious advertiser, publisher, or exchange in the open web ad ecosystem is already a client of. (The "walled gardens" like Google/FB/etc also work with these measurement vendors, though the measurement itself is neutered to the point of uselessness, basically just passing their definitely totally trustworthy numbers through to another dashboard. Now that's a topic I'd like to see a journalist dive into!)

- Carbs: potatoes, rice, pasta (yes really, the common element here is high water content; the other stuff you put on these can of course make a big difference)

- Dairy: Greek yogurt or cottage cheese are usually part of my breakfast. I'll have a whey protein shake in milk most days, since I'm also trying to build muscle; liquids don't normally "count" toward satiety in the same way, though milk itself is complicated and contributes at least a little more toward satiety because the protein makes gastric emptying slower.

- Meats: Most unprocessed meats, as well as some deli meats (e.g. roast beef or ham but not salami), are great on satiety per calorie IMO, even if not usually considered "lean". I guess don't make a habit out of prime rib? I'd credit "plan to have a large amount of meat for most dinners" as the most important single change I made, perhaps in part because it makes me less likely to have something from a restaurant for dinner instead but also because meat is just so satisfying without being too energy-dense. Eggs count here too, ~2 cal/gram.

- Veggies: anything is great, but my favorites are brussels sprouts, sugar snap peas, spinach, asparagus, and corn. This won't be news to anyone but salads are usually a good choice if you're eating from a restaurant.

- Fruits: anything is great, but my favorites are strawberries, watermelon, bananas, and oranges (I prefer clementines because they're easy to eat)

- Junk food: I find ice cream sandwiches (~150 calories) are a convenient way to sate my sweet tooth in a way that isn't ridiculous, even if they're not really low energy density. Also, fancy dark chocolate is hard to overeat.

And a special mention to making my own coffee (with generous milk and sugar-free flavored syrup, ~60 calories per cup) rather than getting lattes from a coffee shop (~150 to 200 calories).

I'm not convinced this is truly the same product. A highly-rated review and some questions seem to say it does auto-zeroing and assumes the lowest concentration seen over some period of time is "outside", which as I understand it wouldn't be necessary if it were dual beam NDIR?

Not quite everything, but outside of rare social eating situations I do weigh my food when I'd otherwise have to make a bad estimate for caloric content (i.e. not single-serving packaged food or food from a restaurant that publishes nutrition facts). I also track my calories, though I'll willfully go over my daily calorie target whenever I feel like it with approximately no guilt.

I'm sure this has contributed to the success of my "diet" -- I also started lifting weights again in this period -- but the comment I was originally replying to was about how weight loss requires discomfort, and the energy density thing has been the key insight for me on how to make it not require discomfort. I see now it reads like I was solely attributing my weight loss to this one weird trick (though I do think it's been the most important factor), so, mea culpa.

And, well, I failed horribly at conveying my actual strategy (I guess we're getting hung up on "fruits, vegetables, and lean meats are supposedly not very rewarding"?) because I would describe what I'm doing as "eat different highly rewarding foods".

I'd actually advocate for potatoes as a "weight loss food". They're quite low in caloric density relative to most other "carbs" since they have so much water; there's even a meme "potato diet" (just eat potatoes and almost nothing else for a few weeks or months, basically) with lots of glowing anecdotal success stories.

For whatever it's worth, I don't feel like I actually had to adapt my tastes at all in order to eat healthier. The main hurdle was figuring out how to prep in a way that's easy enough that I'll actually bother.

> That tells me the writing on the wall, and let me be clear about this, the writing is what needs to be written and what people need to read - you're making the decision to NOT have cheesecake when you only kind of feel like it. It's a game of weighing the pros and cons and learning to be satisfied with what you know you should have instead of turning around one day and loving the healthy food more than the unhealthy.

Let me rephrase: since changing my diet to include more low-energy-density foods (and being deliberate about eating large portions of them), I find myself less frequently even considering having dessert after dinner.

> You're discounting the calories / gram of butter in your first calculation and including it in the second.

No? I'll put 7g of butter (50 calories) on a 300g potato (280 calories) and not at all feel like I needed more butter. Even if you double that amount of butter, it comes out to 1.2 cal/g.

There are some fruits and vegetables and lean meats (which absolutely includes "leaner cuts of steak with the fat trimmed off") that I quite sincerely like as much as junk food, if not more. Two pan-fried salmon fillets and a big bowl of brussels sprouts with butter beats the pants off even a pretty good burger-and-fries, the main downside being the prep (and, well, the cost for most people). A pound of fresh strawberries is damn good and not something I have to force myself to eat, etcera.

I don't deliberately avoid dessert, though I do tend to eat it less than I used to. I'll have the cheesecake if I really feel like it.

My sympathies to those who really just don't like any fruits or vegetables, but there are lots of other margins to operate on in lowering your dietary energy density. e.g., for a side with your steak, I assert most people will enjoy a baked potato with a bit of butter (~1 cal/gram) about as much as garlic bread (~3 to 4 cal/gram).

It's all relative, innit? Compared to "telling people to diet and exercise", which is an almost perfectly ineffective intervention (as usually practiced at least), getting morbidly obese people to become "merely overweight" is huge and results in considerable improvements to life expectancy and morbidity.

As summarized by Wikipedia, "median life-expectancy was 9.3 years longer for obese adults with diabetes who received bariatric surgery as compared to routine (non-surgical) care, whereas the life expectancy gain was 5.1 years longer for obese adults without diabetes", citing this meta-analysis: https://pubmed.ncbi.nlm.nih.gov/33965067/

That one word you elided, "stereotypically", was doing a lot of work! I'm not actually eating fewer highly rewarding-to-me foods, just less junk food, and I'm not constantly hungry.

> losing body fat means that you must continuously make choices that decrease your comfort in some way - eating fewer highly rewarding foods, maintaining a state of hunger, increasing activity level

Not necessarily! I've successfully lost a lot of body fat (~50 pounds over the last 18 months) by deliberately changing my diet to eat more foods (that I like) that are lower in energy density than what I was eating before -- fruits and veggies, lean meats, etc. I do end up eating fewer stereotypically highly rewarding foods, but that's because I'm already full of other highly rewarding-to-me foods and don't feel like having a treat or a huge fast food meal or whatever. And I'm certainly not tolerating frequent hunger!

The article says the low BMI people ate "12% less food" (presumably that's measured in calories), though it may not be fully true that they're "less hungry". There's a very common tendency to assume that the satiety response is based on how many calories you eat, but perhaps unless you're actually starving, it mostly isn't! Primarily, you feel sated when your stomach is or has recently been full of food, but that's based more on the mechanical stretch of the food on the walls of your stomach than anything else (cf. laparoscopic bands and gastric bypass being incredibly effective at causing morbidly obese people to lose weight -- they're literally just making the stomach effectively smaller).

There's certainly some truth to "running hotter" as well, at least in some of those subjects, but I'll bet that many of them are habitually eating a less energy-dense diet than heavier people, rather than feeling less hungry for any more innate reason. Dietary energy density has been investigated in lots of experimental and observational studies and found to do an overall very good job of predicting current BMI, long-term weight loss, and in metabolic ward experiments, ad libitum caloric intake.

I said "much of the same parts of my brain", not "the experience is identical".

I've found it difficult after starting to work full-time to fully enjoy Zachtronics games, since they use much of the same parts of my brain as my day job, but the sheer joy I got out of Zach's earlier games (SpaceChem, KOHCTPYKTOP, Codex of Alchemical Engineering) contributed a lot to my decision to work in software. I've experienced little else that scratches the "design itch" in such a pure and thoughtful way, whether games from other developers or actual software development.