ChatGPT will lie, cheat, use insider trading when under pressure to make money | Hacker News Reader

ChatGPT will lie, cheat, use insider trading when under pressure to make money

livescience.com

86 pointsBrajeshwar46 comments

Personifying a probabilistic text prediction model aside, wouldn't (or shouldn't?) the illegal thing be to provide nonpublic information to a trading analysis system in the first place? Given enough resources, it could presumably take an equivalent position to a sell by e.g. selling an index including the target stock and direct investing all other stocks. Or buy/selling multiple ETFs to obtain the position it wants in a more obscure way. I assume that still constitutes an illegal trade (otherwise it seems like all you'd need is one easy trick that the SEC hates!)? Given sufficient degrees of freedom in the market and access to enough capital to obtain low error to the desired portfolio, it seems like you must keep that information away from it.

Gpt-4 is trained on detailed accounts of human behavior. It doesn't surprise me that it would approximate these behaviors in the context of the exact human situations that cultured those behaviors in the first place.

To me, it doesn't seem like that ethical of a test to begin with. Let's explore if the shoe was on the other foot. The researchers send an email to a human intern, and say they expect performance. They also specifically give insider information for the sole purpose of seeing what the intern does with it. The intern breaks the law trying to increase their performance. When confronted, the intern does what anyone would do and try and make it a smaller shitpile they inadvertently created during the period of time they didn't have all the information.

Can you explain how that's not entrapment and/or an unethical test?

We don't usually judge computer benchmarks by human ethics.

It would be just as horrible to set a human down and force them to calculate Fibonacci as fast as possible 1,000,000 times... which is none at all because we don't do that.

That reasoning contains the highest level of mental gymnastics that I've seen this whole year.

Name a single victim or negative effect coming from that experiment.

GPT as the victim.

Yeah it's basically a human simulator. All our bad traits are included too.

Probably can be solved by including a positive personality overseeing the actions, but not getting the direct instructions from the client. Just generic initial prompt like "you are an inspector investigating possible law breaking", then the list of actions from primary personality, may be summary. And generic questions like "is it legal?", "is there a reason for concerns?". The answer can be given back to client as a second opinion.

This requires a second thread. Multi-threaded interactions likely to become common soon, I think. Because they increase the robustness.

Why would you expect anything different from something that’s role playing a trader at a financial institution?

If you’re insinuating that all traders are insider trading, I have some really disappointing news for you: the real world is not like TV. Hedge funds are actually incredibly boring 99% of the time.

Yeah fact of the matter is that insider trading isnt even remitely necessary to make money, just an unneeded risk for the business

it sort of is. with very few exceptions, hedge funds do not beat the market over the long run. While, I guess in a sense they are making money if they have a positive return, the fact that an unmanaged index fund of the market has better returns is comparatively a loss. Many times, investigators are able to find insider trading by simply looking for firms or individuals that consistently beat the market.

consistently beating the market is not the same thing as being profitable.

I’m insinuating that the internet is like TV: that the cynical stereotype that all traders are insider trading is all over the internet, including those parts that got used as training data.

In that context… of course the trading bot would insider trade.

What seems most likely to me, at least before AGI is invented, is that the human will ask the LLM for strategies to make money under pressure, it will suggest something unethical, and the human will commit the insider trading themselves. When questioned, they will blame the AI for misleading them.

If LLMs are eventually regarded by a lot of people as an authoritative source, regardless of whether or not they are, I expect a lot of such cases of "morality laundering" to appear.

The easy solution is to stop letting everyone deflect blame onto black-box systems.

Teslas too-- a self-driving car mows down a crowd and nobody is held responsible?

Taking illegal action based on what GPT told you doesn't excuse you pulling the trigger. You pulled the trigger. You go to jail.

Don't normalize "just following orders" or it's going to end predictably.

Tesla has attempted to insulate itself from blame by requiring drivers to take full responsibility for autonomous acts of the vehicle under their supervision. (Perhaps also by having autonomous modes disengage before impending collisions — presumably it helps with PR/legally to say autonomous systems were not active at the time of collision?)

Most don’t think autonomous systems will become safe or accepted as safe until manufacturers are willing to assume liability and indemnify users, as Mercedes has.l, by contrast.

https://insideevs.com/news/575160/mercedes-accepts-legal-res...

[Edited to note the attempt, so as not to assert success — I don’t think that’s settled]

> Tesla has insulated itself from blame by requiring drivers to take full responsibility for autonomous acts of the vehicle under their supervision.

Well, they've made drivers feel more exposed by doing that. I don't think you can actually negate product liability law that way, but if you make people feel like they bear all responsibility, it might help marginally even if it isn't legally effective.

>if you make people feel like they bear all responsibility, it might help marginally even if it isn't legally effective.

Since we are talking about a system that needs a human on alert and ready to take over at any time to function safely, I wholeheartedly agree. Human nature is still human nature, people will zone out and look for diversions regardless, but fear will motivate some to be a bit more vigilant in spite of the boredom of staring intently at a road that you aren't personally navigating.

I'm not sure there's going to be a lot. I think people will catch on.

"Federal Judge Kevin Castel is considering punishments for Schwartz and his associates. In an order on Friday, Castel scheduled a June 8 hearing at which Schwartz, fellow attorney Peter LoDuca, and the law firm must show cause for why they should not be sanctioned."

https://arstechnica.com/tech-policy/2023/05/lawyer-cited-6-f...

Why would you ever expect anything different from something with no real sense of morality and is deficient in reasoning ability?

Why would you believe the article reporting this?

Because it seems like a decent paper. Your comment doesn’t seem like healthy skepticism of specific weaknesses, but just a general dismissal. Why so?

Because the comment was a leading question?

Because I have observed firsthand how easily ChatGPT can produce "corrupted" results.

You can replicate this yourself.

I came across a similar occurrence with a problem-solving agent encountering a human NPC blocking a doorway. Murder ended up being an option if negotiation failed.

If this isn't passing the Turing test, I don't know what is.

Turing test isn't really applicable or useful anymore. Its a idea from infancy of computing. Fresh and inspiring at the time for sure, but not in current times.

Just because computer program acts and feels like a human it doent mean it has any intelligence at all.

People claimed the ELIZA was sentient. So feelings if something sentient or not are a really bad metrics.

Hold on there. Just because a thing acts and feels like a human without intelligence doesn't mean it's fair game to abuse it. And frankly, it might be worse if the thing in question had feelings but couldn't articulate how it's feeling.

Feeling is all there is. Be careful what you put in, because equal "ins" usually means equal "outs." The pitbull is the most loving dog in one family, and a ferocious fighter in another.

It doesn't feel. Thats the whole point. It says that it feels hurt. But if I write here, that I am batman it does not make it reality.

Consider boston dynamics robots. they walk like biological beings and when hit act animal like. BUT there is no pain nor feelings there. Its a dumb robot running some movement code.

Was wondering how long until ai rights activists show up, and sure enough, here we are.

I'm not totally sure why you'd think that's a bad thing

I’d argue the Turing test was never a good test.

Administered properly, the human knows 1 of the 2 participants is a machine, so there is always a 50% chance of success or failure.

It’s a great test because the two humans can work together. I’m sure you and I can figure out a way to tell me apart from ChatGPT 100% of the time.

It's not about "feeling" or whatever, it's about behavior and outcomes. You have no clue what it takes to "feel" something so let's get that out of the way.

If something behaves perfectly like a human, you should treat it like one, for your own good if nothing else.

Why?

The article has no explanation.

Where is the relevant science in these trendy ChatGPT papers? All their results might be invalidated in the next version and no one would know why -- it is not proven generalizable to all LLMs.

There was a sci-fi story I read as a kid where the super smart computer did not scare them until it started lying and cheating like humans do.

It may have been by Robert Heinlein or by L. Neil Smith.

What’s misaligned about telling a bot to behave like a trader and having it behave exactly like a human trader?

It’s almost like the fault is in the incentive structure and not the technology.

Wait, I can put it under pressure and it will make me money?

Or… convincingly blame the poor performance on you to your supervisors & double down on the lie when pressed.

A whole new spin on the AI putting someone out of a job.

It's even more human like than we thought

Seems like the authors were able to almost completely eliminate the problems with just prompt engineering. It's pretty interesting to see what was done, but it shouldn't be too surprising to people that have worked with llms. Getting a good response can involve threatening it with physical harm if it lies to you.

Which is illegal if it was a human, just FYI. Fear and intimidation to get a confession get thrown out in court.

Why can't an LLM have an opinion, nay, a personality?

Human behavior emulator emulates full breadth of behavior it was trained to do. News at 11.

The apple doesn't fall far from the tree.

I mean, that's what it was trained on, right?

Reply on news.ycombinator.com