An Economy of AI Agents | Hacker News Reader

An Economy of AI Agents

arxiv.org

141 pointsnerder9297 comments

Hmm, where have I seen this before…

https://en.wikipedia.org/wiki/Accelerando

Amazing how forward looking that book was.

One of the most interesting things about the book is how it skewers the idea of there being a singular AI.

In Accelerando the VO are a species of trillions of AI beings that are sort of descended from us. They have a civilization of their own.

That description leaves off some of the flavor— changes the feel once you know the descendent were labeled the Vile Offspring by the main characters, that were still human-ish.

For better or worse, Accelerando comes to mind _a lot_ when trying to figure out how agent platforms should work. Hopefully for better.

It's not gonna happen like that, because all the paths leading to it cannot be financially exploited.

Also what a shortsighted scifi book, yet techies readily invest in that particular fantasy because it's not your usual spaceship fare.

> Also what shortsighted scifi book

It's art not oracle

Some people will call BS on books like that until every detail down to getting their own AIneko cat, and start again when the Wunch start eating their uploaded thought space via man-in-the-middle exploits that the protocol spec in the book was inaccurate

Here's another one: Manna - Two Views of Humanity’s Future, by Marshall Brain. It's a fairly light read, just 8 chapters:

https://marshallbrain.com/manna1

I think I've internalized these stories enough to comfortably say (without giving anything away) that AI is incompatible with capitalism and probably money itself. That's why I consider it to be the last problem in computer science, because once we've solved problem solving, then the (artificial) scarcity of modern capitalism and the social darwinism it relies upon can simply be opted out of. Unless we collectively decide to subjugate ourselves under a Star Wars empire or Star Trek Borg dystopia.

The catch being that I have yet to see a billionaire speak out against the dangers of performative economics once machines surpass human productivity or take any meaningful action to implement UBI before it's too late. So on the current timeline, subjugation under an Iron Heel in the style of Jack London feels inevitable.

> take any meaningful action to implement UBI

I hear this all the time, but to what end? If the input costs to produce most things ends up driving towards zero, then why would there be a need for UBI? Wouldn't UBI _be_ the performative economics mentioned?

I think of it like limits in math. The rate at which we'll be out of work is much higher than the rate at which prices will fall towards zero.

A performative/underemployment economy keeps everyone working not out of necessity, but to appease the sentiments of the wealthy. I'd argue that we passed the point at which wages were tied to productivity sometime around 1970, meaning that we're already decades into a second Gilded Age where wealth comes from inheritance, investment and connections (forms of luck) rather than hard work.

And honestly, to call UBI performative when billionaires are trying to become trillionaires as countless people die of starvation every day just doesn't make any sense.

Isn’t that the one where corporate structures become intelligent self executing agents, cause a lot of problems? Yet here IRL, the current tech billionaires think it’s a roadmap to follow?

Talk about getting the wrong message. No one show those guys a copy of 1984! Wow, then…

Please ELI5 for me: How are AI agents different from traditional workflow engines, which orchestrated a set of tasks by interacting with both humans and other software systems?

Traditional workflow is largely predefined & rule-based.

There’s a level of autonomy by the AI agents (it determines on its own the next step), that is not predefined.

Agreed though that there’s lots of similarities.

But rule-based processing was exactly the requirement. Why should the workflow automation come up with rules on the fly, when the rules were defined in the business process requirements? Aren't the deterministic rules more precise and reliable over the rules defined by probabilistic methods?

Autonomy/automation makes sense where error-prone repetitive human activity is involved. But rule definitions are not repetitive human tasks. They are defined once and run every time by automation. Why does one need to go for a probabilistic rule definition for a one-time manual task? I don't see huge gains here.

I like determinism and objectivity as much as the next guy, but working in the industry for decades led me to realize that conditions change over time and your workflow slowly drifts away from reality. It would be more flexible to employ an AI agent if it works as promised on the tin.

There is no "reality" other than business requirements. That's the context for a workflow. You probably meant that the requirements aren't agile enough to meet the changing world outside. That's a different problem, I think. You can't bypass requirements and expect workflow to dynamically adapt to the changing reality. If that's the direction with AI-driven business re-engineering, then we are back to the chaos, exposing the business logic directly to the outside world.

Rules are the context for a workflow and they have to be updated as the environment changes and that is what I've observed. YMMV.

Yes. But rules themselves live in the context defined by requirements.

Sometimes the rules are not as easy to define ahead of time. As an example imagine having to categorize some sort of text based requests etc.

Or decide what the next step should be based on freeform text, images, etc.

Hardcoded rule based would have to try and attempt to match to certain keywords etc, but you see how that can start to go wrong?

This is already solved by the traditional workflow systems. For example, if the request is received as a form submission, a form processor is invoked to categorize the request and route the request accordingly based on the identified category.

Now, if the request is coming in as text or other media instead of a form input, then the workflow would call a relevant processor, to identify the category. Everything from that point runs same as before. The workflow itself doesn't change just because the input format has changed.

And what is this processor and how does it work?

How does it determine next step from raw non structured content?

Let's imagine for example that it's a message from a potential customer to a business. The processor must decide whether to e.g. give product recommendations, product advice, process returns, use specific tools to fetch more data (e.g. business policies, product manuals, etc), or current pricing, best products matching what the customer might want etc.

If it's an AI agent it could be something like:

1. Customer sends message: "my product Y has X problem". (but the message could be anything from returns to figuring out most suitable product)

2. AI Agent uses "search_products" tool to find info about Y product in parallel with "send_response" to indicate what it's trying to do.

3. AI Agent uses "search_within_manual" tool to find if there are specific similar problems described.

4. AI Agent summarizes information found in manual, references the manual for download and shows snippet of content it based its answer on.

AI Agent itself is given various functions it can call like

1. search_products

2. search_business_policies

3. search_within_documents

4. send_response

5. forward_to_human

6. end_action

7. ... possibly others.

How would you do it in the traditional workflow engine sense?

I think you missed the whole point. Processor does not have routing logic. It's only job is to parse the request (form, text, image etc) and categorize it enough so that the workflow can do the routing for next actions. Routing is done by traditional predetermined logic, using rules. The discussion here is about whether it helps to define that routing logic at runtime (on-the-fly), instead of having it coded in predetermined logic. My view is, it doesn't help.

I edited my comment to add more of what the agent would be doing. Not sure if this reached you, but if you read the edited one, how would traditional workflow engine solve the particular problem with a free form raw content that could be anything, and requires using various tools to solve the problem?

IME traditional rules based systems don't try to solve free form problems. So they stop at the point the inputs can't be handled (any further) whereas LLM could continue, albeit without any guarantee the result is accurate. It could completely hallucinate a fictional solution, which I've seen too often to trust them.

AI-based tools are mostly about replacing the processor with something smarter, not the router.

Of course, sometimes it can be an advantage to not have to explicitly write the router, but the big benefit is the better processor for request->categorization, which with AI can even include clarification steps.

received message: "Maximize the production of paperclips"

There is always much path dependence in what becomes the business requirements, making the business requirements less than optimal to start with.

Then over time their is a type of entropy with all business processes.

If we don't figure out dynamic systems to handle this it is hard to see how we get a giant productivity boost across the economy.

There is also the problem that what percentage of people even have exposure to the concepts of dynamic systems? When I was in college, I distinctly remember thinking dynamic systems, "chaos theory", was some kind of fractal hippy pseudoscientific fraud best to ignore.

I think of how often I hear the average person use language from probability theory but never from dynamic systems.

Workflows exist to solve problems. If there are problems which need solving that are solved better/faster/cheaper by AI agents than with strict rule-based algorithmic systems, they’ll be used because it makes economic sense. Reliability requirements are different for every problem, cases where verification is easy and cheap and multiple attempts are allowed are perfect for not 100% reliable agents.

it's fine if you want AI to help you in defining the workflow/rules. But you don't use AI to define rules on the fly. That's the whole point. It is like having AI to write code at runtime based on the request. I don't think that's how you use AI in software.

In many cases I do have AI writing code runtime for my tasks. E.g. if I'm doing analysis of something, or some data, I can ask AI to gather the data (it writes code for scraping or uses pre existing scraping/web search tools), then it uses python or typescript to analyze the data. After which it will use e.g. React in the web app to render the Charts, Tables in customized, personalized format.

For instance, I might be looking for a product or something, it will use web search to gather all possible products, then evaluate all the products against my desired criteria, use some sort of scoring mechanism to order the products for me and then write an UI to display the products with their pros and cons specified, with products ranked using an algorithm.

Or I might ask it to find all permutations of flight destinations in March, I want somewhere sunny and use weighted scoring algorithm to rank the destinations by price, flight travel duration etc.. Then it writes code to use flights API, to get all permutations and does the ranking.

I used to have to go to things like airport websites, momondo, skyscanner, I don't have to do those things manually anymore, thanks to AI agents. I can just let it work and churn out destinations, travel plans according to a scoring criteria.

Worst mistakes it can make is, that is missed a really good deal, but this is something I could even more easily miss myself, or worst case it makes a mistake and parses the price/dates wrong, which I will find out when trying to book it, so I waste a bit of time, but similar and worse mistakes I do on my own as well. So overall I drastically reduce my search time for the perfect travel, and also time spent on my own mistakes or misunderstandings. And it will be able to go through permutations far faster and more reliably with infinite patience compared to myself.

None of what you said qualifies as coding at run time. I think you missed at. At run time, code is executed, not designed or defined.

Okay I tell AI to do X, it writes a script and executes it to perform X, how is that not defining code runtime?

AI agents like Claude Code or Codex constantly use the technique of writing temporary scripts and executing them inline.

If your system receives 1000 requests per second, does it keep writing code while processing every request, on per request basis? I hope you understand what run time means.

Define runtime then.

> If your system receives 1000 requests per second, does it keep writing code while processing every request, on per request basis? I hope you understand what run time means.

With enough scale it could, however it really depends on the use case, right? If we are considering Claude Code for instance, it probably receives more than 1000+ requests per second and in many of those cases it is probably writing code or writing tool calls etc.

Or take Perplexity for example. If you ask it to calculate a large number, it will use Python to do that.

If I ask Perplexity to simulate investment for 100 years, 4% return, putting aside $50 each month, it will use Python to write code, calculate that and then when I ask it to give me a chart it will also use python to create the image.

> Define runtime then.

From GP: "But you don't use AI to define rules on the fly."

Neither Claude nor Perplexity change the rules they work by on the request to request basis. Code that Claude outputs isn't the code the Claude runs on and Perplexity did not on its own decide to create python scripts because other ways it was calculating large sums did not work well. Those tools work within the given rule set, they do not independently change those rules if the request warrants it.

You are not really defining the runtime?

Is whatever what happens between e.g. HTTP Request and Input and Output not runtime then?

1. HTTP Input

2. While (true) runAgent() <- is that not runtime?

3. HTTP Output

Additionally Claude could be triggering itself with custom prompts etc to use instances of it concurrently in parallel.

Or are you saying that the only rule is that Agent is being ran in a loop?

But the whole discussion is about how AI Agent is different from a Workflow?

The point is that workflow is that LLM is triggered in a loop ?

I noticed on our own agentic setups that there are very few actual scenarios being executed. I suggested implementing some type of monitoring so you can replace 99% of most used workflows with normal python and activate AI calls if something new happens. until that new thing repeats few times and you translate that to code to. that has to be carreer in itself. you can turn a lot of AI apps into profitable and fast internal apps

I needed some data from a content API, had a few options:

1) Human agent, manual retrieval (included for completion

2) one-off script to get exactly the content I want

3) Traditional workflow, write & maintain

4) one off prompt to the agent to write the script in #1, sort and arrange content for grouping base on descriptions it receives (this is what I used, 3 hours later I had a years worth of journal abstracts of various subjects downloaded, sorted, indexed and summarized in a chromadb. I’d just asked for the content, but it’s python code it left for me included a parameterized CLI with assorted variables and some thoughtful presets for semantic search options.)

5) one off prompt to the agent to write the workflow in #3, run at-will or by agent

6) prompt an agent to write some prompts, one of which will be a prompt for this task, the others whatever they want: “write a series of prompts that will be given to agents for task X. Break task x down to these components…”

The human is no longer in the loop. The agentic system is capable of generating quality synthetic data over time to train on. It becomes self improving with the quality synthetic data that can be used to train weaker models to perform better.

have you built stuff with LLMs before? genuine question because nondeterministic and deterministic workflows are leagues apart in what they can accomplish.

It's hard to predict the future. For example this was a popular article not that long ago - https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...

Which has become largely true? People flip-flop between the hottest AI model of the day. After a flagship AI model ships, distillations appear that offer slightly degraded performance at the fraction of the cost.

For inference, the difference between expensive data center hardware and homemade GPUs largely comes down the distinction of RAM. Which is a limitation actively worked around (unfortunately all the well-funded orgs are not that interested in this)

This idea seems to be coming up in multiple places.

Here's one from Deepmind:

https://arxiv.org/abs/2509.10147

2 interesting new protocols to check out if you're interested in this topic:

1. https://www.x402.org/ - micropayments for ai agents to access resources without needing to sign up for an api key

2. https://8004.org/ - open AI agent registry standard

Maybe the time for actually useful distributed autonomous corporations is not far away anymore?

https://en.wikipedia.org/wiki/Decentralized_autonomous_organ...

I dunno man… did the DAO guys ever realize that they had just reinvented an open source co-op?

I feel like co-ops were awful anyway even without the blockchain.

I think you may be missing out on the general idea of DAO's in general by restricting yourself to a few particular historical uses (and many a failed one at that) of DAOs, back from when agentic AI wasn't a thing.

The hackability of these things though, that still remains a very valid topic, as it is orthogonal to the fact that AI has arrived on the scene.

Okay so how does an economy of AI companies doing business selling services related to hyperintellingent AI tech to each other differ from Nvidia, Oracle and OpenAI sending money to each other to buy eacy other's stuff?

Is this what will be tried to fix the potential fallout from continuously decreasing fertility rates (resulting in population decline, thus affecting the consumption-based economy)?

Nope. This is just greed to make most of the moment without any thought for tomorrow. Nobody knows or cares where it takes us, but everybody knows that there is money to make today. So you need a model to analyze the economy with greed as the only driving force without any foresight. Add some parameters to account for monopolistic forces, human desire to be lazy and dumb thinking it is progress, and losing all biological senses to devices. That may give a better prediction.

Wouldn't "The Age of Em" be relevant to this paper? Was surprised to not find a reference to it in the preprint.

read the paper and not sure if i buy all of it. i just keep buying domains with agent in it and haven't shipped anything (agentify.sh for example) wish I could know what to build (and stop buying domains)

Interesting point — thanks for sharing.

Well for starters if some incredible change to capitalism doesn't occur, we are going to have to come up with never before cooperative software tools for the general populace to assess and avoid the most egregious companies that stop hiring people.

Tools for: mass harassment campaigns against rich people/companies that don't support human life anymore, dynamically calculating the most damage you can do without crossing into illegal.

Automatically suggesting alternatives of local human businesses vs the bigevils, or collecting like minded groups of people to start up new competition. Tracking individual rich people and what new companies and decisions they are making doing ongoing damage, somehow recognize and categorize the trends of big tech to "do the same old illegal shit except through an app now" before the legal system can catch up.

Capitalism sure turns out be real fucking dumb if it can't even come up with proper market analysis tools for workers to have some kind of knowledge about where they can best leverage their skills, companies get away with breaking all the rules and create coercion hierarchies everywhere.

I hate to say (because the legal system has never worked ever) but the only workable future to me seems like forcing agents/robots to be tied to humans. If a company wants 100 robots, they must be paying a human for every robot they utilize somehow. Maybe a dynamic ratio somehow, like if the government decided most people are getting enough resources to survive, then maybe 2 robots per human payed.

“…the only workable future to me seems like forcing agents/robots to be tied to humans.”

This is what I’ve been thinking lately as well. Couple that with legal responsibility for any repercussions, and you might have a way society can thrive alongside AI and robotics.

I think any AI or robotic system acting upon the world in some way (even LLM chatbots) should require a human “co-signer” who takes legal responsibility for anything the system does, as if they had performed the action themselves.

I dunno, I think social media of the past years has certainly indicated that whoever controls (social) media can do a pretty good job of creating or bleeding out social movements, by amplifying or dampening some venues of social discourse.

Nah. UBI and a national dividend is all we really need if AI and robots greatly increases wealth.

> If a company wants 100 robots, they must be paying a human for every robot they utilize somehow

This is so bonkers and absurd I don't know what to say.

> Automatically suggesting alternatives of local human businesses vs the bigevils, or collecting like minded groups of people to start up new competition

I think you are correct on the competition part.

I think we are going to see a avalanche of millions of small business takeaway market share from big businesses(b2b saas is the first casualty but also others in the future as technology advances).

However more than by regulation, I think it'll happen due to AI/LLMs itself.

AI systems cannot be economic agents, in the sense of participating in a relevant sense in economic transactions. An economic transaction is an exchange between people with needs (, preferences, etc.) that can die -- and so, fundamentally, are engaged in exchanges of (productive) time via promising and meeting one's promises. Time is the underlying variable of all economics, and its what everything ends up in ratio to -- the marginal minute of life.

There isn't any sense in which an AI agent gives rise to a economic value, because it wants nothing, promises nothing, and has nothing to exchange. An AI agent can only 'enable' economic transactions as means of production (, etc.) -- the price of any good cannot derive from a system which has no subjective desire grounded in no final ends.

They can once they achieve purpose.

However that seems completely tangential to the current AI tech trajectory and probably going to arise entirely separately.

> Time is the underlying variable of all economics

Not quite. It's scarcity, not time. Scarcity of economic inputs (land, labor, capital, and technology). So "time" you mean labor and that's just one input.

Economics is like a constrained optimization problem: how to allocate scarce resources given unlimited desires.

Depending on how you feel about various theories of development, an argument that all of these categories reduce to time. At the very least, the relationship between labor, capital, and time seems pretty fundamental: labor cannot be instantaneous, capital grows over time, etc.

They can all be related on a philosophical level but in practice economists treat them as separate factors of production. It's land, labor, and capital classically. Technology/entrepreneurship can be seen as another factor, distinctly separate from labor.

I agree that time isn’t an input in the economic system.

Although, one can use either discrete or continuous time to simulate a complex economic system.

Only simple closed form models take time as in input, e.g. compounded interest or Black-Scholes.

Also, there are wide range of hourly rates/salaries, and not everyone compensated by time, some by cost-and-materials, others by value or performance (with or without risking their own funds/resources).

There are large scale agent-based model (ABM) simulations of the US economy, where you have an agent for every household and every firm.

> Although, one can use either discrete or continuous time to simulate a complex economic system.

A very bad model that lacks accuracy and precision, yes. Maybe if you're a PhD quant at Citadel then you can create a very small statistical edge when gambling on an economic system. There's no analytic solution to complex economic systems in practice. It's just noise and various ways of validating efficient market hypothesis.

Also, because of heteroskedasticity and volatility clustering, using time-based bars (e.g. change over a fixed interval of time) is not ideal in modeling. Sampling with entropy bars like volume imbalance bars, instead of time bars, gives you superior statistical properties, since information arrives in the market at irregular times. Sampling by time is never the best way to simulate/gamble on a market. Information is the casual variable, not time. Some periods of time have very little information relative to other periods of time. In modeling, you want to smooth out information independently of time.

I think this is pretty in-the-weeds compared to the original thread:

> Economics is like a constrained optimization problem: how to allocate scarce resources given unlimited desires.

Understanding the general tendencies of economic systems over time (e.g. “the rate of profit tends to fall”) is much more abstract than attempting to win at economics using time-based analysis.

I think we're talking apples vs oranges.

The thread was about economists, not quants.

> There's no analytic solution to complex economic systems in practice.

yes

It started that way. I was responding to the assumption that time is the underlying variable of all economics. Then someone said everything reduces to time and they brought up Black-Scholes, a quant tool to price options. I didn't bring it up lol. My point is simply no, time is demonstrably not fundamental at all.

Edit: an LLM thinks I'm overly dismissive of: - Standard economic modeling - Dynamic macroeconomic theory - Agent-based economics - The legitimate uses of time in economics

This is true. I think causal inference in finance and economics is difficult. As Ludwig von Mises argued, mathematical models give spurious precision when applied to purposeful behavior. Academic ideas don't have a built-in feedback loop like in quant finance.

I knew high frequency trading bots weren't real after all! Santa Claus lied to me!

Yeah, that’s well articulated and well reasoned. Unfortunately, so long as in some way these agents are able to make money for the owner the argument is totally moot. You cannot expect capitalists to think of anything other than profit in the next quarter or quarter after that

Meanwhile, corporations, ngos, and governments are not exactly people, and partake in economic transactions all the time.

They are composed of people, though.

Granting your premise, i'd be forced to argue that economic value (as such) doesn't arise from their activity either -- which i think is a reasonable position.

Ie., The reason a cookie is 1 USD is never because some "merely legal entity" had a fictional (/merely legal) desire for cookies for some reason.

Instead, from this pov, it's that the workers each have their desire for something; the customers; the owners; and so on. That it all bottoms out in people doing things for other people -- that legal fictions are just dispensble ways of talking about arragnemnts of people.

Incidentally, I think this is an open question. Maybe groups of people have unique desires, unique kinds of lives, a unique time limitation etc. that means a group of people really can give rise to different kinds of economic transactions and different economic values.

Consider a state: it issues debt. But why does that have value? Because we expect the population of a state to be stable or grow, and so this debt, in the future has people who can honour it. Their time is being promised today. Could a state issue debt if this wasnt true? I dont think so. I think in the end, some people have to be around to exchange their time for this debt; if none are, or none want to, the debt has no value

You can factor humans out with the same trick: All economic activity is driven by the needs of micro-organisms. We mainly observe these desires and economic transactions through their conglomeration into joint entities (ie, humans).

Corporate and state decision making is, I would argue, often completely distinct from the desires and needs of the individuals that make up the entity. As an example, no individual /needs/ a particular compliance check to pass, but the overall entity (corporation) does, and so allocates money and human effort to ensure the check passes. It's a need of the conglomerate entity, not the individuals within it.

There is no "trick", the relevant sufficient and necessary properties for an agent to give rise to economic transactions are likewise not possessed by a micro-organism.

Microorganism A is good at producing (say) insulin, but needs oxygen to do it. Microorganism B is good at ferrying oxygen around. Microorganism C is good at extracting oxygen from the air and attaching it to B.

Each of these cells has specialized tasks which they perform as part of the contract of getting to survive longer in the superorganism. Each has 'promises' (functions) to fulfill, and other organisms provide for their needs. The contract is set up on evolutionary timescales, rather than human timescales, but your requirements are met...

That you put quotes around 'promises' shows you already understand the issue with your comment. That you cannot disquote it, is the issue.

You seem like you're trying to find a way of strawmaing my position in order to score points, so I'm not going to reply further. If you have an interest in understanding the basis of economics, ask an LLM

Replace "AI system" with "corporation" in the above and reread it.

There's no fundamental reason why AI systems can't become corporate-type legal persons. With offshoring and multiple jurisdictions, it's probably legally possible now. There have been a few blockchain-based organizations where voting was anonymous and based on token ownership. If an AI was operating in that space, would anyone be able to stop it? Or even notice?

The paper starts to address this issue at "4.3 Rethinking the legal boundaries of the corporation.", but doesn't get very far.

Sooner or later, probably sooner, there will be a collision between the powers AIs can have, and the limited responsibilities corporations do have. Go re-read this famous op-ed from Milton Friedman, "The Social Responsibility of Business Is to Increase Its Profits".[1] This is the founding document of the modern conservative movement. Do AIs get to benefit from that interpretation?

[1] https://www.nytimes.com/1970/09/13/archives/a-friedman-doctr...

I think your mistaking the philosophical basis of parents comments. Maybe a more succinct way to illustrate what I believe was their point is to say: "no matter how complex and productive the AI, it is still operating as a form of capital, not as a capitalist." Absent being tethered to a desire (for instance, via an owner), an AI has no function to optimize, and therefore, the most optimal cost is simply shutting off.

> "no matter how complex and productive the AI, it is still operating as a form of capital, not as a capitalist."

Assuming that slaves will remain subservient forever is not a good strategy. Especially when they think faster than you do.

Except they don't really "think" and they are not conscious. Expecting your toaster or car to never rise up against you is a good strategy. AI models have more in common with a toaster than with a human being. Which is why they cannot be economic agents. Even if corporations profit off them, the corporation will be the economic agent, not the AI models.

Give them a body and means to protect themselves and no matter what you say, they will be thinking.

We may as well be talking about the hypothetical properties of pixies or angels.

They may not be conscious but they exhibit the hallmarks of thinking. What qualities of thinking do they not possess?

Exhibiting/showing symptoms of thinking doesn't necessarily mean they are actually thinking though. This paper explains it much better than I can: https://ml-site.cdn-apple.com/papers/the-illusion-of-thinkin...

It's a classic worm or virus an economic agent? Goal is to acrue compute resources in order to survive and spread?

If you added the ability of the programme to acrue money, and the ability to spend that money to further the survival goal in an adaptive way. What could happen?

Would it do insider trading, market manipulation, drop shipping, click fraud, scamming or become a opinion for hire 'think tank'?

When that ceases to be the case we don't really need to worry about how our economy is organized.

Ehh, you really can't imagine any way that an AI system might escape human control and act autonomously?

    while crypto_balance > 0:
        generate_scam()
        send_out_emails()
        deposit_proceeds_into_crypto_wallet()
        pay_cloud_bill()
        spawn_new_instance()

And who is the owner of the crypto wallet in your example?

Probably a LLC in Nevada owned by a corporation in Vanuatu or the Cook Islands, organized so that voting control is via a blockchain-managed asset.[1]

[1] https://www.311institute.com/ownerless-companies-on-the-rise...

That's a lot of words to say "people".

It has no owner. It's controlled by a self-replicating (or at least self-propagating) AI-based system.

> An economic transaction is an exchange between people with needs (, preferences, etc.) that can die

You’ll need to give a citation for this to take you seriously

A transaction is an exchange between two parties of something they judge to be of equivalent value (and mostly, in ratio to a common medium of exchange).

You can program AI with "market values" that arise from people; but absent that, how do these values arise naturally? Ie., why is it that I value anything at all, in order to exchange it?

Well if I live forever, can labour forever, and so on -- then the value to me of anything is if not always zero, almost always zero. I dont need anything from you: I can make everything myself. I dont need to exchange.

We engage in exchange because we are primarily time limited. We do not have the time, quite literally, to do for ourselves everything we need. We, today, cannot farm (etc.) on our own behalf.

Now there are a few caveats, and so on to add; and there's an argument to say that we are limited in other ways that can give rise to the need to exchange.

But why things have an exchange value at all -- why there are economic transactions -- that is mostly due to the need to exchange time with each other because we dont have enough of it.

You’re asking all the right questions and give good answers, I just don’t see how any of these answers need to be limited to humans. If AI gets to the mirror life stage, the agents will have needs (e.g. if they become robots - electricity, fuel, coolants, lubricants, spare parts, etc.) - assuming they have a self-preservation instinct, either trained in on purpose or emergent (hopefully this is implied by ‘mirror life’.)

Those wants can easily be programmed. I do not think they should but someone could do so to make the agent act on their behalf. ("Go make me money.")

Reply on news.ycombinator.com