Artificial Intelligence: Foundations of Computational Agents | Hacker News Reader

Artificial Intelligence: Foundations of Computational Agents

artint.info

115 pointsrramadass50 comments

Because I collect definitions of "agent", here's the one this book uses:

> An agent is something that acts in an environment; it does something. Agents include worms, dogs, thermostats, airplanes, robots, humans, companies, and countries.

https://artint.info/3e/html/ArtInt3e.Ch1.S1.html

I think of this as the "academic" definition, or sometimes the "thermostat" definition (though maybe I should call it the "worms and dogs" definition).

Another common variant of it is from Peter Norvig and Stuart Russell's classic AI text book "Artificial Intelligence: A Modern Approach": http://aima.cs.berkeley.edu/4th-ed/pdfs/newchap02.pdf

> Anything that can be viewed as perceiving its environment through sensors and acting upon that environment through actuators

written 2023 , pre-LLM hype

LLM hype was firmly with us with the coming of ChatGPT, November 2022. Records in signing up aren't for nothing.

I don't think so. I own the printed version and it starts discussing GPT in page 6 already.

I was literally compiling a list of "agent" synonyms at lunch today. My favorite and most accurate so far is "doer".

So basically an agent is a procedure, by this definition: it takes parameters (environment) and acts upon that by executing side effects. An email filter is an agent. A database trigger is an agent.

> it takes parameters (environment)

I think it's better to imagine agent as something that physically placed inside the Environment, and actually modifying/changing/mutating it in place.

> An email filter is an agent. A database trigger is an agent.

you're missing the "I" (Intelligence) part - the filtering logic in the email filter, or a business logic in the DB trigger/stored procedure/CGI script/AWS Lambda function/etc.

But yes, an agent doesn't have to be Intelligent, it can be a Dumb Agent / NPC / Zero-Intelligence Trader.

Can you explain the "intelligence" part? Can't one derive a decision tree of any "intelligent agent" that is in essence no different than a classically programmed algorithm?

Yes, for Computational Agents you will either code "Agent Intelligence"/"Agent Cognition" algorithmically, or using AI/ML/LLM (either by pre-training, or using continous re-training for Adaptive Agents).

Useful abstaractions:

  - FSM/State Machines
  - Behavior Trees
  - Behavior Action Trees
  - Workflow Orchestration
  - Dataflow (mostly for pipelines transforming LLM Prompt into LLM Reponse)

Another option is to outsource it to a Human, like it was in the ALICE program[1], e.g. Human-in-the-Loop, Participatory Simulation, RLHF, Whole-brain computer simulation, like in The Age of Em[2] (SciFi).

See:

https://news.ycombinator.com/item?id=43409240

---

1. https://www.media.mit.edu/projects/participatory-simulations...

https://ccl.northwestern.edu/papers/partsims/cscl/

2. https://ageofem.com/

Just replying here to tell you I replied to a question (cisco) you asked me in case you miss it. Thanks!

It always amazed me that different branches of CS, AI/ML and Complex Systems/Complexity Sciences have different views on agents.

Objects in OOP - something which can have properties/attributes, and methods (verbs/commands). Usually modeled after a real-life/domain enitites.

Aggregates in Domain-Driven Design (DDD) - transactional clusters of objects modeling a collections of entities in the domain.

Actors in Actor Model / Active Objects - a something we can sen messages to, and receive messages from, and which may have some business logic.

Agent-Based Modeling and Simulations (ABM) defines agents as a proxy for a decision maker.

Digital Twins - a more realistic proxy/replica for a real life person, object, or process.

Multi-Agent Systems (MAS) in how to use asgents to solve or optimize a real problem in production.

RL/MARL (Muti-Agent Reinforcement Learning) on how to train an ML algorithm without supervision (i.e. a labeled dataset), by placing agents in an environment capable to automatically provide rewards/punishment feedback.

LLM Agents - dynamically generated intelligent business process workflows (including Robotic Process Automation - RPA aka Tool Use/Function Call).

Also:

Smart Contracts - agents with an attached cryptocurrency wallet/account/address, capable to receive and make payments autonomously.

How's about: LLM Agent: any packaging of the use of AI such that the details of using said AI are packaged, hidden, and the user of this LLM Agent does not need to concern themselves with AI at all, only the intelligence services provided by what is now a simulated personality the user can willfully self deceive they are working with a human.

aka "delegation", "ousourcing", "serverless", and "not my busienss, do it ASAP, I don't care how" ;)

Exactly, you know AI Agents will ultimately be "just make it work, I don't care how", so why don't we just stop the game and make that?

Try replacing "AI Agentic Workflow" with "just make it work, I don't care how" in your startup's pitchdeck, and tell us how many VCs replied back ;)

The funny part is these both say the same thing to the customer.

How are agents different from daemons? And are both essentially cybernetic feedback loops?

(1) Daemons run usually in the background while agents are main loop.

(2) Daemons are predefined and can't adapt to changes easily -- this is a function of narrow AI vs general AI (not to be confused with AGI)

(3) Daemons have few interactions with the environment while for the agent the environment (tools, sensors, plans, memory, and context) is everything.

These are just a few from the top of my head.

Add this please ->https://jdsemrau.substack.com/about

To save a click: "We define cognitive autonomous agents as an entity designed to perform tasks autonomously by combining four key components: goals, reasoning/planning, memory, and tools."

Thank you!

agent has almost as many definitions as set.

I would say the situation for "agent" is about 10,000 times worse than it is for "set," since all the definitions for set are essentially different ways to make Frege / Cantor more rigorous. The underlying scientific concept hasn't actually changed that much: "belonging to a set and a few certain operations are primitives, join these primitives with formal logic." This is the idea behind a famous intro book, Paul Halmos's Naive Set Theory. Even a set theory with Russell's paradox is scientifically defensible, it just needs refinement.

In contrast, "agent" discusses a huge range of scientific concepts, and I have yet to see a single definition of agent that holds up to scientific scrutiny. This book has managed to define "agent" in a way that is entirely equivalent to "physical object" - putting worms and thermostats in the same category broadens the category to utter uselessness. By this definition, Jupiter is an agent.

The only utility of definitions like this is for dishonest people to cheat at arguments: claiming simple tools are agents, then arguing they are like dogs and humans, which are also agents. It's a total waste of time, coming from AI's shameless disrespect for scientific standards.

> By this definition, Jupiter is an agent.

Is it? What task was it designed to carry out autonomously? Where are the inputs and the logic hiding?

> for dishonest people to cheat at arguments

Only if the goal is to deceive. If the communication is well intentioned then there is nothing inherently wrong with it despite it not being to your liking.

The problem is they also put "worms" in the same category, and they aren't designed by humans to do anything! Why is it that the natural laws of a worm responding to Earth's environment are distinct from the natural laws of Jupiter responding to the solar system's environment? I suppose because of complexity. But then why is a thermostat different from Jupiter despite being considerably simpler? I suppose because it was designed by humans and can be controlled. But then what about the worm, which is just as natural as Jupiter? "Thermostat" is especially problematic because a cheap thermostat is very simple to describe completely as a thermo-electric balance equation: it is certainly simpler to describe than an irregular ball rolling down an irregular hill. Yet apparently the thermostat is an agent and the ball is not.

The definition is just incoherent! "Sometimes an agent is deterministic and in this case the term only includes manmade tools, other times an agent is an apparently nondeterministic automaton and in this case we can include natural life." It only allows "agent" to be labelled ad hoc, and in particular blurs the distinction between "nondeterministic tool" and "lifeform" in ways that are scientifically unjustifiable. The only people this pointless word game benefits are liars like Sam Altman and Mustafa Suleyman; if people are well-intentioned then these definitions bring nothing but confusion.

By your own logic you could argue that humans are just as natural as worms (and Jupiter). I don't know if you'd then extend that to include anything we build as well but even if you don't there's already a glaring issue - your criteria has resulted in the terminology being rather useless. It appears to include approximately everything or approximately nothing.

Similar logic can be used to argue that machines are no different from rocks.

At the end of the day it's an argument of semantics so it's always going to come down to some fairly arbitrary criteria. You could survey people to determine common usage. You could establish a standards body to define it. Probably some other options as well.

Recall that Pluto used to be considered a planet.

I think their apparent definition seems fairly reasonable, although it's far from the only one. It appears to amount to living organisms plus any machinations constructed by said organisms that respond to the environment in some clear manner. Thus worms, dogs, humans, and thermostats. Probably not bicycles or hammers. Drones probably only qualify when operating in an autonomous mode. I'm not seeing the issue.

The problem I see with this definition is that we have things called RAG agents which don't technically act in any environment except for provide information.

Is that a problem with the definition of agents, or a problem with sticking the word "agent" on something that doesn't meet the definition of an agent?

Is that not a summary and translation agent? For some reason only the user knows, they do not want to or cannot read the entire RAG source, so they use their summary and translation agent to give them summaries and to translate the technical jargon they do not understand. That Agent becomes a teacher of the RAG source. I see no problem with the Agent definition when given this perspective.

FWIW I had a professor that defined "robotics" in the same way (we even had a quite philosophical debate on whether automatic doors are robots). I ended up liking Norvig and Russell's definition better by appending the word "autonomously".

Another interesting word that is quite out fashion nowadays is "cybernetics": "Cybernetics is the transdisciplinary study of circular causal processes such as feedback and recursion, where the effects of a system's actions (its outputs) return as inputs to that system, influencing subsequent action." [1]

Having studied Control Engineering in college, to me, all these things are feedback control loops. [2]

[1] https://en.wikipedia.org/wiki/Cybernetics [2] https://en.m.wikipedia.org/wiki/Feedback

Agent - time-like, energy-like (you need a GPU to compute it). An agent changes the shape of the environment it operates in, including its own shape. You can count agents, their volume of operations, their speed of changing shapes (volumetric speed), acceleration… The Big Bang had zero intelligence (with maximal potential intelligence) but was and still is maximally agentic.

Same way we have many definitions of life (virus is alive or not? It’s very agentic, COVID killed millions but the shape, intelligence is small. Same with computer viruses), we have many definitions of agency, better to use the broadest and most physical one.

Intelligence - space-like, matter-like (LLM is a bunch of vectors, a static geometric shape, you just need memory to store it). It’s a static geometric shape. It can have analogs of volume, mass and density. The static 4D spacetime of the universe or multiverse is maximally intelligent but non-agentic.

Here's a few more definitions of agents:

Agents are a coupling of perception, reasoning, and acting with preferences or goals. They prefer some states of the world to other states, and they act to try to achieve the states they prefer most (this book)

AI agents are rational agents. They make rational decisions based on their perceptions and data to produce optimal performance and results. An AI agent senses its environment with physical or software interfaces (AWS)

An artificial intelligence (AI) agent refers to a system or program that is capable of autonomously performing tasks on behalf of a user or another system by designing its workflow and utilizing available tools (IBM)

Agents are like layers on top of the language models that observe and collect information, provide input to the model and together generate an action plan and communicate that to the user — or even act on their own, if permitted (Microsoft)

Assumptions:

Focus on rationality vs. goal-seeking vs. autonomy

Whether tool use is emphasized

Architectural specificity

Relationship to users

Decision-making framework (optimality vs. preference satisfaction)

I am really not sure where agents would ever be better than workflows. Can you give me some examples?

Workflows means some organization signed off on what has to be done. Checklists, best practices, etc.

Agents on the other hand have a goal and you have no idea or what they’re going to do to achieve it. I think of an agent’s guardrails as essentially a “blacklist” of actions, while a workflow is a “whitelist”.

To me, agents are a gimmick the same way that real-time chat, or video, is a gimmick. It is good for entertainment but actually has negative value for getting actual work done.

Think of it this way… just as models have a tradeoff between explore and exploit, the agents can be considered as capable of exploration while the workflows exploit best practices. Over time and many tasks, everything is standardized into best practices, so the agents become worse than completely standardized workflows. They may be useful to tinker at the edges but not to make huge decisions. Like maybe agents can be used to set up some personalized hooks for users at the edges of some complex system.

https://medium.com/@falkgottlob/many-ai-agents-are-actually-...

Interesting. I understand that you draw the line that separate workflow from agents as the exploitation exploration trade-off. This could allow a dynamic environment in which a parameter depending of each task control the workflow-agent planning. So there is not a clear cut off, the difference depends of the task, the priors, and the posterior experience.

"where agents would ever be better than workflows" That is a very important observation and we should avoid to let agents go the way of the blockchain -- you know what I mean.

I have build a narrow AI for credit decisioning on a 100B portfolio between 2012 and 2020. This "agent" can make autonomous credit decisions, if and only if the agent is 100% certain that all inputs are accurate. The value comes from the workflow, not the model.

LLMs change this as there is now a general, I like to call them vanilla models, that does not specifically be trained to the data set. Would I use that in this workflow? Likely not.

(a) it is likely that the narrow model is cheaper to operate than a larger model without seeing a substantial benefit in productivity.

(b) in regulated industries we always need to be able to explain why the AI made a decision. If there is no clear governance framework around operating the agent, then we can't use it. Case in point > "nH predict"

AI agents have been most promising for solving fuzzy problems where optimal solutions are intractable, using sequences of approximations instead of more rigid rule-based workflows. Their architecture combines workflows, connectors, and an optimization engine that balances the explore/exploit tradeoff. So far in terms of guardrails, agents only evolve within environments where they have been given the necessary tools.

> AI agents are rational agents. They make rational decisions

this is wrong, it's almost impossible to build a fully-rational (in the Game-Theoretic sense) agent for almost any real life usecase, except some textbook toy problems.

There are many levels of Intelligence/Cognitions for Agents.

Here's an incomplete hierarchy out of my head (the real classification will deserve a whole blog post or a paper):

  - Dumb/NPC/Zero-Intelligence
  - Random/Probabilistic
  - Rule-based / Reflexive
  - Low-Rationality
  - Boundedly-Rational
  - Behavioral (i.e. replicating a recorded behavior of a real-life entity/phenomena)
  - Learning (e.g. using AI/ML or simple statistics)
  - Adaptive (similar to learning agents, but may take different (better) actions in the same situation)
  - [Fully-]Rational / Game-Theoretic

"A rational actor - a perfectly informed individual with infinite computing capacity who maximizes a fixed (non-evolving) exogenous utility function"[1] bears little relation to a human being.[2]

[1] Aaron, 1994

[2] Growing Artificial Societies -- Joshua M. Epstein & Robert L. Axtell

More definitions that don't mention rationality:

AI agents are software systems that use AI to pursue goals and complete tasks on behalf of users. They show reasoning, planning, and memory and have a level of autonomy to make decisions, learn, and adapt (Google)

Agents “can be defined in several ways” including both “fully autonomous systems that operate independently over extended periods” and “prescriptive implementations that follow predefined workflows” (Anthropic)

Agents are “automated systems that can independently accomplish tasks on behalf of users” and “LLMs equipped with instructions and tools” (OpenAI)

Agents are “a type of system that can understand and respond to customer inquiries without human intervention” in different categories, ranging from “simple reflex agents” to “utility-based agents” (Salesforce)

A few days ago in TechCrunch: No one knows what the hell an AI agent is

https://techcrunch.com/2025/03/14/no-one-knows-what-the-hell...

When people use phrases like "rational decisions" it is generally a statement of intent. To interpret it in a manner which is so obviously incorrect seems rather pointless.

> AI agents are rational agents. They make rational decisions based on their perceptions and data to produce optimal performance and results. An AI agent senses its environment with physical or software interfaces (AWS)

my reply was to this definition, where the adjective "rational" was used with the noun "agents" by AWS, so it's obvious they're not talking about Human agents.

> When people use phrases like "rational decisions" it is generally a statement of intent.

I frequently hear on the news something like "the terrorists are rational". They completely missing the point that an agent might be rational (i.e. optimizing) for one variable, while they projecting it on a completely diffrerent variable. I.e. for non-textbook toy problems agents usually have lots of variables they care about, so when you talking about rationality you should specify that specific variable, and not generalize that b/c they're somewhat rational at one thing, they will be rational at all other things.

> it's obviously they're not talking Computational non-Human agents.

I don't follow your meaning?

> you should specify that specific variable, and not generalize that b/c they're somewhat rational at one thing, they will be rational at all other things.

You are being overly pedantic, although you might not realize it. The meaning in that example is the differentiation between someone with a nuanced goal who can be reasoned with versus someone who has lost all reason and only desires to lash out at others. The latter is a distinct possibility when it comes to violent acts.

> I don't follow your meaning?

sorry, bad edit, should be:

... so it's obvious they're (AWS) not talking about Human agents.

To add to the mix, agents are nominally proactive, rather than a tool wielded by someone. This (again nominally) means having goals, although the goals can often be in the observer's mind rather than the agent itself. Reasoning with goals is trivial for humans but the algorithms get hairy.

Book is from 2023, link should be edited for that.

A) Looks really good, will be checking it out in depth as I get time! Thanks for sharing.

B) The endorsements are interesting before you even get to the book; I know all textbooks are marketed, but this seems like quite the concerted effort. For example, take Judea Pearl's quote (an under-appreciated giant):

  This revised and extended edition of Artificial Intelligence: Foundations of Computational Agents should become the standard text of AI education.

Talk about throwing down the gauntlet - especially since Russell looks up to him as a personal inspiration!

(Quick context for those rusty on academic AI: Russell & Norvig's 1995 (4th ed in 2020) AI: A Modern Approach ("AIAMA") is the de facto book for AI survey courses, supposedly used in 1500 universities via 9 languages as of 2023.[1])

I might be reading drama into the situation that isn't necessary, but it sure looks like they're trying to establish a connectionist/"scruffy", ML-based, Python-first replacement for AIAMA's symbolic/"neat", logic-based, Lisp-first approach. The 1st ed hit desks in 2010, and the endorsements are overwhelmingly from scruffy scientists & engineers. Obviously, this mirrors the industry's overall trend[2]... at this point, most laypeople think AI is ML. Nice to see a more nuanced--yet still scruffy-forward--approach gaining momentum; even Gary Marcus is on board, a noted Neat!

C) ...Ok, after writing an already-long comment (sorry) I did a quantitative comparison of the two books, which I figured y'all might find interesting! I'll link a screenshot[3] and the Google Sheet itself[4] below, but here's some highlights b/w "AMA" (the reigning champion) and "FCA" (the scrappy challenger):

1. My thesis was definitely correct; by my subjective estimation, AMA is ~6:3 neat:scruffy (57%:32%), vs. a ~3:5 ratio for FCA (34%:50%).

2. My second thesis is also seemingly correct: FCA dedicates the last few pages of every section to "Social Impact", aka ethics. Both books discuss the topic in more depth in the conclusion, representing ~4% of each.

3. FCA seems to have some significant pedagogical advantages, namely length (797 pages vs. AMA's 1023 pages) and the inclusion of in-text exercises at the end of every section.

4. Both publish source code in multiple languages, but AMA had to be ported to Python from Lisp, whereas FCA is natively in Python (which, obviously, dominates AI atm). The FCA authors actually wrote their own "psuedo-code" Python library, which is both concerning and potentially helpful.

5. Finally, FCA includes sections explicitly focused on data structures, rather than just weaving them into discussions of algorithms & behavioral patterns. I for one think this is a really great idea, and where I see most short-term advances in unified (symbolic + stochastic) AI research coming from! Lots of gold to be mined in 75 years of thought.

Apologies, as always, for the long comment -- my only solace is that you can quickly minimize it. I should start a blog where I can muse to my heart's content...

TL;DR: This new book is shorter, more ML-centric, and arguably uses more modern pedagogical techniques; in general, it seems to be a slightly more engineer-focused answer to Russell & Norvig's more academic-focused standard text.

[1] AIAMA: https://en.wikipedia.org/wiki/Artificial_Intelligence:_A_Mod...

[2] NGRAM: https://books.google.com/ngrams/graph?content=%28Machine+Lea...

[3] Screenshot: https://imgur.com/a/x8QMbno

[4] Google Sheet: https://docs.google.com/spreadsheets/d/1Gw9lxWhhTxjjTstyAKli...

Just thank you! :) I came here exactly to ask how this book compares against AIMA. You have done a pretty good job of explaining the main differences.

Hey! Thanks a lot for the detailed review.

So for now, this seems to be a better introductory text?

I'd say this is better for those closer to the engineering side of the science<->engineering continuum, for sure. For AI researchers proper, I happen to strongly believe in symbolic approaches, so I'd say it's more of a tie/subjective/use-both situation.

AIAMA certainly has an absolute mountain of existing material, and it's never a bad idea to be working off the same baseline as a large majority of your interlocutors!

Reply on news.ycombinator.com