Today I am looking at the evolution of AI agents over time: their architecture, their real world applications, and delving into the discussion of if they should be autonomous or not, as there is many reasons for and against this new wave of freedom that AI seems to have.

What are AI agents?

An AI agent is a system/program that is able to autonomously perform tasks on behalf of the user/different system by designing its own workflow and using the tools it has been provided.

They can, as all LLMs, provide a range of services, such as natural language processing, decision making, problem solving, and executing actions (sometimes in external environments). The difference between an agent and a normal LLM is that an AI agent need no specific input for each task, only needing a goal, with no real predefined parameters. This approach allows agents to decompose complex problems into manageable subtasks, reason over available information, utilize appropriate tools, and learn from feedback while maintaining context across interactions.

They have been categorised (by AWS) into a handful of types:

• Reflex agents - Operate based on predefined rules, good for straightforward tasks in a stable observable environment (Thermostat controller, basic chat bots)
• Model agents - evaluate probable outcomes before deciding, maintains state information not directly observable in current environment for better decision making (Navigation systems, recommendation systems, diagnostic systems)
• Goal based agents - compare different approaches to achieve desured outcomes, has goals and selects actions specifically to achieve them. It will actually compare different approaches instead of just reacting to stimuli (Natural language processors, robotics applications)
• Utility based agents - maximise specific value metrics, refines goal approach by assigning values on world states, so can make nuanced decisions between 2 different goals when they conflict (Financial trading systens, resource allocation algorithms, travelling planners)
• Learning agents - continuously adapt based on experience, improves itself by modifying itself based on feedback (Better recommendation systems, conversational agents that learn from interaction histories)
• Heirarchal agents - coordinate across multiple levels of abstraction. They decompose complex tasks into simple tasks, then assign them to lower level agents (management of complex workflows, multi agent planning systems, enterprise automation)

Despite AI agents having incredible improvements in a short period of time, they still have a number of issues that still plague them:

• The primary focus of all AI agents is accuracy, instead of other factors such as cost effectiveness, applicability or reproducability
• Needs are incredibly different between developers in areas (model developers vs application developers), so it can be very hard to figure out which agent is the best for a specific use case.

AI Architecture

There is multiple components that, when combined, enable autonomous perception, reasoning and action. Let me give a brief overview.

Perception Mechanisms

This component is the main interface between the agent and the environment, allowing the collection and processing of context. Normally has a natural language understander that interprets text into processing. Can also be computer vision systems or speech recognition for example. Important for the agent to act as it can only work with the information it recieves

Knowledge representation systems

This system provide the structures and mechnaisms for the data storage and retrieval within an AI agent. Modern systems are a combination of symbolic and distributed representations, allowing for the full use of the AI’s ability in problem solving and reasoning. Also differentiates between declarative knowledge, procedural knowledge and meta knowledge.

Reasoning and decision making modules

The reasoning and decision making part of the AI agent. Allows the processing of available information, evaluate and select the appropriate actions. These actions obviously need inference and various types of reasoning, all of which this module can handle (with some slight innacuracies). The actions are selected with utility based approaches that determines if an action is good or bad depending on the agents goals.

Action selection and execution components

THis module translates the decisions and actions made into behaviours that affect the environment. This, in general for LLMs, is generating responses, using tools/APIs or asking clarifying questions. In other agents this can be physical movement or changes to environmental parameters. This action selection is balancing all available information and requirements, and typically has a ‘heirarchal action structure’ (high level actions broken down into sequences of simple tasks).

Learning and adaptation mechanisms

This is the module that allows the agents to improve their performance overtime, primarily due to feedback and experience in completing tasks. The way it improves is based on the system, but it can be done in a few ways:

• Supervised learning
• Reinforcement learning
• self-supervised learning
• Meta learning.

This creates a positive feedback loop that allows for proper maintenance and evolving the AI to suit more tasks in a better manner.

And there exist some specialised modules:

Planning modules

Allows agents to construct sequences of actions that achieve desired goals, using a variety of planning algorithms (such as ToT, CoT reasoning, etc). This allows for more complex tasks (that require coordinated sequences of actions instead of just reaction) to be completed.

Memory management systems

This system maintains information across multiple interactions, allowing context to stay present (and allows the learning and adaptation mechanisms to work efficiently). It categorises memory into categories:

• Working memory - task relevant information
• Episodic memory - storing records of interactions
• Semantic memory - conceptual knowledge
• Procedural memory - storing action sequences or skills

This allows a level of continuity that allows AI agents to be autonomous.

Self-monitoring and metacognitive components

Agents can evaluate their own performance, recognize when they lack information or have low confidence, and adjust their approach accordingly. This includes uncertainty estimation, confidence calibration, and reasoning about knowledge gaps - this is essential for operating effectively in complex environments.

Communication interfaces

These enable agents to interact with users and other agents across multiple formats (text, speech, visual, multimodal) while adapting to different user needs and expertise levels. They balance being informative, efficient, transparent, and engaging.

Safety and alignment mechanisms

These ensure agents behave appropriately and align with human values through action constraints, decision verification, interpretability features, and value alignment techniques. This is increasingly critical as agent capabilities grow and get deployed.

Has this changed over time?

This architecure has changed drastically over time.

Originally, the earliest approaches were to implement Rule Based systems, which is where the systems have rule engines that match conditions via if then statements. it was easily understood and trackable, modular and precise, but it was not scalable or adaptable for diverse situations, only for the specific rules it had been given, not for reasoning. It is still present but only for rigid environments.

Another approach is via statistical and probabilistic methods, allowing for reasoning under uncertainty, learning from data and making decisions with not the entire picture of the situation (Such as using bayesian and markov techniques for relationshops and decision making). This allowed agents to handle noisy/vauge inputs, and make decisions based upon uncertainty and was able to improve upon itself. While it can work well, it was unable to complete complex logical reasoning and involve expertise from previous work.

Neural networks is another solution. It has improved perception, language understanding and pattern recognition. It also has reduced the need for manual engineering to add features and can discover its own useful representations directly from data, which means it has got good at very general situations where previously LLMs were unable to handle.

Combining these types of approach is the new modern standard, as all 3 have benefits that the others do not have. Different agents’ functions may use different approaches (i.e. an agent might use neural networks for perception and language understanding, probabilistic methods for handling uncertainty and decision optimization, and rule-based components for implementing safety constraints and business logic).

Real World Application

AI agents are used in a number of ways, transforming businesses and increasing productivitiy:

• Customer service and support
• Business process automation
• Decision support systems
• Supply chain optimisation
• HR and talent management support
• Helping financial operations
• Knowledge management and information access
• Sales and marketing abilities
• IT operations and infrastucture management
• Coding assistants

Personally, they are used in many ways:

• Task management and organisation
• Information retrival
• Creative collaboration
• Learning tools
• Communication assistance
• Personal finance management
• Health/wellbeing support

It has also been used in a number of field-specific ways, such as:

• Being used in the Healthcare system, being used to support in clinical decisions, monitoring patient, helping with the administration and treatment planning; helping reduce the time to diagnosis by 37%, and accuracy has increased by 23%.
• In the finance industry, it has been used to help with risk assessment, fraud detection and regulation compliance. As an agent can analyse market data, it has reduced the volatility of a investment management firm by 12%, but increased the returns by 1.8% annually.
• It is used frequently in the software development industry, regularly sitting in a codebase (such as copilot in VS code, or Coder which Cloudsecure help run!) to help create and debug code. This leads to a lot of automation in the coding process which is incredibly useful and time effective. A study of developers who used AI agents saw a 35% reduction in time developing new features, with less defects by 27%. This massive increase in cost efficiency allows for developers to create better features faster.

These are just a few in depth examples, but its also being used in:

• Scientific research
• Legal research settings
• Engineering applications
• Education and training applications
• Agriculture and environmental management
• Urban architecture.