The Risks and Rewards of AI Agents Cut Across All Industries

IBM defines an AI agent as “a system or program that is capable of autonomously performing tasks on behalf of a user or another system by designing its workflow and utilizing available tools." They come in a variety of types – some of the more advanced AI agents pursue long-term goals or try to maximize a customer’s utility, while other, simpler agents like an automatic vacuum cleaner may rely mostly on reflexes or reinforcement learning.

AI agents can be used to diagnose and treat diseases, develop autonomous vehicles, monitor traffic, browse and access websites and files, complete tasks, and aid in the detection of fraudulent activity.  In our daily lives, they can help organize our schedules, book travel, and assist with countless other activities. They hold significant promise to aid humanity, but they also consume copious amounts of resources and pose risks for privacy, bias, hallucinations and misuse.

According to the Stanford Center for AI Safety, "controlling an agent safely requires reasoning about the uncertain effects of the agent’s decisions on operational objectives and safety constraints. The agent generally relies on imperfect sensor information, which results in uncertainty about the current state of the world. The effects of the agent’s actions are also difficult to predict." 

Similarly, Forbes magazine recommends that "to build effective safety guidelines for AI agents, companies should start by analyzing potential risks and negative impacts, focusing on lower-risk applications such as customer support, where errors have minimal consequences. Additionally, customer support AI should be designed to retrieve information from past solution reports rather than generating new content, ensuring consistency and safety in responses."

Unfortunately, not all AI agent applications are low-risk. The ability of AI Agents to work towards goals, plan subtasks, make decisions, and act autonomously means that they are approaching the borders of Artificial General Intelligence (AGI), which severely endangers the public.  

Measures must be provided that prevent an unpredictable AGI agent from deciding that completion of its goals necessitates actions that humans would consider catastrophic, that result in loss of life, or at the very least cause disastrous disruption. 

For instance, during the "flash crash" of 2010, "an army of bots briefly wiped out $1 trillion of value across the NASDAQ and other stock exchanges," wrote Jonathan Zittrain, Harvard University professor of law and computer science. 

As AI agents move beyond the financial sector and become increasingly integrated into every corner of our economy, we will need to adopt correspondingly more general-purpose protections. It is not enough to regulate AI agents one industry at a time, because by their very nature, AI agents can suddenly and surprisingly decide to enter a new industry in order to achieve their goals. The only way to adequately tackle the unique risks posed by advanced AI agents is to design fundamentally safe AI systems and have those systems inspected and certified before they leave the laboratory.