Automated fraud and automated attacks: How AI agents are changing cybersecurity

Not long after AI agents became widely available, two major frauds were disclosed within weeks of each other. In one, a person with no technical expertise, working completely alone, used the chatbot Claude to “vibe code” malware, identify targets, and launch an attack that hit 17 different organizations. The other case found that more than 320 companies had unknowingly hired North Korean operatives as remote workers after being fooled by false personas created by AI.

We’ve entered an era where complex fraud no longer requires technical skill, and once set up, machines no longer need to wait for instructions. AI agents increasingly make decisions faster, more efficiently, and sometimes more effectively than humans. AI agents now write emails, negotiate contracts, respond to customers, and even run parts of financial systems with minimal oversight.

As with every leap in technology, the same innovation empowering organizations can also be weaponized. The rise of AI agents has brought a new form of cybersecurity risk. Automated fraud and automated attacks happen faster, adapt instantly, and scale far beyond human capacity. With this new technology, fraudsters don’t just hack systems, they train AI agents to perform automated attacks relentlessly until they eventually find a crack in the armor to exploit. The good news is that strong defenses can still be built. In this article, we will explain the basics of AI agents, how fraudsters utilize them as tools, and opportunities to leverage them to prevent and detect fraud.

• Understanding AI agents and their capabilities
• The bright side: AI agents as protectors
• The dark side: Rise of automated fraud
• AI agents behind automated attacks
• How AI agents are used to commit fraud
• AI agents used to detect and prevent fraud
• The role of internal audit
• Winning the AI arms race

AI agents behind automated attacks

While traditional cyberattacks rely on manual configuration and execution, automated attacks utilize AI agents that continuously adapt by scanning networks, adjusting tactics, and identifying vulnerabilities on their own. AI agents gather information by crawling the internet for vulnerable endpoints, analyzing patch histories, and mapping expected user behaviors. Once a target is found, the AI selects or generates an appropriate attack pattern or exploit. The agent learns from each attempt, adjusting its attack and improving its success rate over time. The agent uses natural-language generation to disguise phishing content or modify malware signatures to evade antivirus tools. Examples of automated attacks include:

• AI-driven botnets: Modern botnets now use AI to coordinate attacks intelligently, adjusting frequency to bypass detection systems.
• Adaptive malware: Some strains rewrite their own code based on system defenses.
• Autonomous social engineering: Chatbots posing as customer service agents lure victims into revealing credentials.

Automated attacks are not only faster, but also smarter, stealthier, and self-improving. The ability to run continuously while learning from failures and adjusting the attack pattern makes malicious AI agents a significant threat that everyone in an assurance role needs to understand.

How AI agents are used to commit fraud

While early AI fraud relied on simple automation, modern fraudsters now deploy custom-trained AI models optimized for deception. They even created tools to build automated fraud agents on platforms that work like ChatGPT. In underground forums, FraudGPT and WormGPT have appeared as malicious AI tools capable of generating phishing scripts, fake websites, or social engineering templates. These dark AI agents can:

• Write self-adjusting malware that is undetectable by standard security tools
• Craft sophisticated business email compromise messages
• Simulate financial transactions to bypass fraud detection thresholds
• Create synthetic identities to pose as a human

Dark-web vendors now rent access to malicious AI models for as little as a few dollars per hour, enabling virtually anyone to utilize advanced technology for fraudulent purposes without any expertise, simply by prompting an AI agent.

AI agents used to detect and prevent fraud

Fortunately, the same technology is being used to fight back. AI-driven fraud detection systems analyze massive volumes of transactions in real time, learning normal patterns, and flagging anomalies such as:

• Sudden spending spikes
• Unusual IP or geolocation access
• Repeated login failures
• Micro-transactions testing card validity

Older security tools like intrusion detection systems and data loss prevention are good at identifying anomalies, quarantining the problem, and sending alerts to the right people. The AI agent-based solutions go further by acting. Many systems automatically freeze transactions, request multi-factor authentication, or escalate suspicious activity for review.

Newer AI models can now incorporate behavioral analysis to differentiate between human and bot activity. A simple example would be to look for a transaction from a vendor who normally invoices at the end of the month, but now you receive invoices mid-month with a new bank account. A more complex example could be analyzing inbound emails for text that includes language, tone, links, attachments, or other contextual signals for each individual that do not match a previously established baseline for how that person typically sounds.

The role of internal audit

As AI agents become integral to business operations, the control environment must evolve to address automated fraud and automated attack risks. Even as AI grows more autonomous, humans remain accountable. AI agents may detect anomalies, but humans must interpret context and consequences.

As internal auditors step into this new field, there is much to learn. For now, remember the three principles of responsible AI oversight:

1. Transparency: Every AI decision should be explainable.
2. Accountability: Organizations must own the outcomes of their AI systems.
3. Proportionality: Not every task should be automated, especially those involving ethical judgment.

Internal auditors play a crucial role in this process, validating AI controls, assessing ethical implications, and ensuring the organization’s AI strategy aligns with its governance and risk appetite. Internal auditors should evaluate AI systems with the same rigor as financial controls. Key controls include:

• Access controls: Who can deploy or modify AI models?
• Change controls: Are all changes tested, approved, and deployed appropriately?
• Audit trails: Are actions traceable and explainable?
• Bias and transparency: Does the AI make fair, documented decisions?
• Human-in-the-loop oversight: Are humans still able to override AI decisions?

Start any review by evaluating these basic controls and then expand into the automated and control solutions enabled by AI agents. Then, set up walkthroughs with IT Security to understand the organization’s current threat detection tools and automation, such as the Intrusion Detection System (IDS), Data Loss Prevention (DLP), Security Information and Event Management (SIEM), and Security Orchestration, Automation, and Response (SOAR). Understanding how these systems operate to detect and prevent cyberattacks will guide your options for testing effectiveness.

Winning the AI arms race

The rise of AI agents is both a blessing and a curse. On one side, autonomous systems promise a future of efficiency, precision, and continuous protection. On the other hand, they introduce new forms of automated fraud and automated attacks that evolve faster than traditional defenses can react.

Organizations should not avoid AI, but we all need to use it wisely. By embedding strong governance, continuous monitoring, and ethical design, organizations can ensure their AI agents defend rather than deceive. The next generation of cybersecurity will not come from human analysts alone, but from a partnership between ethical AI agents and accountable humans. Ultimately, victory will belong to the side that automates smarter, not faster.