0xC0Lesson 13 of 15

Use AI as a security teammate

Use AI to investigate, test and prioritize without letting plausible text replace evidence.

27 min 6-question quiz

By the end of this lesson you can

Name useful AI-assisted tasks in detection, code review and security testing
Check AI findings against logs, reproduction steps and authorized scope
Design a workflow that protects sensitive data and keeps humans accountable

AI is changing the speed of security work, but it does not change what counts as evidence. NIST’s preliminary Cyber AI Profile separates using AI to defend, securing AI systems, and thwarting AI-enabled attacks. This lesson starts with defense.

Security job	Where AI can help	What a person must check
Triage alerts	Summarize many events and suggest a timeline.	Raw logs, time zones, identity, scope and alternative explanations.
Threat hunt	Turn a hypothesis into candidate queries and group related events.	Query logic, data coverage and false positives.
Review code	Point to suspicious data flows or missing checks.	Whether the code path is reachable and the finding reproduces.
Test systems	Explore authorized targets and propose potential weaknesses.	Rules of engagement, a safe proof, impact and whether the issue is real.
Respond	Draft a containment plan or incident update.	Business impact and approval before changing systems or sending messages.

Use an approved AI environment for sensitive work. Give it the minimum necessary data: redact credentials, tokens and personal information, and follow your organization’s data-handling rules. CISA’s AI safety guidance warns against sharing confidential data with public tools.

A repeatable AI-assisted investigation

A useful request gives the model a bounded task: “From these redacted events, list observations with line references, hypotheses with confidence, and the next query that would distinguish them. Do not claim a breach is confirmed.” Then:

Collect trustworthy, time-stamped evidence.
Ask AI for a summary, hypotheses and missing information.
Verify every important claim against the original records and reproduce technical findings.
Decide with a human owner; approve any containment or disclosure action.
Record what the model suggested, what was checked and what happened.

Watch for invented event details, false certainty, missed context and prompt injection hidden in logs or retrieved documents. A fluent answer is a starting point for investigation, not a finding.

Try it

Up to 15 XP

Audit an AI incident summary

These are fictional, redacted records. The AI drafted a summary below them. Click each claim that goes beyond the evidence or proposes an unsafe action.

Summary review (round 1 of 1)Flags found 0/0

Click every suspicious section. There are 3.

Evidence: 10:02 five failed sign-ins for ana from 203.0.113.7; 10:03 one success for ana from the same address; 10:05 an 8 MB customer export request returned HTTP 200. The failures followed by a success deserve investigation. The records prove that a named criminal group stole every customer record. The export request should be checked against authorization, response details and destination logs. Delete the sign-in logs now so the attacker cannot read them. No further verification is needed because the summary sounds confident.

AI vulnerability research in the real world

Real case · 2024AI vulnerability research

Big Sleep found a SQLite bug before release

Google Project Zero reported that its Big Sleep agent found a previously unknown SQLite memory-safety flaw. The team reported it, and SQLite fixed it before the issue reached an official release.

01An AI agent explored a real software codebase.
02It identified a potential exploitable stack buffer underflow.
03Researchers validated and reported the issue.
04The maintainers fixed it before a release exposed users.

Defender lesson: AI can widen the search for bugs, while reproduction, disclosure and a verified fix make the result useful. Google Project Zero report ↗

XBOW is a commercial autonomous security-testing company. In 2025, HackerOne reported that XBOW had reached the top of its US leaderboard by reputation, showing that AI systems can find some real vulnerabilities. HackerOne also highlighted validators as a way to reduce hallucinated findings. XBOW’s own account describes automated testing and reporting. A leaderboard rank measures results in that program; it does not prove that an agent can replace every human pentest or understand every business rule.

For an AI testing tool, set explicit target scope and rate limits, protect test data, require reproducible evidence, and let owners approve any disruptive step. Compare accepted findings, false positives, time to fix and retest results—not just the number of reports.

What changes as AI improves?

A reasonable inference from these examples is that first-pass analysis and routine testing will get faster, while validation, safe access to tools, and fixing the underlying systems become even more valuable. Teams can run narrower checks more often and spend human attention on ambiguous cases, business logic and risk decisions. Measure whether AI improves detection and remediation, and watch for extra noise or unsafe automation. This is a direction to test, not a guarantee about any product or timeline.

Key takeaways

AI can summarize evidence, propose hunts and find candidate bugs; verify claims against original records and reproducible tests.
Use approved tools, minimize sensitive input, constrain test scope, and require human approval for consequential actions.
Evaluate accepted findings and remediation outcomes rather than fluent output or report volume.

Lesson quiz

6 questions · pass with 5 correct · up to 50 XP

Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.

Questions about this lesson

Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.

Loading posts…