Discover shadow AI and LLM applications across the organization with Defender for Cloud Apps. Assess AI app risk scores, configure session policies for AI tools, and monitor sensitive data shared with AI services.
This lab covers AI App Governance using Microsoft Defender for Cloud Apps. the practice of discovering, assessing, and controlling AI and large language model (LLM) applications across your organization. With the rapid adoption of generative AI, employees often use tools like ChatGPT, Google Gemini, Claude, and dozens of other AI services without IT approval. creating shadow AI risks. You will learn how to use Cloud Discovery to identify AI app usage, assess risk scores based on data handling and compliance posture, enforce session policies with Conditional Access App Control to block sensitive data uploads to AI services in real time, and build governance policies that balance productivity with data protection.
A technology consulting firm with 8,000 employees discovers that over 400 employees are using unauthorized AI tools including ChatGPT, Claude, and various AI coding assistants. Employees are pasting client code, financial projections, and legal documents into these tools. The security team needs to discover all AI app usage, assess risks, block unauthorized data sharing, and establish an approved AI app catalogue with proper governance controls.
In Defender for Cloud Apps, navigate to Cloud Discovery > Discovered apps. Filter by the Generative AI category to see all AI applications detected in network traffic. Review the risk score, user count, data volume, and compliance certifications for each AI app.
Review each AI application's data handling policies, training data usage terms, and compliance certifications. Tag apps as Sanctioned (approved for use), Unsanctioned (blocked), or Under Review. Create app policies to automatically block unsanctioned AI applications at the network level.
Deploy Conditional Access App Control to proxy AI application sessions and enforce real-time policies that monitor and block sensitive data uploads.
CA-AI-Session-ControlUse the session monitoring dashboard to observe live user sessions with AI applications and identify sensitive data being shared in real time.
// ---------------------------------------------------------------
// PURPOSE: Find all file uploads to AI and LLM services in the last 7 days.
// WHY: Employees may upload sensitive documents, source code, or
// customer data to AI tools without realising the data-loss risk.
// This query surfaces who is uploading, how much, and to which AI app.
// HOW TO USE: Run in Defender XDR > Hunting > Advanced hunting,
// or in Microsoft Sentinel if CloudAppEvents is ingested.
// OUTPUT COLUMNS:
// Application - the AI service name (ChatGPT, Claude, Gemini, etc.)
// UploadCount - total file uploads in the 7-day window
// UniqueUsers - distinct users uploading to that app
// TotalBytes - total data volume uploaded (in bytes)
// INTERPRETATION:
// High UniqueUsers with low UploadCount per user = normal exploration.
// One user with very high UploadCount = potential data exfiltration.
// Large TotalBytes to consumer AI apps = high-priority investigation.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(7d)
| where ActionType == "FileUploaded"
| where Application has_any ("ChatGPT", "Claude", "Gemini", "Midjourney", "Copilot")
| summarize UploadCount=count(), UniqueUsers=dcount(AccountId),
TotalBytes=sum(FileSize) by Application
| order by UploadCount descConfigure session policies that detect and block the paste of sensitive content into AI chat interfaces in real time.
Extend Microsoft Purview DLP policies to cover data shared with AI applications through Defender for Cloud Apps integration.
DLP-AI-Data-ProtectionCreate anomaly detection policies that alert when users transfer unusually large volumes of data to AI platforms.
Bulk-Data-Transfer-AI// ---------------------------------------------------------------
// PURPOSE: Detect users who are uploading unusually large volumes
// of data to AI services in short time windows (30 min).
// WHY: Bulk uploads to AI platforms are a strong indicator of either
// data exfiltration (intentional) or risky copy-paste habits
// (unintentional). Normal use is a few queries; 20+ uploads
// in 30 minutes suggests automated extraction or batch pasting.
// THRESHOLDS:
// UploadCount > 20 - more than 20 file uploads in a 30-min window
// TotalMB > 50 - more than 50 MB transferred in a 30-min window
// Adjust thresholds based on your organisation's baseline.
// OUTPUT COLUMNS:
// TimeGenerated - the 30-minute window when the activity occurred
// AccountId - the user performing the uploads (UPN or object ID)
// Application - which AI service received the data
// UploadCount - number of uploads in that window
// TotalMB - total data volume in megabytes
// ACTION: Investigate high-volume users; check if sensitive data
// (source code, financial docs, PII) was uploaded.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(24h)
| where ActionType == "FileUploaded"
| where Application has_any ("ChatGPT", "Claude", "Gemini", "Copilot")
| summarize UploadCount=count(), TotalMB=round(sum(FileSize)/1048576.0, 2)
by AccountId, Application, bin(TimeGenerated, 30m)
| where UploadCount > 20 or TotalMB > 50
| project TimeGenerated, AccountId, Application, UploadCount, TotalMB
| order by TotalMB descConnect AI app usage signals to Insider Risk Management to detect employees who may be using AI tools to exfiltrate intellectual property or prepare for departure.
Create a dashboard in Microsoft Sentinel that visualises AI app usage trends, policy violations, and risk metrics across the organisation.
// ===============================================================
// AI APP USAGE DASHBOARD - 4 tiles for a Sentinel workbook or
// Defender XDR custom detection dashboard.
// Run each query as a separate tile/visualisation.
// ===============================================================
// ---------------------------------------------------------------
// TILE 1: AI app usage summary by application (last 30 days).
// Shows which AI apps are most popular by session count and user count.
// Sessions = total interactions; Users = distinct people using the app.
// Use a bar chart: X=Application, Y=Users.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(30d)
| where Application has_any ("ChatGPT", "Claude", "Gemini", "Midjourney",
"Copilot", "Perplexity", "GitHub Copilot")
| summarize Sessions=dcount(SessionId), Users=dcount(AccountId) by Application
| order by Users desc
// ---------------------------------------------------------------
// TILE 2: Daily AI usage trend over 30 days.
// Shows adoption growth or decline. A sharp spike may indicate
// a viral AI tool spreading through a department.
// Use a line chart: X=TimeGenerated (daily), Y=DailyUsers.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(30d)
| where Application has_any ("ChatGPT", "Claude", "Gemini", "Midjourney")
| summarize DailyUsers=dcount(AccountId) by bin(TimeGenerated, 1d)
| order by TimeGenerated asc
// ---------------------------------------------------------------
// TILE 3: Policy violations by policy name and application.
// Shows which DLP/session policies are triggering most often.
// High violation counts may indicate overly broad policies (tune)
// or genuine data protection issues (investigate).
// Use a stacked bar chart: X=PolicyName, Y=Violations, stack by Application.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(30d)
| where ActionType == "PolicyViolation"
| where Application has_any ("ChatGPT", "Claude", "Gemini")
| summarize Violations=count() by PolicyName, Application
| order by Violations desc
// ---------------------------------------------------------------
// TILE 4: Top 10 departments using AI apps.
// Joins CloudAppEvents with IdentityInfo to resolve department names.
// Helps target AI training and governance to the heaviest-using teams.
// Use a horizontal bar chart: X=Users, Y=Department.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(30d)
| where Application has_any ("ChatGPT", "Claude", "Gemini")
| join kind=inner (IdentityInfo | project AccountUPN, Department) on $left.AccountId == $right.AccountUPN
| summarize Users=dcount(AccountId) by Department
| top 10 by UsersCreate governance policies that define acceptable use for sanctioned AI applications, including data classification, session duration, and acceptable use cases.
Detect when developers and users connect AI services via OAuth or API integrations that grant access to organisational data.
// ---------------------------------------------------------------
// PURPOSE: Monitor OAuth consent events for AI applications to
// detect when users or admins grant AI apps access to
// organisational data (email, files, calendar).
// WHY: A single OAuth consent to a malicious AI app can expose
// your entire tenant's data. This query surfaces all consent
// events for AI-related apps so you can review what permissions
// were granted and by whom.
// OUTPUT COLUMNS:
// TimeGenerated - when the consent was granted
// AccountId - which user clicked "Allow"
// Application - the AI app name
// ConsentType - "AllPrincipals" (admin consent) or "Principal" (user)
// Permissions - the scopes granted (e.g. Mail.Read, Files.ReadWrite.All)
// RED FLAGS: Admin consent to unrecognised AI apps; user consent
// granting Mail.ReadWrite or Files.ReadWrite.All to any AI app.
// ---------------------------------------------------------------
CloudAppEvents
| where TimeGenerated > ago(30d)
| where ActionType == "Consent to application"
| where Application has_any ("ChatGPT", "Claude", "Gemini", "OpenAI",
"Anthropic", "ai", "copilot")
| project TimeGenerated, AccountId, Application,
ConsentType=tostring(RawEventData["ConsentType"]),
Permissions=tostring(RawEventData["Permissions"])
| order by TimeGenerated descGenerate compliance reports documenting AI app usage, policy violations, and remediation actions for audit and regulatory purposes.
Establish a curated catalogue of approved AI tools with usage guidelines, data handling requirements, and acceptable use policies.
Establish a recurring governance process to review AI app risks, update policies, and adapt to new AI tools entering the market.
Synthesise all the controls implemented in this lab into a comprehensive AI governance framework that aligns with regulatory requirements and industry best practices.
// ---------------------------------------------------------------
// PURPOSE: Generate an executive-level AI governance summary showing
// total sessions, unique users, apps used, violations, and
// the overall violation rate across all AI platforms.
// WHY: Leadership needs a single-number view of AI risk. The violation
// rate (violations / total sessions * 100) tells you what percentage
// of AI interactions triggered a DLP or session policy - a key
// metric for tracking governance effectiveness over time.
// HOW IT WORKS:
// 1. ai_apps: calculates total sessions, unique users, unique apps
// 2. violations: calculates total policy violations and violating users
// 3. Join on a dummy key (d=1) to combine into one summary row
// OUTPUT COLUMNS:
// TotalSessions - total AI app interactions in 30 days
// UniqueUsers - distinct employees using AI apps
// UniqueApps - number of different AI apps in use
// TotalViolations - DLP/session policy violations triggered
// UsersViolating - distinct users who triggered at least one violation
// ViolationRate - percentage of sessions that violated policy
// INTERPRETATION: ViolationRate < 1% = policies working well.
// ViolationRate > 5% = policies may be too broad (tune) or users
// need more training on acceptable AI use.
// ---------------------------------------------------------------
let ai_apps = CloudAppEvents
| where TimeGenerated > ago(30d)
| where Application has_any ("ChatGPT", "Claude", "Gemini", "Midjourney", "Copilot")
| summarize TotalSessions=count(), UniqueUsers=dcount(AccountId),
UniqueApps=dcount(Application);
let violations = CloudAppEvents
| where TimeGenerated > ago(30d)
| where ActionType == "PolicyViolation"
| where Application has_any ("ChatGPT", "Claude", "Gemini")
| summarize TotalViolations=count(), UsersViolating=dcount(AccountId);
ai_apps | extend d=1
| join kind=inner (violations | extend d=1) on d
| project TotalSessions, UniqueUsers, UniqueApps, TotalViolations, UsersViolating,
ViolationRate=round(TotalViolations*100.0/TotalSessions, 2)| Resource | Description |
|---|---|
| Manage OAuth apps in Defender for Cloud Apps | Review and control third-party app permissions |
| Session policies | Configure real-time session controls for cloud apps |
| Working with discovered apps | Review and manage apps found by Cloud Discovery |
| Learn about data loss prevention | DLP policies for protecting sensitive data across locations |
| Configure Insider Risk Management | Set up insider risk policies and indicators |