Automate Risk Remediation with Conditional Access

📋 Overview

About This Lab

Risk-based Conditional Access automation extends Entra ID Protection by translating real-time risk signals into enforcement actions. Instead of manually investigating every risky sign-in, automated policies block high-risk access, force MFA for medium-risk, and require password changes for compromised users. reducing mean time to respond from hours to seconds.

🏢 Enterprise Use Case

A global enterprise with 25,000 users across 40 countries detects 200+ risky sign-ins daily. Manual investigation takes 30 minutes per event. The security team needs automated risk-based policies that respond instantly: blocking impossible-travel sign-ins, forcing step-up MFA for anonymous IP usage, and triggering self-service password reset for users with leaked credentials. all without SOC analyst intervention.

🎯 What You Will Learn

Review risk detection types and severity levels
Create sign-in risk Conditional Access policies with layered controls
Create user risk policies for compromised credential remediation
Configure MFA registration policy for new users
Set up Named Locations to reduce false positives
Configure risk-based sign-in frequency controls
Create exception policies for break-glass accounts
Test risk policies with simulated scenarios
Monitor policy effectiveness with sign-in analytics
Build KQL queries for identity risk dashboards

🔑 Why This Matters

Identity is the primary attack vector for 80% of breaches. Automated risk-based policies transform your identity security from reactive to proactive. stopping compromised accounts in real time without requiring human intervention for every alert.

✅ Prerequisites

Entra ID P2 license - required for Identity Protection risk detections and risk-based Conditional Access policies
Global Administrator or Security Administrator role for creating Conditional Access policies
Conditional Access Administrator role (least-privilege alternative for CA policy management)
Self-Service Password Reset (SSPR) enabled for all users - required for user-risk “require password change” remediation
Microsoft Graph PowerShell SDK installed with Policy.ReadWrite.ConditionalAccess and Policy.Read.All scopes
Break-glass accounts created and documented - at least 2 cloud-only Global Admin accounts excluded from all CA policies
MFA registration - users must have at least one MFA method registered before risk-based policies can remediate via MFA
Defender XDR Advanced Hunting access for KQL queries (Security Reader role minimum)

💡 Pro Tip: Always deploy new Conditional Access policies in Report-only mode first. Monitor the Conditional Access insights workbook for 1–2 weeks to validate impact before switching to On. This prevents accidentally locking out users or blocking legitimate workflows.

Step 1 · Review Risk Detection Types

Entra ID Protection uses Microsoft's global identity intelligence to calculate sign-in risk (real-time) and user risk (offline) scores. Understanding each detection type and its risk level is essential before creating policies, because your Conditional Access rules must map grant controls to the right severity - blocking high-confidence compromises while allowing self-remediation for lower-confidence signals.

Navigate to Entra ID > Protection > Identity Protection > Risk detections
Review real-time detection types: Anonymous IP address, Atypical travel, Malware-linked IP, Unfamiliar sign-in properties, Token issuer anomaly
Review offline detection types: Leaked credentials, Password spray, Threat intelligence, Suspicious browser
Understand risk levels: Low (informational - unusual but not necessarily malicious), Medium (suspicious patterns warranting step-up auth), High (strong compromise indicators requiring block or forced remediation)
Navigate to Risky sign-ins report - filter by risk level and state to see which detections have been remediated vs. still active
Navigate to Risky users report - review users with confirmed compromised or at-risk status and their risk history

💡 Pro Tip: Real-time detections evaluate risk during the sign-in and can trigger CA policies immediately. Offline detections (like leaked credentials from dark web monitoring) evaluate risk after the sign-in and update the user risk level. Design your policies to handle both: sign-in risk policies for real-time response, user risk policies for offline detections.

Step 2 · Create Sign-in Risk Conditional Access Policy

Create layered Conditional Access policies that respond differently based on sign-in risk level. The layering principle is critical: high-risk sign-ins get blocked entirely (the confidence of compromise is high enough to justify denial), while medium-risk sign-ins require step-up authentication (the suspicious indicators warrant verification but not outright denial). This defense-in-depth approach maximises security without unnecessarily disrupting legitimate users.

Navigate to Entra ID > Protection > Conditional Access > + New policy
Name: Block High-Risk Sign-ins
Users: All users (exclude break-glass accounts)
Conditions > Sign-in risk: select High
Grant: Block access
Create a second policy: Require MFA for Medium-Risk Sign-ins
Conditions > Sign-in risk: select Medium
Grant: Require multifactor authentication

# WHAT: Create a Conditional Access policy to block high-risk sign-ins via Graph PowerShell
# WHY: Automatically prevents access when Entra ID Protection detects high-confidence
#      compromise indicators (anonymous IP, malware-linked IP, credential stuffing)
# SCOPES REQUIRED: Policy.ReadWrite.ConditionalAccess
$params = @{
    DisplayName = "Block High-Risk Sign-ins"
    # Report-only mode logs policy matches without enforcing - use for 1-2 week validation
    State = "enabledForReportingButNotEnforced"
    Conditions = @{
        Users = @{ IncludeUsers = @("All"); ExcludeGroups = @("BreakGlassGroup-Id") }
        # SignInRiskLevels values: low | medium | high
        # high = strong indicators like anonymous IP + impossible travel combined
        SignInRiskLevels = @("high")
    }
    GrantControls = @{
        Operator = "OR"
        # "block" = deny access entirely; alternatives: "mfa", "compliantDevice", "passwordChange"
        BuiltInControls = @("block")
    }
}
New-MgIdentityConditionalAccessPolicy -BodyParameter $params

# WHAT: Create a companion CA policy requiring MFA for medium-risk sign-ins
# WHY: Medium-risk sign-ins indicate suspicious but not conclusive compromise.
#      Requiring MFA lets legitimate users prove their identity while blocking
#      attackers who only have stolen credentials (and not the second factor).
# SCOPES REQUIRED: Policy.ReadWrite.ConditionalAccess
$paramsMedium = @{
    DisplayName = "Require MFA for Medium-Risk Sign-ins"
    State = "enabledForReportingButNotEnforced"
    Conditions = @{
        Users = @{ IncludeUsers = @("All"); ExcludeGroups = @("BreakGlassGroup-Id") }
        SignInRiskLevels = @("medium")
        # Optionally scope to specific apps or platforms:
        # Applications = @{ IncludeApplications = @("All") }
    }
    GrantControls = @{
        Operator = "OR"
        BuiltInControls = @("mfa")
    }
    # Session controls: force sign-in frequency to "every time" so risk
    # is re-evaluated at each authentication attempt
    SessionControls = @{
        SignInFrequency = @{
            Value = $null
            Type = $null
            IsEnabled = $true
            AuthenticationType = "primaryAndSecondaryAuthentication"
            FrequencyInterval = "everyTime"
        }
    }
}
New-MgIdentityConditionalAccessPolicy -BodyParameter $paramsMedium

// WHAT: Identify sign-ins that bypassed all Conditional Access policies (coverage gaps)
// WHY: If sign-ins are not evaluated by any CA policy, those users/apps are
//      unprotected by risk-based controls. This query finds the gaps so you
//      can extend policy scope.
// TABLE: AADSignInEventsBeta - Entra ID sign-in events
// KEY FIELD: ConditionalAccessPolicies - array of CA policies that evaluated the sign-in
//   Empty or all "notApplied" = the sign-in was not covered by any policy
// OUTPUT: Applications and user counts with no CA policy coverage
AADSignInEventsBeta
| where Timestamp > ago(7d)
| where ErrorCode == 0  // successful sign-ins only
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| summarize Policies = make_set(CAPolicy.result) by Application, AccountUpn
| where set_has_element(Policies, "notApplied") and array_length(Policies) == 1
| summarize UncoveredUsers = dcount(AccountUpn) by Application
| order by UncoveredUsers desc
| take 20

💡 Pro Tip: Never create a single CA policy that covers both medium and high risk. If you combine them, you can only choose one grant control. With separate policies, high-risk gets Block while medium-risk gets Require MFA - the layered response is what makes risk-based CA effective.

Step 3 · Create User Risk Remediation Policy

User risk differs from sign-in risk: it represents an offline assessment that a user’s credentials have been compromised (e.g., credentials found on the dark web). User risk policies remediate the account itself by forcing a password change, ensuring the attacker’s stolen credentials become invalid. The user must first prove their identity via MFA before setting a new password.

Create a new Conditional Access policy: Force Password Change for Compromised Users
Conditions > User risk: select High
Grant: Require password change and Require multifactor authentication (both required - operator = AND)
This forces users with leaked or compromised credentials to verify via MFA, then reset their password
Prerequisite: Self-service password reset (SSPR) must be enabled for all users
After successful password change, Entra ID Protection automatically dismisses the user risk

# WHAT: Create a Conditional Access policy for user risk remediation
# WHY: When Entra ID Protection detects leaked credentials (offline detection),
#      user risk is elevated to High. This policy forces the user to prove their
#      identity via MFA and then change their password, invalidating stolen creds.
# IMPORTANT: Grant operator is "AND" - user must complete BOTH MFA and password change.
#   If set to "OR", the user could change password without MFA (defeating the purpose).
# PREREQUISITE: SSPR must be enabled or "Require password change" will fail.
$paramsUserRisk = @{
    DisplayName = "Force Password Change for Compromised Users"
    State = "enabledForReportingButNotEnforced"
    Conditions = @{
        Users = @{ IncludeUsers = @("All"); ExcludeGroups = @("BreakGlassGroup-Id") }
        # UserRiskLevels - offline risk: leaked credentials, threat intelligence
        UserRiskLevels = @("high")
    }
    GrantControls = @{
        # AND operator = user must satisfy ALL controls (MFA + password change)
        Operator = "AND"
        BuiltInControls = @("mfa", "passwordChange")
    }
}
New-MgIdentityConditionalAccessPolicy -BodyParameter $paramsUserRisk

💡 Pro Tip: User risk auto-remediation only works if users have MFA and SSPR registered before the compromise is detected. Deploy the MFA Registration Policy (Step 4) first. Otherwise, compromised users without MFA will be unable to self-remediate and will need admin intervention.

Step 4 · Configure MFA Registration Policy

Navigate to Identity Protection > MFA registration policy
Set Users: All users (exclude service accounts)
Set Policy enforcement: Enabled
This ensures all users register for MFA before risk policies require it as remediation

Step 5 · Configure Named Locations

Named Locations define trusted network boundaries (office IPs, VPN exit points) and geographic restrictions. When a user signs in from a trusted location, Entra ID Protection reduces the likelihood of false-positive risk detections like “atypical travel.” Conversely, country-based restrictions can block access from geographies where your organisation has no employees.

Navigate to Conditional Access > Named locations > + IP ranges location
Create trusted locations for your corporate offices and VPN exit points (e.g., HQ-Seattle-Office)
Mark these as Mark as trusted location - this tells Identity Protection to weight sign-ins from these IPs as lower risk
Create a Countries location > name: Blocked-Countries > select countries where your organisation has no presence
Create a CA policy: Block Access from Restricted Countries > Conditions > Locations > Include: Blocked-Countries > Grant: Block
Reference named locations in risk-based policies for refined controls (e.g., skip MFA for trusted office IPs at low risk)

# WHAT: Create a Named Location for corporate office IP ranges via Graph PowerShell
# WHY: Marking corporate IPs as trusted reduces false-positive risk detections
#      (e.g., "atypical travel" when employees sign in from known offices).
#      Named locations can also be referenced in CA policy conditions.
# SCOPES REQUIRED: Policy.ReadWrite.ConditionalAccess
$locationParams = @{
    "@odata.type" = "#microsoft.graph.ipNamedLocation"
    DisplayName  = "HQ-Seattle-Office"
    IsTrusted    = $true
    # Define CIDR ranges for all office egress IPs
    # Include both IPv4 and IPv6 if applicable
    IpRanges     = @(
        @{ "@odata.type" = "#microsoft.graph.iPv4CidrRange"; CidrAddress = "203.0.113.0/24" }
        @{ "@odata.type" = "#microsoft.graph.iPv4CidrRange"; CidrAddress = "198.51.100.0/24" }
    )
}
New-MgIdentityConditionalAccessNamedLocation -BodyParameter $locationParams

💡 Pro Tip: Do not over-trust named locations. Marking a location as “trusted” reduces risk scores but should never be used to skip MFA entirely. Attackers can compromise VPN credentials and sign in from your “trusted” IP ranges. Use trusted locations to reduce friction, not eliminate security.

Step 6 · Set Up Sign-in Frequency Controls

Create a CA policy: Enforce Re-authentication After Risk
Target users with elevated risk where sign-in risk was previously medium
Session controls > Sign-in frequency: Every time
This forces re-evaluation of risk at each authentication attempt

Step 7 · Create Break-Glass Account Exceptions

Break-glass accounts are emergency access accounts that bypass all Conditional Access policies. If a misconfigured CA policy locks out all administrators, these accounts are your only way back in. They must be cloud-only (no on-premises sync), excluded from every CA policy, and monitored for any usage - because any sign-in to a break-glass account is either an emergency or a compromise.

Create an Entra ID security group: BreakGlass-Accounts
Create two cloud-only Global Administrator accounts with long, random passwords (50+ characters)
Add both accounts to the BreakGlass-Accounts group
Exclude this group from ALL Conditional Access policies (risk-based, MFA, device compliance, location-based)
Create an Azure Monitor alert rule to notify on any sign-in by break-glass accounts
Store break-glass credentials in a physical safe with dual-control access (two people required to open)
Test break-glass account sign-in quarterly to ensure it works, and rotate credentials after each test

// WHAT: Monitor break-glass account usage in Defender XDR Advanced Hunting
// WHY: Any sign-in to a break-glass account is either a legitimate emergency
//      or a security incident. Either way, it requires immediate investigation.
//      This query alerts on ALL sign-in attempts (successful or failed).
// TABLE: AADSignInEventsBeta
// HOW: Update the UPN list with your actual break-glass account UPNs
// OUTPUT: All sign-in attempts by break-glass accounts with full context
let breakGlassAccounts = dynamic(["breakglass1@contoso.onmicrosoft.com", "breakglass2@contoso.onmicrosoft.com"]);
AADSignInEventsBeta
| where Timestamp > ago(24h)
| where AccountUpn in~ (breakGlassAccounts)
| project Timestamp, AccountUpn, IPAddress, City, Country,
         ErrorCode, RiskLevelDuringSignIn, Application, ClientAppUsed
| order by Timestamp desc

💡 Pro Tip: Microsoft recommends at least two break-glass accounts. One should use a FIDO2 key (stored in a safe), and the other should use a very long password. Do not register phone-based MFA on break-glass accounts - a carrier SIM-swap attack could compromise them.

Step 8 · Test Risk Policies with Simulations

Validate your policies using controlled risk simulations.

Use the Tor browser to simulate an Anonymous IP address detection
Sign in from a VPN in a different country to simulate Atypical travel
Verify the appropriate CA policy fires: MFA for medium, block for high
Check the Sign-in logs to confirm the Conditional Access result matches expectations
Review Report-only mode results before switching to enforced mode

Step 9 · Monitor Policy Effectiveness

After deploying risk-based policies, you must continuously monitor their effectiveness. Are policies blocking the right sign-ins? Is the MFA success rate high (indicating legitimate users can self-remediate)? Are there false positives causing unnecessary friction? Use the Identity Protection dashboard and KQL queries to answer these questions and fine-tune your policies.

Navigate to Identity Protection > Overview dashboard - review trends for risky users, risky sign-ins, and risk detections over time
Check Conditional Access > Insights and reporting for policy impact analysis (how many sign-ins were blocked, required MFA, or allowed)
Review Report-only mode results: compare “would have been blocked” vs. “would have required MFA” counts
Export sign-in logs for detailed analysis of policy matches and blocks
Identify false positives (legitimate users blocked by high-risk policy) and tune Named Locations or exclusions

// WHAT: Measure Conditional Access policy effectiveness - block rate and MFA success rate
// WHY: Validates that risk-based policies are working as intended:
//   - Block rate: % of high-risk sign-ins actually blocked (should be ~100%)
//   - MFA success rate: % of medium-risk users who successfully complete MFA
//     (high MFA success = legitimate users can self-remediate; low = too much friction)
// TABLE: AADSignInEventsBeta
// KEY FIELDS:
//   ConditionalAccessStatus: success | failure | notApplied
//     success = CA policy granted access (user met controls like MFA)
//     failure = CA policy blocked access
//     notApplied = no CA policy matched this sign-in
//   AuthenticationRequirement: singleFactorAuthentication | multiFactorAuthentication
// OUTPUT: Daily policy match count, block count, MFA challenge count, and success rates
AADSignInEventsBeta
| where Timestamp > ago(30d)
| where RiskLevelDuringSignIn in ("medium", "high")
| extend PolicyResult = case(
    ConditionalAccessStatus == "failure", "Blocked",
    ConditionalAccessStatus == "success" and AuthenticationRequirement == "multiFactorAuthentication", "MFA-Passed",
    ConditionalAccessStatus == "success", "Allowed",
    "NotEvaluated")
| summarize Count = count() by PolicyResult, bin(Timestamp, 1d)
| order by Timestamp desc

// WHAT: Identify false positives - legitimate users repeatedly blocked by high-risk policy
// WHY: Excessive blocking of known, legitimate users indicates overly aggressive risk
//      detection or missing trusted location configuration. These users should be reviewed
//      and their locations potentially added to Named Locations.
// OUTPUT: Users blocked 3+ times with location details for false positive investigation
AADSignInEventsBeta
| where Timestamp > ago(14d)
| where RiskLevelDuringSignIn == "high"
| where ConditionalAccessStatus == "failure"
| summarize BlockCount = count(), Locations = make_set(City), IPs = make_set(IPAddress)
    by AccountUpn, AccountObjectId
| where BlockCount >= 3
| order by BlockCount desc

💡 Pro Tip: Schedule a weekly “CA policy health” review. Check the Insights and reporting workbook for policies with zero matches (may be misconfigured) and policies with excessive blocks (may need tuning). A well-tuned CA environment should have a MFA success rate above 95% for medium-risk sign-ins.

Step 10 · Build Identity Risk KQL Dashboards

Advanced Hunting KQL queries let you go beyond the built-in dashboards to create custom views of identity risk. These queries can be used in Defender XDR, pinned to Azure dashboards, or scheduled as custom detection rules for automated alerting. The goal is to build a continuous feedback loop: detect risk → enforce policy → measure effectiveness → refine.

// WHAT: Trend of risky sign-ins over the past 30 days in Defender XDR Advanced Hunting
// WHY: Tracks daily volume of medium/high-risk sign-ins to measure policy effectiveness
// TABLE: AADSignInEventsBeta - contains Entra ID sign-in logs with risk fields
// KEY FIELDS:
//   RiskLevelDuringSignIn - risk assessed in real-time: none | low | medium | high
//   AccountUpn - the user principal name of the signing-in user
// OUTPUT: Daily count of risky sign-ins and unique affected users by risk level
AADSignInEventsBeta
| where Timestamp > ago(30d)
| where RiskLevelDuringSignIn in ("medium", "high")
| summarize RiskySignIns = count(), UniqueUsers = dcount(AccountUpn) by RiskLevelDuringSignIn, bin(Timestamp, 1d)
| order by Timestamp desc

// WHAT: Identify the top 20 users with the most risk detections
// WHY: Pinpoints repeat targets or compromised accounts needing immediate attention
// HOW: Joins IdentityInfo (HR/directory data) with sign-in risk data
//   IdentityInfo - user directory attributes (department, job title, etc.)
//   AADSignInEventsBeta - sign-in events with real-time risk assessment
//   AccountObjectId - Entra ID object ID used to correlate across tables
// OUTPUT: User name, UPN, department, and total risk detection count
IdentityInfo
| join kind=inner (
    AADSignInEventsBeta
    | where RiskLevelDuringSignIn in ("medium", "high")
    | summarize RiskCount = count() by AccountObjectId
) on $left.AccountObjectId == $right.AccountObjectId
| project AccountDisplayName, AccountUpn, Department, RiskCount
| top 20 by RiskCount

// WHAT: Conditional Access policy summary - which policies fire most and their outcomes
// WHY: Provides a dashboard-ready view of policy effectiveness. Identifies:
//   - Policies that match frequently (most impactful)
//   - Policies that never match (may be misconfigured or redundant)
//   - Block vs. grant ratios per policy (tuning indicator)
// TABLE: AADSignInEventsBeta
// HOW: Parses the ConditionalAccessPolicies JSON array embedded in each sign-in event
// OUTPUT: Per-policy match count, block count, and grant count over 30 days
AADSignInEventsBeta
| where Timestamp > ago(30d)
| mv-expand CAPolicy = parse_json(ConditionalAccessPolicies)
| where CAPolicy.result in ("success", "failure")
| extend PolicyName = tostring(CAPolicy.displayName),
         PolicyResult = tostring(CAPolicy.result)
| summarize Matches = count(),
            Blocked = countif(PolicyResult == "failure"),
            Granted = countif(PolicyResult == "success")
    by PolicyName
| order by Matches desc

💡 Pro Tip: Save these KQL queries as custom detection rules in Defender XDR for automated alerting. For example, schedule the “top risky users” query to run daily and auto-create incidents for users who exceed 10 risk detections per week - turning passive monitoring into active threat response.

Summary

What You Accomplished

Created layered sign-in risk Conditional Access policies (block high, MFA medium)
Configured user risk remediation with forced password change
Set up MFA registration policy and named locations
Created break-glass account exceptions
Tested policies with simulated risk scenarios
Built KQL dashboards for identity risk monitoring

Next Steps

Next Lab: Investigate Identity Attacks in Defender XDR
Integrate identity risk alerts with Microsoft Sentinel for SOAR automation
Deploy Passwordless authentication to reduce credential-based attacks

📚 Documentation Resources

Resource	Description
Identity Protection Overview	Core concepts for risk detection and automated remediation
Risk-based Conditional Access	Step-by-step guide for sign-in risk and user risk policies
Report-only Mode	Test CA policies without enforcement before enabling
Resilient Access Controls	Break-glass account design and emergency access planning
Graph API: CA Policies	Programmatic CA policy creation and management via Microsoft Graph
AADSignInEventsBeta Table	Schema reference for identity sign-in data in Advanced Hunting