Example Playbook to secure your Azure AI Services

AI and AI security are currently hot topics, and I’d like to share some key insights—particularly around securing Azure AI solutions.
Securing Azure AI Services involves multiple layers and domains, each contributing to a robust defense perimeter.
In this blog post, I’ll present practical examples you can implement to harden your Azure AI environment.

The playbook is organized into the following categories:

1. Identity and Access
2. Network Security
3. Data Protection
4. Key and Secret Rotation
5. Monitoring and Threat Detection
6. Governance and Support boundaries
7. Reference Pattern
   

Identity and Access

Goal	What to do	Source and more information
Eliminate shared secrets	Disable local (key-based) auth and require Microsoft Entra ID (Azure AD) + RBAC	(Microsoft Learn, Microsoft Learn)
Avoid credential crawl	Assign a managed identity to the AI resource and grant it only the roles it needs (Reader / Search Index Data Reader, etc.)	Portal → Identity » System-assigned » On → Save, then add role assignment in Key Vault or Search (Microsoft Learn, Microsoft Learn)
Fine-grained entry conditions	Use Conditional Access (e.g. MFA, trusted device, Geo-fencing) on the Microsoft Entra enterprise app that backs the service	Policy in Entra ID → Conditional Access → “Azure AI Services” target (Microsoft Learn)
Delegate least privilege	Back-end code that calls Azure OpenAI from an Azure Function? Give the Function’s managed identity only Cognitive Services OpenAI User role on the resource—not Contributor.

Network Security

Private Link & VNet isolation

• Create a private endpoint for the Cognitive Services or Azure OpenAI account and switch Public network access to Disabled.
• DNS: use the auto-generated privatelink.* zone or your own Private DNS zone.
• Example (bicep):

resource ai 'Microsoft.CognitiveServices/accounts@2024-10-01' = {
  name: 'chat-ai-prod'
  location: location
  sku: { name: 'S0' }
  kind: 'OpenAI'
  properties: {
    publicNetworkAccess: 'Disabled'
    networkAcls: {
      defaultAction: 'Deny'
    }
  }
}


resource pe 'Microsoft.Network/privateEndpoints@2023-11-01' = {
  name: 'chat-ai-pe'
  location: location
  properties: {
    subnet: { id: vnet::subnets.app.id }
    privateLinkServiceConnections: [
      {
        name: 'openai-conn'
        properties: {
          privateLinkServiceId: ai.id
          groupIds: [ 'account' ]
        }
      }
    ]
  }
}

Configure Firewall for DLP function

Here is a Bash code to configure the firewall for DLP:

az rest -m PATCH -u \
"/subscriptions/$SUB_ID/resourceGroups/$RG/providers/Microsoft.CognitiveServices/accounts/chat-ai-prod?api-version=2024-10-01" \
-b '{ 
  "properties": {
    "restrictOutboundNetworkAccess": true,
    "allowedFqdnList": ["contoso.blob.core.windows.net"]
  } 
}'

It is also a good practice to create a Shared-private-link for dependent services (e.g., Azure AI Search from Azure OpenAI) to keep traffic on the backbone.

Data Protection

Layer	Action	Source
In-transit	TLS 1.2 is enforced automatically – nothing to configure here.	(Microsoft Learn)
At-rest (default)	Platform-managed keys are always on.	(Microsoft Learn)
Customer-managed keys (CMK)	Store an RSA-2048/3072/4096 or AES-256 key in Key Vault (Soft-delete + Purge-protection ON). Requires the resource’s managed identity to have wrapKey, unwrapKey, get on the key.	(Microsoft Learn)
Bring-your-own-storage (BYOS)	For Speech / Custom Voice data, point to a customer Storage account instead of Microsoft-managed one.	(Microsoft Learn)

Key and Secret Rotation

Service keys – each resource gets 2; script in Bash for monthly rotation:

az cognitiveservices account keys renew \
   -n chat-ai-prod -g rg-app --key-name key1

CMK rotation – set a Key Vault rotation policy; after Key Vault creates a new version, patch the AI resource’s keyVersion property. For more information, check Microsoft Learn.

Automate both with an Azure Function or Logic App triggered on Key Vault events.

Monitoring and Threat Protection

• Make sure, that Diagnostic Settings are enabled and logs are forwarded to Microsoft Sentinel
• Azure Policy & Defender for Cloud – enable built-in policies such as “Cognitive Services should use Private Link” or “Disable local authentication”. Non-compliant resources to take an effect in Defender for Cloud’s Secure Score.
• Example Bash script for Diagnostic setting creation:

az monitor diagnostic-settings create \
  --name send-to-logs \
  --resource /subscriptions/$SUB_ID/resourceGroups/$RG/providers/Microsoft.CognitiveServices/accounts/chat-ai-prod \
  --workspace $LOGWS \
  --logs '[{"categoryGroup":"allLogs","enabled":true}]' \
  --metrics '[{"category":"AllMetrics","enabled":true}]'

Governance and Support Boundaries

• Customer Lockbox – require an approval workflow before Microsoft support engineers can touch your data. For more information, visit Microsoft Learn.
• Microsoft Cloud Security Benchmark – the security baseline for Azure AI Services is published and updated quarterly; map each control (NS-1, IM-3, DP-5, …) to an internal control in your organisation’s catalogue. For more information, visit Microsoft Learn.

Reference Pattern

Scenario: Internal chatbot that uses Azure OpenAI + Azure AI Search on corporate PDFs.

1. Deploy both resources in the same region.
2. Give the Function-app’s system-assigned identity the Cognitive Services OpenAI User role on the OpenAI account and Search Index Data Reader on Search.
3. Create two private endpoints, one per resource, in the Bot VNet; disable public access on both.
4. Enable DLP on the OpenAI account – only allow outbound traffic to Search (<search-name>.search.windows.net).
5. Encrypt both resources with the same CMK preferably in a  Azure Key Vault Managed HSM.
6. Turn on diagnostic settings to a dedicated Log Analytics workspace; Sentinel rule alerts on >100 chat completions/minute from a single IP.
7. Rotate the CMK yearly and the OpenAI keys monthly (even though they’re disabled, you keep them ready for DR).

Key Takeaways

1. Identity beats keys – switch to Entra ID and managed identities on day 0.
2. Private Link everywhere – deny the public internet unless you truly need it.
3. Encrypt with something you own – CMK + HSM where compliance demands it.
4. Log first, then build – you can’t protect what you can’t see.
5. Automate compliance – let Azure Policy break the build, not prod.

Share this post: