Infra & Security Architecture

APPLICATIVE SECURITY : Combining thes milestones 2 & 3

There are 2 potential solutions : One using the AzCopy Utility because the extraction of data from the on-premise server is totally free. But, if we want to automate the process, then the utility could be questionable. The second one is ADF (Azure Data Factory) which is not normally the appropriate solution because of the following reason : the extraction from ADF is never free. In our situation, the use of ADF won’t be expensive.

Azure Storage - Key Vault

Azure Key Vault is a cloud service for securely storing and accessing secrets. It contains :
1. 1 TOKEN for Azure Data Factory (ADF) : ADF accesses Key Vault via RBAC Permissions (Role-Based Access Control)
2. OR 1 SAS TOKEN for AzCopy Utility : PowerShell accesses Key Vault via a Service Principal

$SecureStringPwd = $sp.PasswordCredentials.SecretText | ConvertTo-SecureString -AsPlainText -Force
$pscredential = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList $sp.AppId, $SecureStringPwd
Connect-AzAccount -ServicePrincipal -Credential $pscredential -Tenant $tenantId

Retrieve Token from Azure Key Vault :
1. ADF via Key Vault Reader & Key Vault Secrets User permissions can retrieve the Token
2. PowerShell via the Service Pirncipal can retrieve the SAS Token
Access the Storage Account (Blob Storage) :
1. ADF using the Token can access the Blob Storage
2. AzCopy (triggered by PowerShell) using the SAS Token can access the Blob Storage without permissions (RBAC)
Retrieve the documents :
1. ADF via Integration Runtime with the synchronized authentication key can access the storage in the on-premise server
2. The connectivity between the on-premise servers is already set, no need to use a key

Azure Web Apps - SSO

Users can log into the Web App via SSO (Authentication Method allowing users to sign in using one set set of credentials to multiple software systems) - What is single sign-on? - Microsoft Entra ID | Microsoft Learn
Azure Web Apps via a Service Principal, can retrieve the Token from the Key Vault before using Azure AI Search
Azure Web Apps can access Azure AI Search via the Token

Azure AI Search - Key filter

Azure AI Search does not provide document-level permissions. As a workaround, we can create a filter that trims search results based on a string containing a group or user identity (Security filters for trimming results - Azure AI Search | Microsoft Learn)

POST https://[search service].search.windows.net/indexes/securedfiles/docs/index?api-version=2023-11-01
{
     "name": "securedfiles",  
     "fields": [
         {"name": "file_id", "type": "Edm.String", "key": true, "searchable": false },
         {"name": "file_name", "type": "Edm.String", "searchable": true },
         {"name": "file_description", "type": "Edm.String", "searchable": true },
         {"name": "group_ids", "type": "Collection(Edm.String)", "filterable": true, "retrievable": false }
     ]
 }
 //https://learn.microsoft.com/en-ca/azure/search/search-security-trimming-for-azure-search#create-security-field

In order to trim documents based on group_ids access, we should issue a search query with a filter.

{
   "filter":"group_ids/any(g:search.in(g, 'group_id1, group_id2'))"  
}

Azure AI Search is already connected to the Blob Storage via a connectionstring using an encrypted account key

DATA PROTECTION

Azure Storage

By default, all data stored in Azure Storage is automatically encrypted using service-side encryption (SSE) because data is persisted to the cloud. Moreoever, all metadata is also encrypted. And, there is no additional cost for that. However, we can manage our own encryption keys if we have specific security and compliance needs (Azure Storage encryption for data at rest | Microsoft Learn | Azure Data Encryption-at-Rest - Azure Security | Microsoft Learn).

Azure Web Apps

By default, all data stored in Azure is automatically encrypted without any additional configuration. If we have specific security or compliance needs, then we can add additional security and configure encryption at rest using customer-managed keys (Encrypt your application source at rest - Azure App Service | Microsoft Learn, Azure security baseline for App Service | Microsoft Learn).

Azure AI Search

By default, Azure AI Search automatically encrypts data at rest with service-managed keys. If we need more protection, we can supplement the dfault encryption with another layer of encryption using keys that we can create and manage in Azure Key Vault (Encrypt data using customer-managed keys - Azure AI Search | Microsoft Learn).

INFRA & SECURITY : Combining the milestones 2 & 3

Azure Web App : IP restriction and Azure VNET Integration

Access restrictions in App Service are equivalent to a firewall allowing you to block and filter traffic. Access restrictions apply to inbound access only (App Service Access restrictions - Azure App Service | Microsoft Learn). (The ability to restrict access to your web app from an Azure virtual network uses service endpoints. With service endpoints, you can restrict access to a multitenant service from selected subnets (Azure App Service access restrictions - Azure App Service | Microsoft Learn).
Virtual Network Integration : It gives your app access to resources in your virtual network, but it doesn't grant inbound private access to your app from the virtual network. Iy is used only to make outbound calls from your app into your virtual network. For inbound private access, we need to refer to private endpoints and so, we need to deploy the ap behind private endpoint (Connect privately to an App Service apps using private endpoint - Azure App Service | Microsoft Learn | Azure App Service access restrictions - Azure App Service | Microsoft Learn)

Virtual Network Peering would be added if we decide to create ressources deployed in other VNet : Azure Virtual Network peering | Microsoft Learn

connectivity between on-premise server and Azure VNet will be implemented via an ExpressRoute (private peering) or VPN

To prevent data exfiltration or the risk of malicious program implantation, we can control the outbound traffic with Azure Firewall. By default, App Service can make outbound request to the public Internet. As our App Service is integrated with Azure VNet, we can control the outbound traffic with Network Security Group to a limited extend (such as target IP address, Port and Protocol) : App Service outbound traffic control with Azure Firewall - Azure App Service | Microsoft Learn

Azure Storage

Require Secure Transfer during the creation of the ressource Storage Account. So, a call to an Azure Storage REST API operation must be made over HTTPS.
Configuration of Firewalls and Virtual Networks : There are 2 configurations to take account…
1. Configure access from on-premise networks : Configure Azure Storage firewalls and virtual networks | Microsoft Learn
  To allow access to your service resources, you must allow these public IP addresses in the firewall setting for resource IPs (Azure ExpressRoute, VPN).
2. Configure access from Other Azure Services / Azure Ressources : Configure Azure Storage firewalls and virtual networks | Microsoft Learn
  We can control access to Storage Account over network endpoints, from selected virtual network subnets using private endpoints and not using service endpoints. Virtual Network Service Endpoints are public and accessible via Internet and we don’t want that. On the other side, Virtual Network Private Endpoints use a private IP address to access the Storage Account over the Microsoft backbone network.

When you create a private endpoint, the DNS CNAME resource record for the storage account is updated to an alias in a subdomain with the prefix privatelink. By default, we also create a private DNS zone, corresponding to the privatelink subdomain, with the DNS A resource records for the private endpoints : Use private endpoints - Azure Storage | Microsoft Learn

Network & CyberSecurity : Combining the milestones 2 & 3

- Azure security baseline for Azure Cognitive Search | Microsoft Learn

Inbound connection - IP Firewall or Private endpoint (for Azure AI search - Create a Private Endpoint for a secure connection - Azure AI Search | Microsoft Learn)

Outbound connection - Connect through Firewall (for Azure AI Search - Configure an IP firewall - Azure AI Search | Microsoft Learn)

Configure Azure Storage Firewall (Configure Azure Storage firewalls and virtual networks | Microsoft Learn)

Azure Web apps : Connect privately to an App Service apps using private endpoint - Azure App Service | Microsoft Learn, with NAT GATEWAY (Azure NAT Gateway integration - Azure App Service - Azure App Service | Microsoft Learn), control outbound with App Service outbound traffic control with Azure Firewall - Azure App Service | Microsoft Learn; Integrate your app with an Azure virtual network - Azure App Service | Microsoft Learn, access restriction : App Service Access restrictions - Azure App Service | Microsoft Learn

Tutorial: Isolate back-end communication with Virtual Network integration - Azure App Service | Microsoft Learn

static IP restrictions : Azure App Service access restrictions - Azure App Service | Microsoft Learn

Control outbound traffic with NSG : App Service outbound traffic control with Azure Firewall - Azure App Service | Microsoft Learn si notre app est intégré à a VNET. On peut aussi le faire de façon centralisée à travers les abonnements Azure (Azure Firewall Standard features | Microsoft Learn)

Web Apps - App Service Environment networking - Azure App Service Environment | Microsoft Learn