We’ve been playing with Azure Stack for quite a while now and its pretty easy to forget you’re not using the real Azure. But there are a few key differences that you should always keep in mind when deploying your applications on to Azure Stack; with one of the biggest being around Network Security.
At one of my consulting engagement this year my team were unable to communicate from a claims aware azure web application via client browser to an on-premise, windows authenticated SOAP endpoint. To overcome the identity endpoint mismatch, Claims and Windows, we were using the Azure Application Proxy to perform the identity translation. Our problems primarily occur in communicating between the browser and the Azure Application Proxy. We found multiple potential solutions to solve this problem, but each one has a fatal flaw. We do not need to solve for every solution, rather we only need 1 to work.
Hide an iframe in the page that authenticates to the proxy by hitting a proxy endpoint and performing the redirect dance. Because the user must first log-in to the application, the iframe can reuse these credentials.
Process Flow Description:
- Iframe makes a request to the proxy endpoint (without authentication)
- Proxy returns a 302 redirect
- Iframe is redirected to AAD login page. Login cookies are submitted to AAD login because application requires authentication.
- Login successful returning token
- Sends token to proxy
- Proxy returns cookie that is valid for the proxy.
- Any future calls to the proxy can use the proxy cookie and make successful calls.
This solution works for the majority of cases except…
Fatal flaw: During step 3, if the user has multiple logins to Azure AD the user can not automatically be logged in because AAD returns an HTML to the hidden iframe asking which to use for login.
- Enable home realm discovery (Domain_Hint) for the Application Proxy
- When enabling domain hints, step 2 will return an updated redirect URL to include an extra parameter, ‘&domain_hint=fmi.com’. With this extra information in Step 3, the AAD login page can automatically determine which user to login as. Now the iframe can successfully login and the requests going forward will succeed.
- Blocker: this feature is not available yet for App Proxy.
- Use a Smart Link
- A smart link tells AAD login similar information as the domain hint. I’ve tried using the below smart link, but it does not automatically detect the user domain to use. e.g. https://myapps.microsoft.com/fmi.com/signin/Home%20Proxy%20Dev/000-000-000-000
- Blocker: The Smart link still requires me to choose which user to select.
Use ADAL.js to retrieve a bearer token for authentication to the Application Proxy endpoint.
- ADAL.js calls AcquireToken to requesting a bearer token for the Application Proxy Endpoint.
- AAD returns an authentication token.
This solution works for Internet Explorer but in any other browser it fails
Fatal Flaw: When making requests in step 3 with the authentication header, the browser sends a CORs preflight request. The proxy is not handling the OPTIONS request properly and is returning a 302.
- Enable CORs on the Application Proxy so that Preflight requests are handled gracefully.
- Blocker: this feature is not available yet for App Proxy.
We communicated these shortcomings of AAD Application Proxy to Microsoft and hope they would prioritize this feature in next release. Hope you would be able to customize your design keeping the above solutions and it's shortcomings in mind.
Network Security Groups are a simple yet powerful tool within the Azure networking toolkit. They are an Azure resources that provides the ability to apply an Access Control List (ACL) ruleset to allow or deny network traffic to a VM. NSGs can be associated with either a subnet within a Virtual Network, or to a VM’s NIC (ARM). Think of them as a lightweight firewall. There is already some excellent documentation available on what they are (https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-nsg/) and best practices on how to implement them (https://blogs.msdn.microsoft.com/igorpag/2016/05/14/azure-network-security-groups-nsg-best-practices-and-lessons-learned/) but for this post I wanted to focus on a particular part of the NSG rule set, the ‘tags’ functionality. Note, this is separate from the resource tags functionality in Resource Manager.
NSG Rule Tags: What are they?
Azure offers three ‘tags’ that can be used as a source or destination within a NSG rule. They are as follows:
All three of these tags are utilised in the Default Rules created with any new Network Security Group resource:
Inbound Default Rules
Outbound Default Rules
They also can be utilised for any custom rules you wish to add to a NSG Ruleset.
Recently Microsoft has released the Diagnostics feature for Network Security Groups. This feature allows us to see Events and Rule Counts on each of the rules within the NSG. It also, however, allows us to dig a bit deeper into what these tags are and how traffic is handled by the default rules. (For how to enable Diagnostics logs see here.)
Let’s start with the ‘AzureLoadBalancer’ tag. MS describes this tag as "denotes Azure’s Infrastructure load balancer. This will translate to an Azure datacenter IP where Azure’s health probes originate." Looking into the diagnostics logs we can see that this tag actually translates to the following CIDR block: 126.96.36.199/32.
This IP Address is a special IP Address within Azure and is the same for every Azure region. It maps to the physical IP address of the server machine (host node) hosting the virtual machine. Although they label as Load Balancer, it’s actually the same IP address that is used for the Azure Load Balancer health probe, DHCP, DNS and the virtual machine health probe. Any communication from this IP address should be considered as trusted and if blocked it will cause the Azure virtual machine to become unresponsive (i.e. never remove this rule).
Things get a little more interesting when we look at the ‘Internet’ tag. Microsoft’s documentation here says that this tag "denotes the IP address space that is outside the virtual network and reachable by public Internet. This range includes Azure owned public IP space as well."
When we look into the diagnostics logs, rules that use this tag have the following value for the IP Address array to match (Warning – long list):
188.8.131.52/8, 184.108.40.206/7, 220.127.116.11/6, 18.104.22.168/7, 22.214.171.124/8, 126.96.36.199/6, 188.8.131.52/5, 184.108.40.206/8, 220.127.116.11/7, 18.104.22.168/6, 22.214.171.124/3, 126.96.36.199/3, 188.8.131.52/6, 184.108.40.206/10, 220.127.116.11/9, 18.104.22.168/8, 22.214.171.124/7, 126.96.36.199/5, 188.8.131.52/5, 184.108.40.206/6, 220.127.116.11/7, 18.104.22.168/8, 22.214.171.124/3, 126.96.36.199/5, 188.8.131.52/8, 184.108.40.206/9, 220.127.116.11/10, 18.104.22.168/11, 22.214.171.124/12, 126.96.36.199/13, 188.8.131.52/14, 184.108.40.206/15, 220.127.116.11/16, 18.104.22.168/7, 22.214.171.124/12, 126.96.36.199/11, 188.8.131.52/10, 184.108.40.206/9, 220.127.116.11/8, 18.104.22.168/7, 22.214.171.124/4, 192.0.0.0/23, 126.96.36.199/24, 188.8.131.52/22, 184.108.40.206/21, 220.127.116.11/20, 18.104.22.168/19, 22.214.171.124/18, 126.96.36.199/17, 188.8.131.52/16, 184.108.40.206/15, 220.127.116.11/14, 18.104.22.168/13, 22.214.171.124/12, 126.96.36.199/11, 188.8.131.52/10, 184.108.40.206/11, 220.127.116.11/13, 18.104.22.168/16, 22.214.171.124/15, 126.96.36.199/14, 188.8.131.52/12, 184.108.40.206/10, 220.127.116.11/8, 18.104.22.168/7, 22.214.171.124/7, 126.96.36.199/12, 188.8.131.52/15, 184.108.40.206/14, 220.127.116.11/13, 18.104.22.168/11, 22.214.171.124/10, 126.96.36.199/9, 188.8.131.52/8, 184.108.40.206/5, 220.127.116.11/4
Now, if you are Rain Man you can see there are a few gaps in there but this is because these are reserved IP addresses (for reference the list of reserved IPv4 addresses is located here: http://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml). Thus this tag is exactly what is advertised: public internet addresses.
Note: As stated in the Microsoft summary, this list does include the Azure public IP addresses. Therefore if a rule exists with the Internet tag and Deny action, it can result in some unexpected behaviour of virtual machine extensions (https://blogs.msdn.microsoft.com/mast/2016/04/27/vm-stuck-in-updating-when-nsg-rule-restricts-outbound-internet-connectivity/) unless those Azure Public IP Addresses are allowed with a lower priority.
The 'VirtualNetwork' tag is the one that is a little misleading. Microsoft states that this tag "denotes all of your network address space. It includes the virtual network address space (CIDR ranges defined in Azure) as well as all connected on-premises address spaces and connected Azure VNets (local networks)." Now I’m not 100% sure, but I think it means that this tag should correspond to all of the routes the VNet’s gateway can see.
To prove this out, I created an NSG and attached it to a few different virtual networks. The first virtual network I attached it to was an isolated VNet with no VPN or Express Route connections (i.e. no VNet Gateway). When I retrieved the NSG diagnostic log, the list of addresses for the ‘Virtual Network’ tag was as follows: 10.1.0.0/16 and 18.104.22.168/32. The first CIDR block corresponds to the Address Space for the attached virtual network and the second is the same special IP Address as the ‘AzureLoadBalancer’ listed above.
The second virtual network I attached the NSG to was one that had a site to site connection to a VNet in a separate Azure Region. The list of addresses within the ‘Virtual Network’ tag and what they correspond to are as follows:
- 10.1.0.0/16 – Local Virtual Network Address Space
- 10.2.0.0/16 – Remote Virtual Network Address Space
- 10.2.0.0/24 – Remote Virtual Network first subnet (only subnet configured)
- 10.2.254.0/24 – Remote Virtual Network gateway subnet
- 22.214.171.124/32 – Special Azure VM Host IP Address
Now the ‘VirtualNetwork’ tag includes the complete address space of the combined Site to Site VPN (plus the subnets of the remote VNet curiously).
The third and final virtual network was one that had an Express Route circuit back to an on premise datacenter. To make things more interesting, this Express Route circuit also had a public peering connection to Azure (if you’re unsure what Peering connections are see here: https://azure.microsoft.com/en-us/documentation/articles/expressroute-circuit-peerings/) which means it also contained routes to Azure Public IPs. When I pulled the NSG log, the list of addresses to match contained a list of 400+ CIDR blocks.
On closer analysis, the list of subnets within this NSG log seemed to be much broader than just the virtual network, on premise and Azure public IPs. It turns out that although it’s broken down into 400+ subnets, it actually contains nearly every IPv4 address from 0.0.0.0 to 255.255.255.255. This is likely because the Gateway of the Virtual Network its attached to was a member of a BGP domain (as it’s an Express Route Gateway) which included advertised routes to on premise private and public subnets, Azure public subnets as well as public internet routes.
Overall, Network Security Groups are a great tool within the Azure security stack with the Rule Tags functionality allowing some extra capabilities for restricting traffic to and from your Virtual Networks and VMs, albeit with some considerations:
- The ‘AzureLoadBalancer’ tag is actually the Azure VM Host IP Address, and should always be permitted.
- The ‘Internet’ tag corresponds to the public IP address space, and should be allowed or denied accordingly.
- The ‘VirtualNetwork’ tag address space depends entirely on the VPN/Express Route configuration of your virtual network. The rules using this tag within a Network Security Group might be allowing a lot more traffic than first thought. If you want to make a rule to just allow or deny your virtual network’s address space, the best way would be to define that address space as a CIDR Block rather than use the ‘VirtualNetwork’ tag. If you want to have a rule that restricts traffic to your 'known' network then this is a good candidate.