Creating Highly Available CDP and AIA Locations with Azure, Part 1

Hello everyone, this is the first blog post in the “Creating Highly Available CDP and AIA Locations with Azure” series.

Posts in this series:

• Part 1 (this post)
• Part 2
• Part 3
• Part 4

Foreword

With the fast TLS expansion over the last decade, PKIs and their administrators face new challenges in order to provide PKI services reliably. Every TLS connection involves a certificate validation process, which consists of two main parts:

• Chain building and chain validation
• Revocation checking

Chain building and revocation checking rely on external services CA server has to provide: CA certificate and CRL distribution so clients can execute these steps. Locations to these files are embedded in digital certificates (in Authority Information Access, or AIA and in CRL Distribution Points, or CDP certificate extensions) and clients use these URLs to fetch the required files. Certificate validation most likely will fail if any of these locations are not available. This adds more strict high availability (HA) requirements to PKI infrastructure. In practice, most of the certificate chain building and validation process time is spent on network operations — CA certificate and CRL retrieval. This means that in addition high availability requirements, PKI should provide lower latencies to speed up the certificate validation process.

In this blog series, I will provide guidance on how to provide high availability and low latencies.

Typical On-Premises CDP/AIA Distribution

In typical environments I’ve built over years, PKI employed multiple VMs with different responsibilities:

• One or more CAs (online, offline). These are main PKI components.
• SMB server (DFS). CA server publishes CDP/AIA files to remote share.
• IIS Web Server(s) which provides HTTP interface to PKI clients.

This is based on my other article Designing CRL Distribution Points and Authority Information Access locations and the setup looks like like this:

• CA servers generate CRLs and publish them to a dedicated SMB share located on one or more DFS servers. In case of an offline CA, files are copied to a SMB share manually by an administrator.
• On the IIS server, we have a dedicated website with a virtual directory that points to that SMB share.
• PKI clients communicate with IIS server to retrieve CA certificates and CRLs.

So far, so good. Due to rising CDP/AIA availability requirements, we have to put more effort on making this setup highly available. This means that we should have at least two DFS servers with replication enabled and at least two IIS servers in NLB pool. While not a big deal, this severely increases on-prem environment maintenance costs, because you nearly double your VM corpus for PKI support only.

The next challenge is to provide lower latency. This is not a problem when you only have a single office and all PKI clients reside in this office. If you run a geo-distributed company (have WFH users) then remote PKI clients will suffer from increased network latency. In fully on-premises environment, you would mirror IIS servers in remote locations and use DNS tricks (such as netmask ordering) to connect clients to closest IIS server. This also increases your PKI maintenance bill.

With cloud solutions, it is relatively easy and cheap to move PKI asset distribution locations to cloud services, that will provide high availability, redundancy and low latency out of the box. You are not required to be fully migrated to cloud or even have a hybrid environment.

Cloud-Based CDP/AIA Distribution Design

In this series, I will provide guidance on migrating on-premises CDP/AIA distribution points to Azure. While I will talk about Azure, same concepts and principles apply to other cloud providers, such as AWS or GCP. Depending on your requirements you can extend an existing on-premises setup with cloud services or completely move to a cloud-only setup.

Option 1 — On-Premises Extension with Cloud Services

The first option keeps your on-premises setup unchanged and moves external PKI clients to cloud services.

In this setup, you keep existing CDP/AIA setup as is and let internal users communicate with internal IIS servers, while remote users communicate with cloud services. Cloud services are also used should internal IIS servers fail. This is the most expensive setup, because you still have to maintain internal IIS servers and add cloud service costs. And when a client reports issues with the CDP/AIA connection, you will have to spend extra time to figure out whether the problem is with internal IIS servers or cloud service.

Option 2 — On-Premises Replacement with Cloud Services

A reasonable tradeoff is to completely eliminate internal IIS servers for CDP/AIA distribution and move to cloud-only setup as shown in fig.3:

In this setup, all clients (internal and external) communicate with cloud services to reach CDP/AIA distribution locations. This provides a single administration point, which is easier to troubleshoot, should any failure happen. This series is applicable to both setups (with or without internal IIS servers).

Notes

I’ve seen organizations attempted to host CRLs in Azure by creating a single storage account and configuring CDP/AIA URLs to point to that storage account. While it will work in demo/PoC setups, it is not sufficient for production. Azure storage is only part of the solution and adds a lot of drawbacks you don’t want to encounter in the future. I strongly recommend to follow this guide for a production-grade solution.

Important: Test this guide in test/dev environment first to ensure that your production environment is migrated smoothly.

Required Cloud Services

First of all, you are not required to have a hybrid environment, which allows even on-premises-only environments to utilize external cloud services for specific workloads without having to make a big jump into cloud. Let’s talk about required cloud services and their properties.

Azure Front Door (Formerly, Azure CDN)

Azure Front Door is…I will just quote what Microsoft says:

Azure Front Door is Microsoft’s modern cloud Content Delivery Network (CDN) that provides fast, reliable, and secure access between your users and your applications’ static and dynamic web content across the globe.

And it is very true. At the moment of this article writing, Azure maintains 192(!) points of presence (or edge locations) around the globe clients can connect to. Let’s say your data is hosted in Germany and your client is from Australia. Instead of making a network connection to Germany, the client will be automatically connected to the regionally closest (in Australia) point of presence (or edge endpoint). Then Azure will use internal high-speed networks to get the requested data and return it back to the client. From the client’s perspective, it will look like the data is located right here, in Australia.

Another benefit of Azure Front Door is caching. If you have multiple clients in Australia, while the data is located in Germany, only the first call will result in data retrieval from Germany, then the file will be cached by Azure Front Door and subsequent calls to the same file will cause file retrieval from the Front Door cache, which reduces network latency even more. Speaking about availability, there are other 191 edge endpoints to connect to should one of them fail. I personally call Azure Front Door an “NLB on steroids.” Really, in most NLB setups you have 2-4-8 backend nodes, here you have almost 200 out of the box.

Estimated Pricing

Previously, Azure CDN Classic (predecessor of Azure Front Door) was quite cheap, for me it cost around 2-4€/mo. However Azure CDN Classic was somewhat limited in functionality (which affected CDP/AIA distribution workload) and it’s announced to be retired on September 30, 2027.

While it is still possible to create Azure CDN Classic, there is virtually no real point in doing this. Azure CDN Classic is replaced with a full-blown CDN solution, with customizable routing, rules and policies. At the moment of this article writing, the Standard tier of Azure Front Door instance pricing is 32€/mo + VAT. Network traffic charges apply too (though network traffic is relatively cheap). Consult the Azure Pricing Calculator to estimate your costs. The good part of this is that you are not required to set up a dedicated Azure Front Door instance for every use case. You may have a single AFD instance for various different workloads, meaning it’s a matter of maintaining endpoints for each workload.

Azure Storage Account

AFD is a content delivery network, so it needs some data storage to serve. We will use Azure Storage accounts (two accounts for HA, I will talk about this separately when we come to it). Azure Storage is a durable, highly available and scalable storage cloud solution. While it is highly available, there are reasons to deploy two storage accounts with replication. This includes global Azure region failure and data residency binding. The latter part is more important: storage account is region-specific. If you need frequent access to same data in US and Europe, it may be reasonable to deploy two accounts, one in the US, another in Europe and configure data replication between them. This will reduce Azure Storage latency as well.

Estimated pricing

Azure Storage is really cheap, so having two storage accounts is quite affordable. Given that CDP/AIA storage requirements are extremely low (we are talking about kilobytes, rather than terabytes—upper limit for storage account at the time of this article is 5PiB), so you may expect 2-5€/mo + VAT for both storages combined.

So, all in all, you can expect something like 40€/mo + VAT. In exchange, you will reduce existing IIS web servers maintenance, since they aren’t much needed anymore.

Automation tools

I will provide a deployment and configuration guide using both, Azure Portal and PowerShell modules. While Azure Portal doesn’t require any dependencies except a web browser, the PowerShell approach will require the following modules from PowerShell gallery:

• Az.Accounts
• Az.Storage
• Az.Cdn

If you have them installed, ensure they are updated to the latest and greatest versions.

Part 1 Summary

In this post, I’ve outlined some theoretical aspects on designing durable, highly available and scalable PKI CDP/AIA distribution points using cloud services. In the next few posts, I will expand details and provide a step-by-step guide to implement our solution.