TG
All posts
2 min read

Migrating to Azure ASE v3: Dropping p99 Latency to 120ms

Escaping noisy-neighbour contention on a single-tenant App Service Environment, behind Front Door, a WAF and APIM — and closing every VAPT finding on the first re-test.

A healthcare API on shared Azure App Service was spiking to 850ms at p99. The application had not changed. Profiling the .NET and SQL Server tier with Kusto Query Language in Azure Application Insights pointed at two causes, both of them properties of the platform rather than the code: compute contention from co-tenants, and SNAT port exhaustion on outbound connections.

Neither is tunable from inside the application. The fix was to stop sharing the tenancy — a migration to a single-tenant Azure App Service Environment (ASE v3).

Rebuilding the perimeter around a private VNet

ASE v3 injects the compute environment directly into a private Virtual Network. That solves the contention and takes the public ingress with it: once the environment is private, every route into it has to be rebuilt. We rebuilt it as three filters in front of a network with no public ingress at all.

PRIVATE ASE V3 BEHIND FRONT DOOR, WAF AND APIMPUBLIC INTERNETAZURE GLOBAL EDGEDMZ SUBSCRIPTIONISOLATED ASE V3 VNETprivate endpoints onlySTATE, SECRETS AND TELEMETRYClientsweb · mobile · third-partyAzure Front Dooranycast · CDN · TLS 1.3Web Application FirewallOWASP CRS, L3–L7Azure API ManagementJWT validation · rate limitsInternal load balancerprivate IP.NET compute stampdedicated, single tenantMirth ConnectHL7 interface engineSQL Servercredential from vaultAzure Key Vaultmanaged identityApplication InsightsKQL profiling1. HTTPS :4432. inspect L73. clean traffic4. VNet peering / injection5. private IP6. HL7 routing7. queryfetch secrettraces + latencynumbered happy pathsecrets / telemetry
Three filters before the workload — WAF at L7, APIM for JWT and rate limits, then a private-only VNet with no public ingress at all.

How a request reaches the workload

  1. Clients — web, mobile and third-party systems — open an HTTPS connection on :443 to Azure Front Door, which provides global anycast routing, CDN caching and TLS 1.3 termination.
  2. The Web Application Firewall inspects it against the OWASP Core Rule Set at L3–L7.
  3. Clean traffic passes to Azure API Management in a DMZ subscription, which validates the JWT, enforces rate limits and quotas, and records the call.
  4. APIM crosses into the isolated ASE v3 VNet by VNet peering and injection — private connectivity, never the public backbone.
  5. An internal load balancer on a private IP distributes to the dedicated .NET microservices compute stamp.
  6. Healthcare messages route on to Mirth Connect, the HL7 interface engine.
  7. The service queries SQL Server, fetching its connection string from Azure Key Vault by managed identity, and emits latency and trace telemetry to Application Insights as it goes.

If Front Door cannot reach the peered VNet, the request fails at the edge rather than inside the private network — which is the correct place for it to fail, since nothing downstream is publicly reachable to fall back on.

Cryptographic hardening

The architecture also had to pass third-party penetration testing. We closed 100% of the VAPT findings on the first re-test with three controls:

Lucky13 mitigation. CBC-mode ciphersuites were disabled at the Front Door level, removing the timing side channel that the padding-oracle attack depends on, and TLS 1.3 is enforced across the whole perimeter.

HSTS. Strict Transport Security is enabled globally, so a downgrade to plaintext is refused by the browser rather than negotiated.

No standing secrets. Database connection strings and Azure Data Factory SSIS integration tokens were removed from application code and web configs entirely. They live in Key Vault and are read by managed identity, so there is no credential in the deployment artifact to leak.

The migration removed the noisy-neighbour effect and cut p99 API latency by 86%, from 850ms to 120ms.

GitHub
LinkedIn