Case 01 Shipped

Private DNS for a PrivateLink‑only data platform

The platform rule said no public control-plane endpoints. I built the private DNS that makes the rule work in every region and account, traced a missing return route, and cut the account allowlist to entries with owners.

Context
AI platform of a global mobility company, 2026
My part
I provisioned the Route 53 private hosted zones and cross-account associations, diagnosed a missing Cloud WAN return route, and audited the account IP allowlist.
Team
A small platform team set the PrivateLink-only rule. A separate networking team owns Cloud WAN.
Status
In production

Stack: Route 53 private hosted zones · AWS PrivateLink · AWS Cloud WAN · Databricks · OpenTofu · Terragrunt · Atlantis

  • 3.19M audit-log events checked against the account IP allowlist Measured
  • 37 → 28 allowlist entries after the audit, each with a named owner Shipped
On this page

Context

The AI platform of a global mobility company runs on two clouds. Since May 2026 I have owned infrastructure workstreams on it, inside a small platform team, with everything in OpenTofu and Terragrunt and every change applied by Atlantis after review.

The platform has a rule: Databricks clusters reach the Databricks control plane over AWS PrivateLink, never over public endpoints.

Problem

PrivateLink puts the Databricks control plane behind VPC endpoints, but endpoints alone do nothing. A cluster reaches them only if the workspace hostnames resolve to the endpoint addresses inside its own VPC. Every VPC that runs or reaches a workspace needs that answer, including VPCs in other AWS accounts. Where the answer is missing, traffic fails or heads for a public address.

Constraints

  • Several AWS accounts, and every region and environment of the platform.
  • The corporate network reaches platform VPCs through AWS Cloud WAN, which a separate networking team owns.
  • Every change is code: reviewed in a pull request, then planned and applied by Atlantis.

Options

For name resolution across accounts there were three usual answers: a private zone per VPC, which duplicates records and drifts; Route 53 Resolver rules that forward to a central resolver, which adds a hop and a component to run; or one set of private hosted zones, associated with every VPC across accounts. I chose the shared zones.

Engineering decisions

  1. Route 53 private hosted zones answer for the workspace endpoint names.
  2. Each zone is associated with every VPC that needs it, across accounts, in code.
  3. Allowlist entries on the account console need a named owner, or they go.
  1. Corporate
    • Corporate network engineers and internal services
  2. Core network
    • AWS Cloud WAN routing between corporate and platform networks
  3. Platform VPCs
    • Workspace VPC, region A cluster compute
    • Workspace VPC, region B cluster compute
    • More regions every region and environment
  4. VPC endpoints
    • Workspace REST API interface endpoint
    • Secure cluster connectivity relay interface endpoint
  5. Control plane
    • Databricks control plane operated by Databricks
  • DNS (my part) Route 53 private hosted zones answer for the workspace names. Each zone is associated with the VPCs that need it, across accounts, so a name resolves to the private endpoint everywhere.
  • Routing Cloud WAN carries corporate traffic to the platform VPCs. A networking team owns it; I traced a missing return route and raised the fix.
  • Access The account console keeps an IP access list, and every entry has a named owner.
Cluster traffic reaches the Databricks control plane through VPC endpoints and PrivateLink. It finds those endpoints only because private DNS answers inside every VPC, in every account.Abstracted view. Names and counts are illustrative, not the real topology.

Implementation

A private hosted zone answers only inside the VPCs associated with it, and an association across accounts takes two steps: the zone’s account authorises it, then the VPC’s account accepts it. Both steps live in code, so a new VPC gets name resolution in the same change that creates it.

When the corporate network could not reach a workspace, I traced the failure to a missing return route in Cloud WAN and raised the fix with the networking team that owns it.

Validation

In each region and environment I checked that workspace hostnames resolve to private endpoint addresses from every associated VPC, and that clusters start through the private path.

Outcome

  • ShippedPrivate DNS for PrivateLink-only Databricks access, in every region and environment, kept in code.
  • ShippedA diagnosed Cloud WAN return-route gap, fixed by the team that owns the network.
  • ShippedAn audit of the account IP allowlist against 3.19 million audit-log events. The list went from 37 entries to 28, each with a named owner, after nine stale entries came out.