ACI can leak routes between VRFs inside the fabric, so endpoints in different VRFs can talk without an external routing or security device. Conceptually that’s the same as VRF route leaking on any traditional platform. The configuration is not.
On a traditional platform you export a route target from one VRF and import it into another. Two lines, one concept, done. In ACI, the original method configures leaking and traffic filtering together through contracts and subnet flags, which means you can end up leaking routes as a side effect of applying a contract. People who come from a traditional background find this genuinely disorientating, and I don’t think that’s unreasonable.
ESGs fix it. That’s most of what this post is about, along with shared L3Out and the handoff options at the other end of the fabric.
The previous post covered route control and transit routing, and a couple of the subnet flags here will look familiar from it.
The EPG method
With the original EPG method, route leaking and traffic filtering are configured together through EPGs and contracts. The requirements are specific and all of them have to be right.
- The contract scope must be set to Tenant or Global.
- The provider EPG must carry a subnet marked Shared Between VRFs and No Default SVI Gateway.
- The consumer bridge domain subnet must be marked Shared Between VRFs.
- Intra-tenant and inter-tenant configurations are both supported.
- For an inter-tenant contract, the contract must be visible in both tenants, either by defining it in the common tenant or by exporting it as a contract interface.
Note where the subnets sit. The provider side needs a subnet under the EPG, not just under the bridge domain, and that EPG subnet is marked No Default SVI Gateway because it isn’t a gateway. It exists to identify the provider side for leaking. The consumer side uses the bridge domain subnet as normal.
That asymmetry is the part people get wrong, and it produces a configuration that looks symmetrical and isn’t.
There’s also a hard constraint worth knowing before you design around it. All provider-side EPGs in the same VRF must use non-overlapping subnets. With overlapping subnets, ACI cannot determine which provider EPG a cross-VRF packet belongs to, so the classification is ambiguous and the design doesn’t work. If you’re leaking into a VRF from several tenants with overlapping address space, this method has no answer for you.
The ESG method
ESGs decouple route leaking from traffic filtering. The two functions are configured separately, which is how it should have worked all along.
Route leaking is configured under Tenant, Networking, VRF, Inter-VRF Leaked Routes. Traffic filtering is configured with ESG-to-ESG contracts. ESGs always use a global class ID regardless of VRF, which is what makes the cross-VRF policy work cleanly.
Two configuration objects do the leaking:
EPG and BD Subnets specifies which bridge domain subnet to leak, to which tenant and VRF, and whether it can be advertised externally through an L3Out.
External Prefixes specifies prefix lists for leaking external routes learned from an L3Out into another VRF.
The practical difference is that ESG-based leaking does not require Shared Between VRFs on bridge domain subnets, and does not require subnets under the provider EPG. That removes the most error-prone part of the EPG method entirely, along with the asymmetry.
For anything new, I’d use ESG-based leaking. Keep the EPG method for backward compatibility with what’s already built, and for the situations where you’re extending an existing design rather than starting one.
Where subnets get defined
Worth having this straight, because it’s the thing that gets typed into the wrong box.
| Subnet location | Purpose |
|---|---|
| Bridge domain subnet, primary | The default gateway for servers. Always configure the gateway here |
| EPG subnet, provider side | Only for EPG-based VRF leaking. Marked No Default SVI Gateway. Identifies the provider side |
The gateway always lives on the bridge domain. The EPG subnet is a leaking construct and nothing else. If you’re using ESG-based leaking, you don’t need the EPG subnet at all, which is one fewer thing to get wrong.
Shared L3Out
Rather than giving every tenant its own L3Out, you can define a single shared L3Out in the common tenant and let multiple tenants consume it. This is the standard pattern for shared internet or WAN access, and it’s the right shape for most multi-tenant estates.
How it fits together:
- The shared L3Out is defined in a shared tenant, normally common.
- Other tenants consume it through shared service contracts.
- External routes can be selectively leaked into consuming tenant VRFs using the External Prefixes leak configuration.
- The policy control enforcement direction of the VRF affects how the shared L3Out behaves.
That last point connects back to the previous post. Enforcement direction is a VRF-level setting and it interacts with shared L3Out, so confirm it before build rather than after. It’s another instance of the same lesson: the VRF-level settings in ACI reach further than they look like they should.
The design advantage of a shared L3Out is that external connectivity is built once, with one set of peerings, one routing policy and one set of border leaf switches to size. The disadvantage is that it’s a shared dependency, and every tenant consuming it inherits its availability. That’s usually the right trade, but it should be a decision rather than a default.
Handoff options
Everything so far has assumed an L3Out on a border leaf. There are two alternatives at the fabric edge, one of which I’d not choose for anything new.
GOLF
GOLF is an L3Out terminated on the spine switches rather than on border leaf switches. It was introduced in ACI 2.0(1) to address route scale and traffic asymmetry limitations of the border leaf L3Out as it stood at the time. It uses BGP EVPN over OSPF between the spines and an external WAN router.
That problem statement is the important context, because it was a real problem then and it isn’t now.
Treat GOLF as legacy. From ACI 5.0, Cisco moved the underlay and overlay handoff back to the border leaf with the SR-MPLS handoff feature, which is a fairly clear signal about direction. GOLF also depends on OpFlex support in the external router, which restricts your platform choice considerably.
The limitations, for the sake of completeness:
- No multicast routing support.
- No VRF route leaking across VRFs within the fabric.
- Transit routing with shared services is not supported.
- Route scale is limited by spine LPM capacity.
- The GOLF L3Out is defined in the infra tenant and referenced from user tenants using a consumer label, which is an extra operational concept to carry.
The second one on that list is worth pausing on, given the first two thirds of this post. If you deploy GOLF, VRF leaking inside the fabric is off the table.
If you inherit GOLF, fine, it works. If somebody proposes it for a new design, the answer is a border leaf L3Out or SR-MPLS.
SR-MPLS handoff
From ACI 5.0(1), a border leaf switch can hand off to an external provider edge router using SR-MPLS or MPLS. A single BGP EVPN session carries routes for all VRFs, and MPLS labels identify the individual VRFs.
This replaces per-VRF routing protocol sessions, and that’s the whole point of it. If you have fifty VRFs handing off to an MPLS core, the border leaf L3Out approach means fifty sets of peerings, fifty sets of protocol policies, and fifty opportunities to configure something slightly differently from the other forty-nine. SR-MPLS handoff means one session.
The specifics:
- One BGP EVPN session carries every VRF.
- Traffic between the border leaf and the provider edge router is MPLS label encapsulated per VRF.
- FX generation leaf switches or later are required. Not supported on first-generation or EX switches.
- An SR-MPLS infra L3Out provides the underlay connectivity, and tenant L3Outs reference it.
The hardware requirement is the one to check first, because it’s binary and it’s easy to assume.
Comparison
| Attribute | Border leaf L3Out | GOLF | SR-MPLS handoff |
|---|---|---|---|
| L3Out location | Border leaf | Spine | Border leaf |
| VRF identification | Per-VRF protocol session | VXLAN VNID | MPLS label |
| Control plane | BGP, OSPF, EIGRP or static | BGP EVPN and OpFlex | BGP EVPN |
| Sessions | One per VRF | One for all tenants | One for all VRFs |
| Multicast | Supported | Not supported | Limited |
| VRF leaking | Supported | Not supported | Supported |
| Route scale | High with High LPM profile | Limited by spine LPM | High with High LPM profile |
| Hardware | Any supported leaf | Compatible spine and DCIG | FX leaf or later |
| Positioning | Default choice | Legacy | Service provider and multi-VRF |
Border leaf L3Out is the default for almost every design. SR-MPLS where there’s a large multi-VRF handoff to a provider edge router and the hardware supports it. GOLF is something you inherit, not something you choose.
What I’d take from this
ESG-based leaking for anything new. It separates routing from policy, drops the Shared Between VRFs flags and the provider EPG subnet, and removes the overlapping-subnet constraint from your design conversation.
A shared L3Out in the common tenant is the right pattern for shared internet or WAN, with external prefixes leaked selectively rather than wholesale.
Confirm the VRF policy control enforcement direction before you build a shared L3Out, not after.
And treat GOLF as legacy. If it’s in a proposal for a new fabric, that’s worth a question.
The last post in the series covers the multi-location architectures: Multi-Pod, Multi-Site and remote leaf. It’s the one where the design decisions have the largest consequences and the smallest number of chances to change your mind.