Spanning-tree (STP) and ACI

Posted on March 28, 2019March 28, 2019 by Jody

Notables about ACI and Spanning-tree co-existence

ACI switches do not actively participate in spanning-tree
- To be specific, there is no Spanning-tree protocol processes running on any ACI switches in the fabric
- This means that ACI switches do not craft and send their own STP BPDUs
Even though ACI does not generate STP BPDUs, ACI switches will forward STP BPDUs across EPGs on which they are received.
- In fact, we want ACI to forward the BPDUs across to other switches! This allows the externally connected switches to maintain a loop-free topology and avoid broadcast storms and other nastiness that goes hand-in-hand when layer-2 loops form!
STP BPDUs from Legacy switches are flooded within the EPG, not the BD.This is a change from most all flooded traffic in ACI. Most of the time when we talk about traffic being flooded inside of ACI, the flooding is occurring at the BD level**.
- TCNs generated from switches running Spanning-tree will cause ACI to flush endpoints from EPGs in which they are received. This can result in intermittent traffic for devices on those EPGs. If you want to know more about this, check out this article – STP and ACI: Intermittent packet loss due to TCNs.
- **NOTE – By default, there are protocols are that flooded within a Bridge Domain. These protocols are OSPF/OSPFv3, BGP, EIGRP, CDP, LACP, LLDP, ISIS, IGMP, PIM, ARP/GARP, RARP, and ND (refer to the APIC L2 Config Guide on CCO).
ACI FEX ports have BPDUguard enabled by default, and this cannot be turned off. ACI never expects to see a BPDU from a device attached to a FEX.
The best way to sum up how ACI operates with STP; ACI acts as a hub.

Best Practices when connecting External Switches to your ACI Fabric via Layer-2

When you are connecting an external environment to your ACI Fabric via a Layer-2 connection, you’re probably doing so for one of two reasons: Number-1, you are establishing a link between the existing environment and your ACI Fabric in order to migrate workloads. Number-2, you’re in this for the long haul, and perhaps your ACI Fabric will co-exist with your existing environment for months or years to come.

Either way, you need to be armed with the right information that will allow you to design your Layer-2 connection between the two environments in such as a way that allows you and you to effectively troubleshoot and operate both environments, while ensuring maximum uptime.

For your Legacy Switches (i.e., switches which run Spanning-tree)

Configure your Spanning-tree link types to Shared on your external switches interfaces which connect to ACI.
1. As explained in this article, ACI Operation with L2 Switches and Spanning-tree Link Types, by default, the STP link type on Legacy switches is P2P. An STP link type using P2P ensures a rapid transition from blocking to forwarding, which is great until you remember that ACI acts as a hub for BPDUs. By configuring the STP link type as Shared for your external switch interfaces which connect to ACI, you ensure that you allow the switches to take their time with the STP transition process, thereby protecting your environment from potential STP loop formation.
Limit TCNs coming into ACI.Limiting TCNs can be done primarily by ensuring you have best practices configured for STP in your Legacy environment. We want to avoid STP TCNs in ACI, because they can lead to intermittent traffic loss when the endpoint tables are flushed. For more information on this, check out the article – STP and ACI: Intermittent packet loss due to TCNs. Below are recommendations which will help to limit the impact of STP TCNs on ACI:
1. Use vPC connections from Legacy switches to the ACI Fabric. The idea here is that the fewer forwarding STP ports connecting to ACI, the better.
2. BPDUguard and Portfast should be enabled on EVERY non-switch port. Portfast ensures that TCNs are not generated when host ports go up and down.
3. Limit the amount of switches below the first layer of external switches that connect to ACI (i.e., only host ports on switches) wherever possible << This limits the STP radius
4. Prune unnecessary Vlans from Trunk ports.
5. Don’t enable all 4094 Vlans on your External switches. Only configure what is needed.
6. And if all else fails, consider Outside/Inside EPGs for your Vlan. The Outside/Inside EPG design for Vlans allow us to isolate Outside devices and their TCNs from the Inside devices already migrated to ACI. We’ll explain more about this in the article STP and ACI: Intermittent packet loss due to TCNs.
(optional) – Understand the design differences between MST and RPVST/PVST.This is optional because you may not be using MST at all. However, if you are using MST, or plan on it, you need to be aware of the fundamental differences between it and the more commonly used RPVST and PVST.
1. BPDU frames in MST are sent on the Native Vlan (these are sent untagged), and not on a per-vlan basis as is the case with RPVST and PVST. This means we will have to make configuration specifically to accommodate for these differences in the ACI Fabric.
2. To deal with the untagged MST BPDUs, you’ll need to configure a couple of things differently than what you do when dealing with RPVST or PVST.
  1. Create a Switch Policy Group for your MST region – Under Fabric Access Policies, you’ll need to create a Switch Policy Group (note – I’m not talking about Interface Policy Groups). For this Policy Group, you’ll create a Spanning-tree policy and add in information about your MST region.
  2. Native Vlan EPG – For ACI, in your Tenant where you Layer-2 connection resides, you will need to create a specific EPG to carry the MST BPDUs. The static path bindings for your Legacy switches will be configured as dot1p (native). Failure to do this could very likely result in a loop.

For your ACI Fabric

Enable MisCabling Protocol (MCP) Fabric-wide, and on all interfaces. MCP detects loops from external sources (i.e., misbehaving servers, external networking equipment running STP) and will err-disable the interface on which ACI receives its own packet. Enabling this feature is a best practice and it should be enabled globally and on all interfaces, regardless of the end device.
1. NOTE – Because MCP works to stop Layer-2 loops, it should be enabled right away on an ACI Fabric prior to connecting Layer-2 devicesfor migration purposes.
2. If you want a detailed explanation of MCP, please take a look at the article here, Using MCP for ACI.

Spanning-tree (STP) and ACI - (2024)