NX-OS that we may come across on Cisco Nexus switches provide very interesting feature which is called FabricPath. FabricPath has evolved from TRILL, consists in “layer 2 routing” and definitely is one od the most important feature in modern datacenter.
What is and why would you need to use fabricpath ?
Thanks to Fabricpath we may get rid of Spanning Tree Protocol. We may build large topology with 40G and 100G links that may be easily extended. Slower routing would be a bottleneck in this evironment also having expensive 40G links that would be blocked by STP is something what we would like to avoid. Fabricpath relies on “Layer 2 routing” and uses “forgotten” IS-IS routing protocol, works on layer 2 and uses its own transport on layer 3, doesn’t require IP protocol. FabricPath makes decision based on fabricpath IS-IS routing table not on MAC addresses, the next hop is just a “Switch ID”. The main goal of FabricPath is calculation the SPT (Shortest Path Tree) between all FabricPath nodes. IS-IS doesn’t advertise MAC addresses in FabricPath, it has src and dst fabricpath switch ID in the header, traffic is routed via the Shortest Path Tree to destination “Switch ID” exact as L3 IS-IS or OSPF routing protocol. Also is very well scalable, we just add another Spine and Leaf switches as we need and run fabricpath feature on appropriate interfaces. Fabricpath is called MAC-in-MAC routing because forwarding decision is taken based on layer 2.
Before I go over the lab, let’s learn some terminology
Classic Ethernet – regular ethernet port, CE frames are encapsulated with new fabric path header
Leaf switch – connects CE domain to FP domain
Spine switch – FP backbone switch with all ports in FP domain
FP core ports – links on leaf up to spine, or spine to spine np switchport mode fabricpath links
CE edge ports – links on leaf to classic ethernet domain
Configuration:
Firstly we have to install fabricpath and run its
install feature-set fabricpath
feature-set fabricpath
We may set up switch ID manually but we don’t have to
SPINEs:
fabricpath switch-id 50
interface e2/1 – 6
switchport mode fabricpath
fabricpath switch-id 60
interface e2/1 – 6
switchport mode fabricpath
LEAFs:
fabricpath switch-id 51
interface e1/7 – 10
switchport mode fabricpath
interface e1/1
switchport mode access
switchport access vlan 100
fabricpath switch-id 61
interface e1/7 – 10
switchport mode fabricpath
interface e1/1
switchport mode access
switchport access vlan 100
For all ports that are not in Fabric Path we have to create SVIs on all FP switches Spines and Leafs:
vlan 100
mode fabricpath
show fabricpath route