A gateway is a device that ensures communication between VXLANs identified by different VNIs and between VXLANs and non-VXLANs. With the distributed gateway routing Layer 3 gateways are configured on leaf devices. With the distributed gateway deployment a leaf node only needs to learn the ARP entries of servers attached to it.
VXLAN routing of an encapsulated frame, involves the routing of traffic based not on the destination IP address of the outer VXLAN header but the inner header or overlay tenant IP address.
According to topology, 2 gateways are configured at the Leaf layer, namely on VTEP 1. If VM1 wants to communicate with VM2 the routing will be done on VTEP1. VTEP 1 plays 2 role in this example; Layer 2 gateway and Layer 3 gateway.
We will not need VxLAN tunnels to Spine layer. Spine layer will only do IP forwarding on based outer VxLAN header. And Spine layer will not have ARP entries for the tenant networks, VM1 and VM2.
In the central gateway deployment all ARP entries for all tenant VMs will be handled by Spine.
We will need a VxLAN tunnel between VTEP 1 and VTEP 2, after the routing happened.
Step by step explanation of routing between 2 different VNIs;
- VM1 sends the packet to VM2.
- VTEP 1 hosts the gateway of VM1 and VM2. It does routing lookup and sees that the packet should be forwarded over VNI 29820. VTEP 1 updates it ARP table with VM1, encapsulates traffic into VxLAN tunnel with a VNI of 2980 and destination VTEP is 192.1.1.4
- Packets are sent over the data center fabric as below
- Finally the packet will be de-capsulated and send to the VM2.
Best practice for VxLAN routing and bridging is doing it with EVPN control plane. In this example I just gave the simple encapsulation and de-capsulation example.
Suggested Readings
Underlay Design for VxLAN EVPN Networks
