VXLAN allows us to create layer 2 networks over layer 3 infrastructure. The VXLAN base use case is to connect two or more layer three network domains and make them look like a common layer two domain. In the Data center environment this allows virtual machines on different networks to communicate as if they were in the same layer 2 subnet.
VxLAN is basically MAC in UDP encapsulation with a VxLAN header .Mos timportant part of VxLAN header is VxLan Network Identifier (VNI) field. This is a 24-bit value used to designate the individual VXLAN overlay. Devices or VMs in different VXLAN overlay networks cannot communicate with each other.The UDP destination port number is 4789, and the source port number is obtained after the inner Ethernet packet header is hashed. In the IP packet header, the source IP address and the destination IP address are the IP address of the local and remote VTEP of the VXLAN tunnel.
VXLAN bridging is the concept of using the VXLAN protocol to provide layer 2 connectivity across the layer 3 infrastructure. This is achieved on VTEP, by taking a traditional layer 2 domain and mapping the layer 2 domain into a VXLAN VNI. With a pair of VTEPs deployed, layer 2 connectivity can be achieved between the VTEPs across a layer 3 infrastructure.
According to topology, we have 2 different VM hosted in different servers and connected to different ToR switches. These ToR switches are VTEPs. Let’s assume VM1 wants to communicate with VM2 for the first time;
- VM1 sends and ARP request. VTEP 1 learns the mac address of VM1 and adds it to its mac table. (Bu default VxLAN doesn’t have control plane learning. Most common use case of VxLAN is using with EVPN control plane. My example is based on default flood and learn behavior and it will be flooded to all VTEPs which is participating the same VNI)
MAC-1: Port Vlan 10
- VTEP1 will encapsulate this arp request with VxLAN. Let’s assume VxLAN VNI is 29810. In this encapsulation, UDP header, IP header and VxLAN header will be added to original packet coming from VM1. (You also know that new layer 2 header will have the mac information of p2p interfaces) Source and Destination ip addresses of packet are the VTEP ip addresses.
- From the core point of view, they will only do ip forwarding on based on destination address, which is 192.1.1.6 in this example. (By the way in the data center environment there will usually be ECMP)
- The packet will arrive the destination VTEP and it will be de-capsulated. Meanwhile VTEP 2 will learn the MAC address of VM1 and install this information to its mac table. It can be later used to forward packets to VM1.
MAC-1: VTEP 192.1.1.1
- After de-capsulation, packet will be forwarded to VM2 and VM2 will response the ARP request. VTEP 2 will have the following mac table after receiving packets from VM2;
MAC-1: VTEP 192.1.1.1 MAC-2: Port Vlan 10
As I said earlier, this flood and learn behavior can create a lot of traffic because of ARP broadcasts. But this is the very basic mechanism for VxLAN. In the default behavior, VxLAN tunnels should be created manually but in case of using EVPN as a control plane, VxLAN tunnels can automatically created by VTEPs.
