VXLAN - Extended VLAN

The Virtual Extensible VLAN (VXLAN) standard proposed by Cisco and VMware this month will create logical networks -- or extended VLANs -- for long distance virtual machine (VM) migration across geographically dispersed data centers. VXLAN will ultimately enable the kind of long-distance segmentation necessary for multi-tenant cloud networks.
Cisco and VMware have already incorporated the proposed VXLAN standard into products, but the VXLAN draft, which was also co-authored by Cisco, VMware, Arista Networks, Broadcom Corporation, Citrix Systems and Red Hat Inc., awaits standardization by the Internet Engineering Task Force (IETF).

Why the need for the VXLAN standard?
VLANs have long been used to separate data streams, but the IEEE 802.1Q VLAN specification provides only 4,094 VLAN identifiers. A top-of-rack switch may be connected to more than 40 servers. Each server may support multiple VMs with each VM communicating on multiple VLANs. A data center may contain many racks so the total number of VLANs may exceed the 4,094 limit. In addition, VMs comprising a single application may reside in geographically different data centers. These VMs must be connected via a Layer 2 network, so VLAN identifiers must be unique across geographies.

VXLAN standard isolates application data
TheVXLAN standard, outlined in this  RFC draft, groups VLANs associated with an application into a segment defined by a 24-bit identifier called a VXLAN Network Identifier (VNI).  As many as 16 million VNIs may be defined in any administrative domain and each VNI may contain up to 4,094 VLANs. Customer data is kept separate because only VMs operating within the same VNI can communicate.
The Layer 2 network visible to VMs is carried over a Layer 3 network in UDP packets. This enables data centers to operate within different IP subnets. Only the Layer 2 network is visible to VMs, so it's possible for a VM within an application to migrate from one data center to another without the change becoming visible to the relocating VM or any of the other VMs comprising the application.

Communicating between VMs in a VXLAN environment
VM software need not be modified to operate within a VXLAN environment. VNIs are assigned to the VMs operating on a server by a Virtual Tunnel End Point (VTEP). The VTEP resides within the server hypervisor and generates the headers needed to specify the VNI and communicate across the Layer 3 network.

To communicate with other VMs in the segment, a VM creates packets identical to the packets that would be created in a non-VXLAN, non-virtual environment. Packets consist of standard MAC frames with an Ethernet header, source and destination MAC addresses, and Ethernet type. A VLAN tag is included if the VM utilizes multiple VLANs.
The VTEP supporting the VM then encapsulates the packet with an 8-byte VXLAN header. The header contains a single-bit VXLAN flag, the 24-bit VNI and a total of 39 bits reserved for future use. The resulting packet is then enclosed in a UDP packet. The destination IP address is the address of the VTEP to which the packet will be sent. The source IP is the address of the sending VTEP.
The packet is then enclosed in an outer Ethernet packet with the destination MAC of either the destination server or the router that will forward the packet.

Communication between VMs requires IP multicast
When a VM initiates communication with another VM, it operates just as it would in a non-VXLAN environment. If the destination is on the same subnet, it generates an ARP request broadcast for the destination. If they are on different subnets, it ARPs for the first hop router. The VTEP encapsulates the ARP request in VXLAN and UDP headers and sends  the broadcast packet to the IP multicast group associated with the VXLAN segment to which the communicating VMs belong. Assignment of segments to IP multicast groups is done by the management layer.
The VTEP supporting the destination VM strips off the UDP and VXLAN headers and passes the ARP request to the VM, which then responds to the request.  When the response reaches the source VTEP and VM, both VTEPs and VMs have learned the IP and MAC addresses needed to communicate.  Subsequent communication travels VTEP to VTEP without need for multicast.
In some cases, VMs will need to communicate outside the VXLAN environment. A VXLAN gateway strips off the VXLAN and UDP headers and forwards the packet to the destination MAC specified in the packet prepared by the VM. Neither the source VM nor the destination equipment needs to be modified to take part in the interchange. A VXLAN gateway could be implemented in software or in a switch or router.

credit: techtarget.com