🛜 BGP-EVPN -- BGP At Doors of Autonomous Systems is Simple.
Brace yourselves the packets are coming!.
Ù¡- Topics
AS (Autonomous Systems)
Autonomous system is a group of networks managed by one organization (Internet Service Provider, Cloud Provider, Enterprises) that shares a common set of routing rules. These organizations control their own AS and decide how data flows in and out of their network while communicating with other networks.
Each AS is assigned by a unique number, known as Autonomous System Number(ASN) that helps it exchange routing information with other networks on the internet.
BGP (Border Gateway Protocol)
BGP is routing protocol that exchanges network reachability information between routers. In a BGP-EVPN setup, BGP extended to share not only IP prefixes but also MAC and IP address mappings, which enables routers to learn where hosts are located across the entire network.
VXLAN (Virtual eXtensible Local Area Network)
VXLAN is an overlay protocol that encapsulates Layer 2 frames within Layer 3 packets. Allowing to extend VLANs over a larger, IP-based network. Each VXLAN segment is identified by a VXLAN Network Identifier(VNI), ensuring that the encapsulated traffic reaches the correct destination.
EVPN (Ethernet Virtual Private Network)
EVPN is a modern networking technology used to extend Ethernet networks over a large distances, such as between data centers or branch offices. It enables devices in different locations to communicate as if they were on the same network.
EVPN provides Layer 2 and Layer 3 connectivity over a Layer 3 underlay by using VXLAN to transport Layer 2 frames across the network. BGP is then used to distribute MAC and IP address mappings, ensuring seamless communication between devices across different locations.
Ù¢- How Does This Work?
Bird View-Eye
Ethernet frames are the fundamental data units at the OSI Layer 2, containing source and destination MAC addresses, Payload data (the actual data being transmitted) and Error-checking informations. VXLAN(Data Plane) is designed to transport Layer 2 Ethernet frames across a Layer 3 (IP based) network, enabling devices on different physical networks to behave as if they’re on the same local network.
To transport Ethernet frames across networks that natively don’t support Layer 2, such as an IP-based (Layer 3) network, Encapsulation is the process of wrapping one type of data structure within another (here Layer 2 Ethernet frames to a Layer 4 UDP datagrams).
By transporting Ethernet frames, VXLAN(Control Plane) extends the Layer 2 network, so devices across multiple physical locations can use the same VLAN id or subnet, making them appear as if they’re part of the same local network. As well as preserving Layer 2 functionality by making communication using MAC addresses, VLANs and broadcast domains works seamlessly, even across geographically distributed networks. In short it’s like creating a virtual Layer 2 network over a Layer 3 infrastructure, let’s imagine we’re building private roads (virtual networks) on top of a public highway (the internet or our office network). This allows different companies or groups to share the same highway but still have their own private space.
AS24 AS46 and AS777 are connected via BGP sessions.
How Does It Work?
First step is BGP Peering or establishing connections, BGP routers in different AS form a connection called BGP session, these sessions exchange routing information with their peers.
The next step is the Route Advertisement, where BGP routes advertise the networks (prefixes or IP ranges) they can reach, example AS1 advertises it can reach 192.168.0.0/24. AS2 learns this route and advertise it to AS3 and AS3 now know it can reach 192.168.0.0/24 via AS2.
The third step is Path Selection, when a router receives multiple routes to the same destination, BGP selects the best path based on policies such as shortest AS path (fewer hops are preferred), local preference (routes with higher priority within an AS).
Last step is Routing Table Updates, the selected routes are added to the router's routing table, then the router forwards packets based on these routes.
Ù£ - Implementation
1- Environment Setup
Setup and Install required tools such as Docker and GNS3.
#!/bin/bash
set -e
# Setup of GNS3 & Docker for ubuntu
# Install & Configure GNS3
sudo apt update
sudo apt upgrade
sudo add-apt-repository ppa:gns3/ppa
sudo apt update
sudo apt-get install gns3-server gns3-gui
sudo dpkg --add-architecture i386
sudo apt update
sudo apt-get install gns3-iou
# Install & Configure Docker
sudo apt update
sudo apt install docker.io -y
sudo systemctl enable docker --now
sudo usermod -aG docker $USER
getent group docker
newgrp docker
sudo apt-get update
sudo apt install docker-compose -y
2- Docker Images
We will be configuring the required services (BGPd, OSPFd, IS-IS) and integrate into GNS3 with the use of a pre-built Docker image called FRRouting image.
This will configure a container with FRRouting services such as BGPd, OSPF and IS-IS.
- Host image Dockerfile:
FROM alpine:latest
RUN apk add --no-cache busybox
CMD ["/bin/sh"]
- Router image Dockerfile:
FROM frrouting/frr:latest
RUN \
echo 'service integrated-vtysh-config' > /etc/frr/vtysh.conf && \
sed -i \
-e 's/bgpd=no/bgpd=yes/g' \
-e 's/ospfd=no/ospfd=yes/g' \
-e 's/isisd=no/isisd=yes/g' \
/etc/frr/daemons
3- BGP-EVPN Implementation
Let's setup BGP-EVPN topology containing router reflector, 3 leafs routers each connected with a host machine like this:
-Router Reflector:
Configuration of the Route Reflector; the responsible for reflecting BGP updates between routers to reduce the need for full-mesh IBGP peering and simplify network scalability.
# Opening vtysh, the command line interface for FRRouting
# Opening terminal in config mode
vtysh
configure t
# eth0, eth1, eth2 and lo interfaces configuration
# Assigning IP addresses to the interfaces
interface eth0
ip address 10.1.1.1/30
interface eth1
ip address 10.1.1.5/30
interface eth2
ip address 10.1.1.9/30
interface lo
ip address 1.1.1.1/32
# Configuring the BGP routing protocol with the AS number 1
# Creating a peer group named ibgp, which is used to configure the BGP session with the neighbor
# The update source is the loopback interface, which is used to establish the BGP session
# Configuring the BGP session with any ip address in the range whithin the peer group ibgp
router bgp 1
neighbor ibgp peer-group
neighbor ibgp remote-as 1
neighbor ibgp update-source lo
bgp listen range 1.1.1.0/29 peer-group ibgp
# Enabling address-family for the L2VPN EVPN.
# Activating the neighbor and configuring it as a route reflector client
address-family l2vpn evpn
neighbor ibgp activate
neighbor ibgp route-reflector-client
exit-address-family
# Enabling the OSPF routing protocol
# Enabling OSPF on all interfaces, and have them participating for OSPF routing.
router ospf
network 0.0.0.0/0 area 0
-Leaf Routers:
Configuration of three leaf routers, which act as intermediaries between the Route Reflector and the end hosts, facilitating network traffic within the fabric.
Leaf Router 1:
# Creating new interface br0 type bridge
# Creating new interface vxlan10 type vxlan with id 10 and dstport 4789
# Seting the vxlan10 interface to up
# Adding the vxlan10 interface to the bridge br0
# Adding the eth1 interface to the bridge br0
/sbin/ip link add br0 type bridge
/sbin/ip link set dev br0 up
/sbin/ip link add vxlan10 type vxlan id 10 dstport 4789
/sbin/ip link set dev vxlan10 up
brctl addif br0 vxlan10
brctl addif br0 eth1
# Opening vtysh, the command line interface for FRRouting
# Opening terminal in config mode
vtysh
configure t
hostname _router-2
no ipv6 forwarding
# eth0 and lo interfaces configuration
# Assigning IP addresses to the interfaces
# Assigning the interfaces to the OSPF area 0
interface eth0
ip address 10.1.1.2/30
ip ospf area 0
interface lo
ip address 1.1.1.2/32
ip ospf area 0
# Configuring the BGP routing protocol with the AS number 1
# Mentioning route reflector in the AS number 1
# The update source is the loopback interface, which is used to establish the BGP session
router bgp 1
neighbor 1.1.1.1 remote-as 1
neighbor 1.1.1.1 update-source lo
# Enabling address-family for the L2VPN EVPN.
# Activating the neighbor
# All the VNI should be advertised with the configuration, in order to update their network information
address-family l2vpn evpn
neighbor 1.1.1.1 activate
advertise-all-vni
exit-address-family
# Enabling the OSPF routing protocol
router ospf
Leaf Router 2:
# Creating new interface br0 type bridge
# Creating new interface vxlan10 type vxlan with id 10 and dstport 4789
# Seting the vxlan10 interface to up
# Adding the vxlan10 interface to the bridge br0
# Adding the eth0 interface to the bridge br0
/sbin/ip link add br0 type bridge
/sbin/ip link set dev br0 up
/sbin/ip link add vxlan10 type vxlan id 10 dstport 4789
/sbin/ip link set dev vxlan10 up
brctl addif br0 vxlan10
brctl addif br0 eth0
# Opening vtysh, the command line interface for FRRouting
# Opening terminal in config mode
vtysh
configure t
hostname _router-3
no ipv6 forwarding
# eth1 and lo interfaces configuration
# Assigning IP addresses to the interfaces
# Assigning the interfaces to the OSPF area 0
interface eth1
ip address 10.1.1.6/30
ip ospf area 0
interface lo
ip address 1.1.1.3/32
ip ospf area 0
# Configuring the BGP routing protocol with the AS number 1
# Mentioning route reflector in the AS number 1
# The update source is the loopback interface, which is used to establish the BGP session
router bgp 1
neighbor 1.1.1.1 remote-as 1
neighbor 1.1.1.1 update-source lo
# Enabling address-family for the L2VPN EVPN.
# Activating the neighbor
# All the VNI should be advertised with the configuration, in order to update their network information
address-family l2vpn evpn
neighbor 1.1.1.1 activate
advertise-all-vni
exit-address-family
# Enabling the OSPF routing protocol
router ospf
Leaf Router 3:
# Creating new interface br0 type bridge
# Creating new interface vxlan10 type vxlan with id 10 and dstport 4789
# Seting the vxlan10 interface to up
# Adding the vxlan10 interface to the bridge br0
# Adding the eth0 interface to the bridge br0
/sbin/ip link add br0 type bridge
/sbin/ip link set dev br0 up
/sbin/ip link add vxlan10 type vxlan id 10 dstport 4789
/sbin/ip link set dev vxlan10 up
brctl addif br0 vxlan10
brctl addif br0 eth0
# Opening vtysh, the command line interface for FRRouting
# Opening terminal in config mode
vtysh
configure t
hostname _router-4
no ipv6 forwarding
# eth2 and lo interfaces configuration
# Assigning IP addresses to the interfaces
# Assigning the interfaces to the OSPF area 0
interface eth2
ip address 10.1.1.10/30
ip ospf area 0
interface lo
ip address 1.1.1.4/32
ip ospf area 0
# Configuring the BGP routing protocol with the AS number 1
# Mentioning route reflector in the AS number 1
# The update source is the loopback interface, which is used to establish the BGP session
router bgp 1
neighbor 1.1.1.1 remote-as 1
neighbor 1.1.1.1 update-source lo
# Enabling address-family for the L2VPN EVPN.
# Activating the neighbor
# All the VNI should be advertised with the configuration, in order to update their network information
address-family l2vpn evpn
neighbor 1.1.1.1 activate
advertise-all-vni
exit-address-family
# Enabling the OSPF routing protocol
router ospf
-End Hosts:
Configuration of the end hosts.
Host 1:# eth1 interface configuration
# Assigning IP addresses to the interface
ip addr add 20.1.1.1/24 dev eth1
Host 2:
# eth0 interface configuration
# Assigning IP addresses to the interface
ip addr add 20.1.1.2/24 dev eth0
Host 3:
# eth0 interface configuration
# Assigning IP addresses to the interface
ip addr add 20.1.1.3/24 dev eth0
For our verification, a simple ping allows us to see that we can access all the machines
through our RR using the VTEPs.
We must see the VXLAN configured to 10 as well as our
packets ICMP, as well as the OSPF packets configured.