--- /dev/null
+.. _evpn:
+
+****
+EVPN
+****
+
+:abbr:`EVPN` stands for Ethernet Virtual Private Network. This is an extension
+of BGP that enables the signaling of bridged (L2) and routed (L3) VPNs over a
+common network. EVPN is described in :rfc:`7432` and is updated by several
+additional RFCs and IETF drafts including :rfc:`9135` (Integrated Routing
+and Bridging in Ethernet VPN), :rfc:`9136` (IP Prefix Advertisement in Ethernet
+VPN), :rfc:`8584` (Framework for Ethernet VPN Designated Forwarder Election
+Extensibility), and :rfc:`8365` (A Network Virtualization Overlay Solution Using
+Ethernet VPN). FRR supports All-Active Layer-2 Multihoming for devices (MHD) via
+LACP Ethernet Segments as well as both Symmetric and Asymmetric IRB.
+FRR implements MAC-VRFs using a "VLAN-Based Service Interface" (:rfc:`7432`)
+and performs processing of Symmetric IRB routes following the
+"Interface-less IP-VRF-to-IP-VRF Model" (:rfc:`9136`).
+
+.. _evpn-concepts:
+
+EVPN Concepts
+=============
+BGP-EVPN is the control plane for the transport of Ethernet frames, regardless
+of whether those frames are bridged or routed. In the case of a VLAN-Based
+Service Interface with VXLAN encap, a single VNI is used to represent an EVPN
+Instance (EVI) and will have its own Route Distinguisher and set of
+Import/Export Route-Targets.
+
+A VNI is considered to be either Layer-2 (tied to a MAC-VRF) or Layer-3
+(tied to an IP-VRF), which indicates what kind of information is represented by
+the VRF. An IP-VRF represents a routing table (operating in much the same way as
+a VRF traditionally operates in L3VPN), while a MAC-VRF represents a bridging
+table i.e. MAC (fdb) and ARP/NDP entries.
+
+A MAC-VRF can be thought of as a VLAN with or without an SVI associated with it.
+An SVI is a Layer-3 interface bound to a bridging domain. In Linux an SVI can
+either be a traditional bridge or a VLAN subinterface of a VLAN-aware bridge.
+If there is an SVI for the VLAN, ARP/NDP entries can be bound to the MACs within
+the broadcast domain. Without an SVI, the VLAN operates in traditional L2
+fashion and MACs are the only type of host addresses known within the VLAN.
+
+In the same way that there can be a many-to-one relationship of SVIs to a VRF,
+there can also be a many-to-one relationship of MAC-VRFs (L2VNIs) to an IP-VRF
+(L3VNI). In FRR the L3VNI association for an L2VNI is determined by the
+presence of an SVI for the VLAN and the VRF membership of the SVI.
+If an L2VNI does not have an SVI or its SVI is not enslaved to a VRF, the L2VNI
+will be associated with the "default" VRF. If an L2VNI has an SVI whose master
+device is a VRF, then that L2VNI will be associated with its master VRF.
+
+.. _evpn-frr-configuration:
+
+FRR Configuration
+=================
+FRR learns about the system's Linux network interface configuration from the
+kernel via Netlink, however it does not manage network interfaces directly.
+The following sections will include examples of Linux interface configurations
+that are compatible with FRR's EVPN implementation. While there are multiple
+interface managers that can setup a proper kernel config (e.g. ifupdown2),
+these examples will use iproute2 to add/configure the interfaces.
+
+All of the examples will follow the same basic setup but use different, yet
+compatible, interface configurations.
+
+In this example we will setup the following:
+
+* An IP-VRF named vrf1, associated with L3VNI 100
+* An IP-VRF named vrf2, associated with L3VNI 200
+* An IP-VRF named vrf3, with no L3VNI associations
+* A MAC-VRF using VLAN 10, associated with L2VNI 110 and IP-VRF vrf1
+* A MAC-VRF using VLAN 20, associated with L2VNI 220 and IP-VRF vrf2
+* A MAC-VRF using VLAN 30, associated with L2VNI 330 and IP-VRF vrf3
+* A MAC-VRF using VLAN 40, associated with L2VNI 440 and IP-VRF default
+* A MAC-VRF using VLAN 50, associated with L2VNI 550 and operating L2-Only
+
+.. _evpn-sample-configuration:
+
+Sample Configuration
+--------------------
+This is a sample FRR configuration that implements the above EVPN environment.
+The first snippet will be the config in its entiretly, then each config element
+will be explained individually later in the document.
+
+The following snippet will result in a functional EVPN control plane if the
+corresponding Linux interface configuration is correct, compatible, and active:
+
+.. code-block:: frr
+
+ vrf vrf1
+ vni 100
+ exit-vrf
+ !
+ vrf vrf2
+ vni 200
+ exit-vrf
+ !
+ router bgp 4200000000
+ neighbor 192.168.122.12 remote-as internal
+ !
+ address-family ipv4 unicast
+ network 100.64.0.1/32
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.122.12 activate
+ advertise-all-vni
+ advertise-svi-ip
+ exit-address-family
+ exit
+ !
+ router bgp 4200000000 vrf vrf1
+ !
+ address-family ipv4 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ advertise ipv4 unicast
+ advertise ipv6 unicast
+ exit-address-family
+ exit
+ !
+ router bgp 4200000000 vrf vrf2
+ !
+ address-family ipv4 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ advertise ipv4 unicast
+ advertise ipv6 unicast
+ exit-address-family
+ exit
+
+A VRF will get its L3VNI association as a result of the ``vni`` command under
+the ``vrf`` stanza. Until this L3VNI association is made, zebra will discover
+the VNI from netlink but will consider it to be an L2VNI. The current L2 vs L3
+context of a VNI can be seen in the output of ``show evpn vni``.
+
+In this configuration we are telling zebra to consider VXLAN-ID 100 to be the
+L3VNI for vrf1 and VXLAN-ID 200 to be the L3VNI for vrf2.
+
+.. code-block:: frr
+
+ vrf vrf1
+ vni 100
+ exit-vrf
+ !
+ vrf vrf2
+ vni 200
+ exit-vrf
+
+The VTEP-IP (100.64.0.1) needs to be reachable by other VTEPs in the EVPN
+environment in order for VXLAN decapsulation to function. In this example we
+will advertise our local VTEP-IP using BGP (via the ``network`` statement), but
+static routes or other routing protocols like IS-IS or OSPF can also be used.
+
+In order to enable EVPN for a BGP instance, we must use the command
+``advertise-all-vni``. In this example we will be using the default VRF to
+carry the l2vpn evpn address-family, so we will enable EVPN for the default VRF.
+
+In this example, we plan to exchange EVPN routes with 192.168.122.12, so we
+will activate the l2vpn evpn address-family for this peer in order to allow
+EVPN NLRI to be advertised and received.
+
+The ``advertise-svi-ip`` command also belongs in the BGP instance where EVPN is
+enabled. This command tells FRR to originate "self" Type-2 routes for all the
+MAC/IP pairs associated with the local SVI interfaces.
+
+.. code-block:: frr
+
+ router bgp 4200000000
+ neighbor 192.168.122.12 remote-as internal
+ !
+ address-family ipv4 unicast
+ network 100.64.0.1/32
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ neighbor 192.168.122.12 activate
+ advertise-all-vni
+ advertise-svi-ip
+ exit-address-family
+ exit
+
+IPv4 and IPv6 BGP Prefixes from an IP-VRF are not exported to EVPN as Type-5
+routes until the respective ``advertise <afi> unicast`` command has been
+configured in the BGP instance of the VRF in question. All routes in the BGP
+RIB (locally originated, learned from a peer, or leaked from another VRF) will
+be eligible to be exported to EVPN so long as they are valid and selected in
+the VRF's unicast table.
+
+In this example, the BGP instances for vrf1 and vrf2 will have their static
+routes redistributed into the BGP loc-rib for the ipv4 unicast and ipv6 unicast
+address-families via the ``redistribute static`` statements. These unicast
+prefixes will then be exported into EVPN as Type-5 routes as a result of the
+``advertise ipv4 unicast`` and ``advertise ipv6 unicast`` commands.
+
+.. code-block:: frr
+
+ router bgp 4200000000 vrf vrf1
+ !
+ address-family ipv4 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ advertise ipv4 unicast
+ advertise ipv6 unicast
+ exit-address-family
+ exit
+ !
+ router bgp 4200000000 vrf vrf2
+ !
+ address-family ipv4 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family ipv6 unicast
+ redistribute static
+ exit-address-family
+ !
+ address-family l2vpn evpn
+ advertise ipv4 unicast
+ advertise ipv6 unicast
+ exit-address-family
+ exit
+
+.. _evpn-linux-interface-configuration:
+
+Linux Interface Configuration
+=============================
+The Linux kernel offers several options for configuring netdevices for an
+EVPN-VXLAN environment. The following section will include samples of a few
+netdev configurations that are compatible with FRR which implement the
+environment described above.
+
+Some high-level config considerations:
+
+* The local VTEP-IP should always be set to a reachable IP on the lo device.
+* An L3VNI should always have an SVI (aka the L3-SVI).
+* An L3-SVI should not be assigned an IP address, link-local or otherwise.
+
+ * IPv6 address autoconfiguration can be disabled via ``addrgenmode none``.
+
+* An SVI for an L2VNI is only needed for routing (IRB) or ARP/ND suppression.
+
+ * ARP/ND suppression is a kernel function, it is not managed by FRR.
+ * ARP/ND suppression is enabled per bridge_slave via ``neigh_suppress``.
+ * ARP/ND suppression should only be enabled on vxlan interfaces.
+ * IPv4/IPv6 forwarding should be disabled on SVIs not used for routing (IRB).
+
+* Dynamic MAC/VTEP learning should be disabled on VXLAN interfaces used in EVPN.
+
+ * Dynamic MAC learning is a function of the kernel bridge driver, not FRR.
+ * Dynamic MAC learning is toggled per bridge_slave via ``learning {on|off}``.
+ * Dynamic VTEP learning is a function of the kernel vxlan driver, not FRR.
+ * Dynamic VTEP learning is toggled per vxlan interface via ``[no]learning``.
+
+* The VXLAN interfaces should not have a ``remote`` VTEP defined.
+
+ * Remote VTEPs are learned via EVPN, so static VTEPs are unnecessary.
+
+.. _evpn-traditional-bridge-traditional-vxlan-devices:
+
+Traditional Bridges and Traditional VXLAN Devices
+-------------------------------------------------
+In the traditional bridge model, we use a separate ``bridge`` interface per
+MAC-VRF which acts as the SVI for that broadcast domain. A bridge is considered
+"traditional" if ``vlan_filtering`` is set to ``0`` (disabled) which indicates
+the bridge only has one broadcast domain which does not consider VLAN tags.
+Similarly, only one VNI is carried by each "traditional" ``vxlan`` interface.
+So in this deployment model, each VXLAN-enabled broadcast domain will have one
+traditional vxlan interface enslaved to one traditional bridge.
+
+Bridges created for an L3VNI broadcast domain should only have one member: the
+L3VNI vxlan device. Bridges created for an L2VNI broadcast domain generally
+have multiple members: the L2VNI vxlan device, plus any host/network ports
+where the L2 domain will be carried.
+
+To carry the broadcast domains of multiple traditional bridges over the same
+host/network port, a tagged ``vlan`` sub-interface of the port must be created
+per broadcast domain. The vlan sub-interfaces would then be enslaved to the
+traditional bridge, ensuring that only packets tagged with the expected VID are
+associated with the expected broadcast domain.
+
+.. code-block:: shell
+
+ ###################
+ ## vxlan vtep-ip ##
+ ###################
+ ip addr add 100.64.0.1/32 dev lo
+
+ #############################
+ ## ip-vrf vrf1 / l3vni 100 ##
+ #############################
+ ip link add vrf1 type vrf table 1100
+ ip link set vrf1 up
+ ip link add br100 type bridge
+ ip link set br100 master vrf1 addrgenmode none
+ ip link set br100 addr aa:bb:cc:00:00:64
+ ip link add vni100 type vxlan local 100.64.0.1 dstport 4789 id 100 nolearning
+ ip link set vni100 master br100 addrgenmode none
+ ip link set vni100 type bridge_slave neigh_suppress on learning off
+ ip link set vni100 up
+ ip link set br100 up
+
+ #############################
+ ## ip-vrf vrf2 / l3vni 200 ##
+ #############################
+ ip link add vrf2 type vrf table 1200
+ ip link set vrf2 up
+ ip link add br200 type bridge
+ ip link set br200 master vrf2 addrgenmode none
+ ip link set br200 addr aa:bb:cc:00:00:c8
+ ip link add vni200 type vxlan local 100.64.0.1 dstport 4789 id 200 nolearning
+ ip link set vni200 master br200 addrgenmode none
+ ip link set vni200 type bridge_slave neigh_suppress on learning off
+ ip link set vni200 up
+ ip link set br200 up
+
+ #################
+ ## ip-vrf vrf3 ##
+ #################
+ ip link add vrf3 type vrf table 1300
+ ip link set vrf3 up
+
+ ###############
+ ## l2vni 110 ##
+ ###############
+ ip link add br10 type bridge
+ ip link set br10 master vrf1
+ ip link set br10 addr aa:bb:cc:00:00:6e
+ ip addr add 10.0.10.1/24 dev br10
+ ip addr add 2001:db8:0:10::1/64 dev br10
+ ip link add vni110 type vxlan local 100.64.0.1 dstport 4789 id 110 nolearning
+ ip link set vni110 master br10 addrgenmode none
+ ip link set vni110 type bridge_slave neigh_suppress on learning off
+ ip link set vni110 up
+ ip link set br10 up
+
+ ###############
+ ## l2vni 220 ##
+ ###############
+ ip link add br20 type bridge
+ ip link set br20 master vrf2
+ ip link set br20 addr aa:bb:cc:00:00:dc
+ ip addr add 10.0.20.1/24 dev br20
+ ip addr add 2001:db8:0:20::1/64 dev br20
+ ip link add vni220 type vxlan local 100.64.0.1 dstport 4789 id 220 nolearning
+ ip link set vni220 master br20 addrgenmode none
+ ip link set vni220 type bridge_slave neigh_suppress on learning off
+ ip link set vni220 up
+ ip link set br20 up
+
+ ###############
+ ## l2vni 330 ##
+ ###############
+ ip link add br30 type bridge
+ ip link set br30 master vrf3
+ ip link set br30 addr aa:bb:cc:00:01:4a
+ ip addr add 10.0.30.1/24 dev br30
+ ip addr add 2001:db8:0:30::1/64 dev br30
+ ip link add vni330 type vxlan local 100.64.0.1 dstport 4789 id 330 nolearning
+ ip link set vni330 master br30 addrgenmode none
+ ip link set vni330 type bridge_slave neigh_suppress on learning off
+ ip link set vni330 up
+ ip link set br30 up
+
+ ###############
+ ## l2vni 440 ##
+ ###############
+ ip link add br40 type bridge
+ ip link set br40 addr aa:bb:cc:00:01:b8
+ ip addr add 10.0.40.1/24 dev br40
+ ip addr add 2001:db8:0:40::1/64 dev br40
+ ip link add vni440 type vxlan local 100.64.0.1 dstport 4789 id 440 nolearning
+ ip link set vni440 master br40 addrgenmode none
+ ip link set vni440 type bridge_slave neigh_suppress on learning off
+ ip link set vni440 up
+ ip link set br40 up
+
+ ###############
+ ## l2vni 550 ##
+ ###############
+ ip link add br50 type bridge
+ ip link set br50 addrgenmode none
+ ip link set br50 addr aa:bb:cc:00:02:26
+ ip link add vni550 type vxlan local 100.64.0.1 dstport 4789 id 550 nolearning
+ ip link set vni550 master br50 addrgenmode none
+ ip link set vni550 type bridge_slave neigh_suppress on learning off
+ sysctl -w net.ipv4.conf.br50.forwarding=0
+ sysctl -w net.ipv6.conf.br50.forwarding=0
+ ip link set vni550 up
+ ip link set br50 up
+
+ ##################
+ ## create vlan subinterface of eth0 for each l2vni vlan and enslave each
+ ## subinterface to the corresponding bridge
+ ##################
+ ip link set eth0 up
+ for i in 10 20 30 40 50; do
+ ip link add link eth0 name eth0.$i type vlan id $i;
+ ip link set eth0.$i master br$i;
+ ip link set eth0.$i up;
+ done
+
+
+To begin with, it creates a ``vrf`` interface named "vrf1" that is bound to the
+kernel routing table with ID 1100. This will represent the IP-VRF "vrf1" which
+we will later allocate an L3VNI for.
+
+.. code-block:: shell
+
+ ip link add vrf1 type vrf table 1100
+
+This block creates a traditional ``bridge`` interface named "br100", binds it to
+the VRF named "vrf1", disables IPv6 address autoconfiguration, and statically
+defines the MAC address of "br100". This traditional bridge is used for the
+L3VNI broadcast domain mapping to VRF "vrf1", i.e. "br100" is vrf1's L3-SVI.
+
+.. code-block:: shell
+
+ ip link add br100 type bridge
+ ip link set br100 master vrf1 addrgenmode none
+ ip link set br100 addr aa:bb:cc:00:00:64
+
+Here a traditional ``vxlan`` interface is created with the name "vni100" which
+uses a VTEP-IP of 100.64.0.1, carries VNI 100, and has Dynamic VTEP learning
+disabled. IPv6 address autoconfiguration is disabled for "vni100", then the
+interface is enslaved to "br100", ARP/ND suppression is enabled, and Dynamic
+VTEP Learning is disabled.
+
+.. code-block:: shell
+
+ ip link add vni100 type vxlan local 100.64.0.1 dstport 4789 id 100 nolearning
+ ip link set vni100 master br100 addrgenmode none
+ ip link set vni100 type bridge_slave neigh_suppress on learning off
+
+This completes the necessary configuration for a VRF and L3VNI.
+
+Here a traditional bridge named "br10" is created. We add "br10" to "vrf1" by
+setting "vrf1" as the ``master`` of "br10". It is not necessary to set the SVI
+MAC statically, but it is done here for consistency's sake. Since "br10" will
+be used for routing, IPv4 and IPv6 addresses are also added to the SVI.
+
+.. code-block:: shell
+
+ ip link add br10 type bridge
+ ip link set br10 master vrf1
+ ip link set br10 addr aa:bb:cc:00:00:6e
+ ip addr add 10.0.10.1/24 dev br10
+ ip addr add 2001:db8:0:10::1/64 dev br10
+
+If the SVI will not be used for routing, IP addresses should not be assigned to
+the SVI interface and IPv4/IPv6 "forwarding" should be disabled for the SVI via
+the appropriate sysctl nodes.
+
+.. code-block:: shell
+
+ sysctl -w net.ipv4.conf.<ifname>.forwarding=0
+ sysctl -w net.ipv6.conf.<ifname>.forwarding=0
+
+The following commands create a ``vxlan`` interface for VNI 100. Other than the
+VNI, The interface settings are the same for an L2VNI as they are for an L3VNI.
+
+.. code-block:: shell
+
+ ip link add vni110 type vxlan local 100.64.0.1 dstport 4789 id 110 nolearning
+ ip link set vni110 master br10 addrgenmode none
+ ip link set vni110 type bridge_slave neigh_suppress on learning off
+
+Finally, to limit a traditional bridge's broadcast domain to traffic matching
+specific VLAN-IDs, ``vlan`` subinterfaces of a host/network port need to be
+setup. This example shows the creation of a VLAN subinterface of "eth0"
+matching VID 10 with the name "eth0.10". By enslaving "eth0.10" to "br10"
+(instead of "eth0") we ensure that only Ethernet frames ingressing "eth0"
+tagged with VID 10 will be associated with the "br10" broadcast domain.
+
+.. code-block:: shell
+
+ ip link add link eth0 name eth0.10 type vlan id 10
+ ip link set eth0.10 master br10
+
+If you do not want to restrict the broadcast domain by VLAN-ID, you can skip
+the creation of the VLAN subinterfaces and directly enslave "eth0" to "br10".
+
+.. code-block:: shell
+
+ ip link set eth0 master br10
+
+This completes the necessary configuration for an L2VNI.
projects / mirror / frr.git / commitdiff