From: Quentin Young Date: Fri, 8 Dec 2017 23:22:26 +0000 (-0500) Subject: doc: more organizing & updating X-Git-Tag: frr-5.0-dev~165^2~96 X-Git-Url: https://git.puffer.fish/?a=commitdiff_plain;h=b22ba015ef8c42e584876d9ac73df8e2b71b6eaf;p=mirror%2Ffrr.git doc: more organizing & updating * Add chapter on BGPD * Add diagram for git workflow * Convert next-hop tracking documents to ReST * Update & organize workflow document * Move ldpd docs back up to the parent directory Signed-off-by: Quentin Young --- diff --git a/doc/developer/bgpd.rst b/doc/developer/bgpd.rst new file mode 100644 index 0000000000..053c0779de --- /dev/null +++ b/doc/developer/bgpd.rst @@ -0,0 +1,8 @@ +BGPD +========================= + +.. toctree:: + :maxdepth: 2 + + next-hop-tracking + diff --git a/doc/developer/conf.py b/doc/developer/conf.py index 6c39c6e26d..233056666d 100644 --- a/doc/developer/conf.py +++ b/doc/developer/conf.py @@ -231,7 +231,7 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'FRR.tex', u'FRR Developer\'s Documentation', + (master_doc, 'FRR.tex', u'FRR Developer\'s Manual', u'FRR', 'manual'), ] @@ -261,7 +261,7 @@ latex_documents = [ # One entry per manual page. List of tuples # (source start file, name, description, authors, manual section). man_pages = [ - (master_doc, 'frr', u'FRR Developer\'s Documentation', + (master_doc, 'frr', u'FRR Developer\'s Manual', [author], 1) ] @@ -275,7 +275,7 @@ man_pages = [ # (source start file, target name, title, author, # dir menu entry, description, category) texinfo_documents = [ - (master_doc, 'FRR', u'FRR Developer\'s Documentation', + (master_doc, 'FRR', u'FRR Developer\'s Manual', author, 'FRR', 'One line description of project.', 'Miscellaneous'), ] diff --git a/doc/developer/git_branches.png b/doc/developer/git_branches.png new file mode 100644 index 0000000000..21001c30a1 Binary files /dev/null and b/doc/developer/git_branches.png differ diff --git a/doc/developer/index.rst b/doc/developer/index.rst index 6f0dbebdfd..f1a39d2828 100644 --- a/doc/developer/index.rst +++ b/doc/developer/index.rst @@ -6,5 +6,6 @@ Welcome to FRR's documentation! workflow library + bgpd building diff --git a/doc/developer/ldpd-basic-test-setup.md b/doc/developer/ldpd-basic-test-setup.md deleted file mode 100644 index b25a2b6d4b..0000000000 --- a/doc/developer/ldpd-basic-test-setup.md +++ /dev/null @@ -1,681 +0,0 @@ -## Topology - -The goal of this test is to verify that the all the basic functionality -of ldpd is working as expected, be it running on Linux or OpenBSD. In -addition to that, more advanced features are also tested, like LDP -sessions over IPv6, MD5 authentication and pseudowire signaling. - -In the topology below there are 3 PE routers, 3 CE routers and one P -router (not attached to any consumer site). - -All routers have IPv4 addresses and OSPF is used as the IGP. The -three routers from the bottom of the picture, P, PE2 and PE3, are also -configured for IPv6 (dual-stack) and static IPv6 routes are used to -provide connectivity among them. - -The three CEs share the same VPLS membership. LDP is used to set up the -LSPs among the PEs and to signal the pseudowires. MD5 authentication is -used to protect all LDP sessions. - -``` - CE1 172.16.1.1/24 - + - | - +---+---+ - | PE1 | - | IOS XE| - | | - +---+---+ - | - | 10.0.1.0/24 - | - +---+---+ - | P | - +------+ IOS XR+------+ - | | | | - | +-------+ | - 10.0.2.0/24 | | 10.0.3.0/24 -2001:db8:2::/64 | | 2001:db8:3::/64 - | | - +---+---+ +---+---+ - | PE2 | | PE3 | - |OpenBSD+-------------+ Linux | - | | | | - +---+---+ 10.0.4.0/24 +---+---+ - | 2001:db8:4::/64 | - + + - 172.16.1.2/24 CE2 CE3 172.16.1.3/24 -``` - -## Configuration - -#### Linux -1 - Enable IPv4/v6 forwarding: -``` -# sysctl -w net.ipv4.ip_forward=1 -# sysctl -w net.ipv6.conf.all.forwarding=1 -``` - -2 - Enable MPLS forwarding: -``` -# modprobe mpls-router -# modprobe mpls-iptunnel -# echo 100000 > /proc/sys/net/mpls/platform_labels -# echo 1 > /proc/sys/net/mpls/conf/eth1/input -# echo 1 > /proc/sys/net/mpls/conf/eth2/input -``` - -3 - Set up the interfaces: -``` -# ip link add name lo1 type dummy -# ip link set dev lo1 up -# ip addr add 4.4.4.4/32 dev lo1 -# ip -6 addr add 4:4:4::4/128 dev lo1 -# ip link set dev eth1 up -# ip addr add 10.0.4.4/24 dev eth1 -# ip -6 addr add 2001:db8:4::4/64 dev eth1 -# ip link set dev eth2 up -# ip addr add 10.0.3.4/24 dev eth2 -# ip -6 addr add 2001:db8:3::4/64 dev eth2 -``` - -4 - Set up the bridge and pseudowire interfaces: -``` -# ip link add type bridge -# ip link set dev bridge0 up -# ip link set dev eth0 up -# ip link set dev eth0 master bridge0 -# ip link add name mpw0 type dummy -# ip link set dev mpw0 up -# ip link set dev mpw0 master bridge0 -# ip link add name mpw1 type dummy -# ip link set dev mpw1 up -# ip link set dev mpw1 master bridge0 -``` - -> NOTE: MPLS support in the Linux kernel is very recent and it still -doesn't support pseudowire interfaces. We are using here dummy interfaces -just to show how the VPLS configuration should look like in the future. - -5 - Add static IPv6 routes for the remote loopbacks: -``` -# ip -6 route add 2:2:2::2/128 via 2001:db8:3::2 -# ip -6 route add 3:3:3::3/128 via 2001:db8:4::3 -``` - -6 - Edit /etc/frr/ospfd.conf: -``` -router ospf - network 4.4.4.4/32 area 0.0.0.0 - network 10.0.3.4/24 area 0.0.0.0 - network 10.0.4.4/24 area 0.0.0.0 -! -``` - -7 - Edit /etc/frr/ldpd.conf: -``` -debug mpls ldp messages recv -debug mpls ldp messages sent -debug mpls ldp zebra -! -mpls ldp - router-id 4.4.4.4 - dual-stack cisco-interop - neighbor 1.1.1.1 password opensourcerouting - neighbor 2.2.2.2 password opensourcerouting - neighbor 3.3.3.3 password opensourcerouting - ! - address-family ipv4 - discovery transport-address 4.4.4.4 - label local advertise explicit-null - ! - interface eth2 - ! - interface eth1 - ! - ! - address-family ipv6 - discovery transport-address 4:4:4::4 - ttl-security disable - ! - interface eth2 - ! - interface eth1 - ! - ! -! -l2vpn ENG type vpls - bridge br0 - member interface eth0 - ! - member pseudowire mpw0 - neighbor lsr-id 1.1.1.1 - pw-id 100 - ! - member pseudowire mpw1 - neighbor lsr-id 3.3.3.3 - neighbor address 3:3:3::3 - pw-id 100 - ! -! -``` - -> NOTE: We have to disable ttl-security under the ipv6 address-family -in order to interoperate with the IOS-XR router. GTSM is mandatory for -LDPv6 but the IOS-XR implementation is not RFC compliant in this regard. - -8 - Run zebra, ospfd and ldpd. - -#### OpenBSD -1 - Enable IPv4/v6 forwarding: -``` -# sysctl net.inet.ip.forwarding=1 -# sysctl net.inet6.ip6.forwarding=1 -``` - -2 - Enable MPLS forwarding: -``` -# ifconfig em2 10.0.2.3/24 mpls -# ifconfig em3 10.0.4.3/24 mpls -``` - -3 - Set up the interfaces: -``` -# ifconfig lo1 alias 3.3.3.3 netmask 255.255.255.255 -# ifconfig lo1 inet6 3:3:3::3/128 -# ifconfig em2 inet6 2001:db8:2::3/64 -# ifconfig em3 inet6 2001:db8:4::3/64 -``` - -4 - Set up the bridge and pseudowire interfaces: -``` -# ifconfig bridge0 create -# ifconfig bridge0 up -# ifconfig em1 up -# ifconfig bridge0 add em1 -# ifconfig mpw0 create -# ifconfig mpw0 up -# ifconfig bridge0 add mpw0 -# ifconfig mpw1 create -# ifconfig mpw1 up -# ifconfig bridge0 add mpw1 -``` - -5 - Add static IPv6 routes for the remote loopbacks: -``` -# route -n add 4:4:4::4/128 2001:db8:4::4 -# route -n add 2:2:2::2/128 2001:db8:2::2 -``` - -6 - Edit /etc/frr/ospfd.conf: -``` -router ospf - network 10.0.2.3/24 area 0 - network 10.0.4.3/24 area 0 - network 3.3.3.3/32 area 0 -! -``` - -7 - Edit /etc/frr/ldpd.conf: -``` -debug mpls ldp messages recv -debug mpls ldp messages sent -debug mpls ldp zebra -! -mpls ldp - router-id 3.3.3.3 - dual-stack cisco-interop - neighbor 1.1.1.1 password opensourcerouting - neighbor 2.2.2.2 password opensourcerouting - neighbor 4.4.4.4 password opensourcerouting - ! - address-family ipv4 - discovery transport-address 3.3.3.3 - label local advertise explicit-null - ! - interface em3 - ! - interface em2 - ! - ! - address-family ipv6 - discovery transport-address 3:3:3::3 - ttl-security disable - ! - interface em3 - ! - interface em2 - ! - ! -! -l2vpn ENG type vpls - bridge br0 - member interface em1 - ! - member pseudowire mpw0 - neighbor lsr-id 1.1.1.1 - pw-id 100 - ! - member pseudowire mpw1 - neighbor lsr-id 4.4.4.4 - neighbor address 4:4:4::4 - pw-id 100 - ! -! -``` - -8 - Run zebra, ospfd and ldpd. - -#### Cisco routers -CE1 (IOS): -``` -interface FastEthernet0/0 - ip address 172.16.1.1 255.255.255.0 - ! -! -``` - -CE2 (IOS): -``` -interface FastEthernet0/0 - ip address 172.16.1.2 255.255.255.0 - ! -! -``` - -CE3 (IOS): -``` -interface FastEthernet0/0 - ip address 172.16.1.3 255.255.255.0 - ! -! -``` - -PE1 - IOS-XE (1): -``` -mpls ldp neighbor 2.2.2.2 password opensourcerouting -mpls ldp neighbor 3.3.3.3 password opensourcerouting -mpls ldp neighbor 4.4.4.4 password opensourcerouting -! -l2vpn vfi context VFI - vpn id 1 - member pseudowire2 - member pseudowire1 -! -bridge-domain 1 - member GigabitEthernet1 service-instance 1 - member vfi VFI -! -interface Loopback1 - ip address 1.1.1.1 255.255.255.255 -! -interface pseudowire1 - encapsulation mpls - neighbor 3.3.3.3 100 -! -interface pseudowire2 - encapsulation mpls - neighbor 4.4.4.4 100 -! -interface GigabitEthernet3 - ip address 10.0.1.1 255.255.255.0 - mpls ip -! -router ospf 1 - network 0.0.0.0 255.255.255.255 area 0 -! -``` - -P - IOS-XR (2): -``` -interface Loopback1 - ipv4 address 2.2.2.2 255.255.255.255 - ipv6 address 2:2:2::2/128 -! -interface GigabitEthernet0/0/0/0 - ipv4 address 10.0.1.2 255.255.255.0 -! -interface GigabitEthernet0/0/0/1 - ipv4 address 10.0.2.2 255.255.255.0 - ipv6 address 2001:db8:2::2/64 - ipv6 enable -! -interface GigabitEthernet0/0/0/2 - ipv4 address 10.0.3.2 255.255.255.0 - ipv6 address 2001:db8:3::2/64 - ipv6 enable -! -router static - address-family ipv6 unicast - 3:3:3::3/128 2001:db8:2::3 - 4:4:4::4/128 2001:db8:3::4 - ! -! -router ospf 1 - router-id 2.2.2.2 - address-family ipv4 unicast - area 0 - interface Loopback1 - ! - interface GigabitEthernet0/0/0/0 - ! - interface GigabitEthernet0/0/0/1 - ! - interface GigabitEthernet0/0/0/2 - ! - ! -! -mpls ldp - router-id 2.2.2.2 - neighbor - 1.1.1.1:0 password clear opensourcerouting - 3.3.3.3:0 password clear opensourcerouting - 4.4.4.4:0 password clear opensourcerouting - ! - address-family ipv4 - ! - address-family ipv6 - discovery transport-address 2:2:2::2 - ! - interface GigabitEthernet0/0/0/0 - address-family ipv4 - ! - ! - interface GigabitEthernet0/0/0/1 - address-family ipv4 - ! - address-family ipv6 - ! - ! - interface GigabitEthernet0/0/0/2 - address-family ipv4 - ! - address-family ipv6 - ! - ! -! -``` - -## Verification - Control Plane - -Using the CLI on the Linux box, the goal is to ensure that everything -is working as expected. - -First, verify that all the required adjacencies and neighborships sessions -were established: - -``` -linux# show mpls ldp discovery -Local LDP Identifier: 4.4.4.4:0 -Discovery Sources: - Interfaces: - eth1: xmit/recv - LDP Id: 3.3.3.3:0, Transport address: 3.3.3.3 - Hold time: 15 sec - LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3 - Hold time: 15 sec - eth2: xmit/recv - LDP Id: 2.2.2.2:0, Transport address: 2.2.2.2 - Hold time: 15 sec - LDP Id: 2.2.2.2:0, Transport address: 2:2:2::2 - Hold time: 15 sec - Targeted Hellos: - 4.4.4.4 -> 1.1.1.1: xmit/recv - LDP Id: 1.1.1.1:0, Transport address: 1.1.1.1 - Hold time: 45 sec - 4:4:4::4 -> 3:3:3::3: xmit/recv - LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3 - Hold time: 45 sec - -linux# show mpls ldp neighbor -Peer LDP Identifier: 1.1.1.1:0 - TCP connection: 4.4.4.4:40921 - 1.1.1.1:646 - Session Holdtime: 180 sec - State: OPERATIONAL; Downstream-Unsolicited - Up time: 00:06:02 - LDP Discovery Sources: - IPv4: - Targeted Hello: 1.1.1.1 - -Peer LDP Identifier: 2.2.2.2:0 - TCP connection: 4:4:4::4:52286 - 2:2:2::2:646 - Session Holdtime: 180 sec - State: OPERATIONAL; Downstream-Unsolicited - Up time: 00:06:02 - LDP Discovery Sources: - IPv4: - Interface: eth2 - IPv6: - Interface: eth2 - -Peer LDP Identifier: 3.3.3.3:0 - TCP connection: 4:4:4::4:60575 - 3:3:3::3:646 - Session Holdtime: 180 sec - State: OPERATIONAL; Downstream-Unsolicited - Up time: 00:05:57 - LDP Discovery Sources: - IPv4: - Interface: eth1 - IPv6: - Targeted Hello: 3:3:3::3 - Interface: eth1 -``` - -Note that the neighborships with the P and PE2 routers were established -over IPv6, since this is the default behavior for dual-stack LSRs, as -specified in RFC 7552. If desired, the **dual-stack transport-connection -prefer ipv4** command can be used to establish these sessions over IPv4 -(the command should be applied an all routers). - -Now, verify that there's a remote label for each PE address: -``` -linux# show mpls ldp binding -1.1.1.1/32 - Local binding: label: 20 - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 imp-null - 2.2.2.2 24000 - 3.3.3.3 20 -2.2.2.2/32 - Local binding: label: 21 - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 18 - 2.2.2.2 imp-null - 3.3.3.3 21 -3.3.3.3/32 - Local binding: label: 22 - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 21 - 2.2.2.2 24003 - 3.3.3.3 imp-null -4.4.4.4/32 - Local binding: label: imp-null - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 22 - 2.2.2.2 24001 - 3.3.3.3 22 -10.0.1.0/24 - Local binding: label: 23 - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 imp-null - 2.2.2.2 imp-null - 3.3.3.3 23 -10.0.2.0/24 - Local binding: label: 24 - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 20 - 2.2.2.2 imp-null - 3.3.3.3 imp-null -10.0.3.0/24 - Local binding: label: imp-null - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 19 - 2.2.2.2 imp-null - 3.3.3.3 24 -10.0.4.0/24 - Local binding: label: imp-null - Remote bindings: - Peer Label - ----------------- --------- - 1.1.1.1 23 - 2.2.2.2 24002 - 3.3.3.3 imp-null -2:2:2::2/128 - Local binding: label: 18 - Remote bindings: - Peer Label - ----------------- --------- - 2.2.2.2 imp-null - 3.3.3.3 18 -3:3:3::3/128 - Local binding: label: 19 - Remote bindings: - Peer Label - ----------------- --------- - 2.2.2.2 24007 -4:4:4::4/128 - Local binding: label: imp-null - Remote bindings: - Peer Label - ----------------- --------- - 2.2.2.2 24006 - 3.3.3.3 19 -2001:db8:2::/64 - Local binding: label: - - Remote bindings: - Peer Label - ----------------- --------- - 2.2.2.2 imp-null - 3.3.3.3 imp-null -2001:db8:3::/64 - Local binding: label: imp-null - Remote bindings: - Peer Label - ----------------- --------- - 2.2.2.2 imp-null -2001:db8:4::/64 - Local binding: label: imp-null - Remote bindings: - Peer Label - ----------------- --------- - 3.3.3.3 imp-null -``` - -Check if the pseudowires are up: -``` -linux# show l2vpn atom vc -Interface Peer ID VC ID Name Status ---------- --------------- ---------- ---------------- ---------- -mpw1 3.3.3.3 100 ENG UP -mpw0 1.1.1.1 100 ENG UP -``` - -Check the label bindings of the pseudowires: -``` -linux# show l2vpn atom binding - Destination Address: 1.1.1.1, VC ID: 100 - Local Label: 25 - Cbit: 1, VC Type: Ethernet, GroupID: 0 - MTU: 1500 - Remote Label: 16 - Cbit: 1, VC Type: Ethernet, GroupID: 0 - MTU: 1500 - Destination Address: 3.3.3.3, VC ID: 100 - Local Label: 26 - Cbit: 1, VC Type: Ethernet, GroupID: 0 - MTU: 1500 - Remote Label: 26 - Cbit: 1, VC Type: Ethernet, GroupID: 0 - MTU: 1500 -``` - -## Verification - Data Plane - -Verify that all the exchanged label mappings were installed in zebra: -``` -linux# show mpls table - Inbound Outbound - Label Type Nexthop Label --------- ------- --------------- -------- - 17 LDP 2001:db8:3::2 3 - 19 LDP 2001:db8:3::2 24005 - 20 LDP 10.0.3.2 24000 - 21 LDP 10.0.3.2 3 - 22 LDP 10.0.3.2 24001 - 23 LDP 10.0.3.2 3 - 24 LDP 10.0.3.2 3 - 25 LDP 10.0.3.2 3 - -linux# show ip route ldp -Codes: K - kernel route, C - connected, S - static, R - RIP, - O - OSPF, I - IS-IS, B - BGP, P - PIM, A - Babel, L - LDP, - > - selected route, * - FIB route - -L>* 1.1.1.1/32 [0/0] via 10.0.3.2, eth2 label 24000 -L>* 3.3.3.3/32 [0/0] via 10.0.3.2, eth2 label 24001 -``` - -Verify that all the exchanged label mappings were installed in the kernel: -``` -$ ip -M ro -17 via inet6 2001:db8:3::2 dev eth2 proto zebra -19 as to 24005 via inet6 2001:db8:3::2 dev eth2 proto zebra -20 as to 24000 via inet 10.0.3.2 dev eth2 proto zebra -21 via inet 10.0.3.2 dev eth2 proto zebra -22 as to 24001 via inet 10.0.3.2 dev eth2 proto zebra -23 via inet 10.0.3.2 dev eth2 proto zebra -24 via inet 10.0.3.2 dev eth2 proto zebra -25 via inet 10.0.3.2 dev eth2 proto zebra -$ -$ ip route | grep mpls -1.1.1.1 encap mpls 24000 via 10.0.3.2 dev eth2 proto zebra metric 20 -3.3.3.3 encap mpls 24001 via 10.0.3.2 dev eth2 proto zebra metric 20 -``` - -Now ping PE1's loopback using lo1's address as a source address: -``` -$ ping -c 5 -I 4.4.4.4 1.1.1.1 -PING 1.1.1.1 (1.1.1.1) from 4.4.4.4 : 56(84) bytes of data. -64 bytes from 1.1.1.1: icmp_seq=1 ttl=253 time=3.02 ms -64 bytes from 1.1.1.1: icmp_seq=2 ttl=253 time=3.13 ms -64 bytes from 1.1.1.1: icmp_seq=3 ttl=253 time=3.19 ms -64 bytes from 1.1.1.1: icmp_seq=4 ttl=253 time=3.07 ms -64 bytes from 1.1.1.1: icmp_seq=5 ttl=253 time=3.27 ms - ---- 1.1.1.1 ping statistics --- -5 packets transmitted, 5 received, 0% packet loss, time 4005ms -rtt min/avg/max/mdev = 3.022/3.140/3.278/0.096 ms -``` - -Verify that the ICMP echo request packets are leaving with the MPLS -label advertised by the P router. Also, verify that the ICMP echo reply -packets are arriving with an explicit-null MPLS label: -``` -# tcpdump -n -i eth2 mpls and icmp -tcpdump: verbose output suppressed, use -v or -vv for full protocol decode -listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes -10:01:40.758771 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 1, length 64 -10:01:40.761777 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 1, length 64 -10:01:41.760343 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 2, length 64 -10:01:41.763448 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 2, length 64 -10:01:42.761758 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 3, length 64 -10:01:42.764924 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 3, length 64 -10:01:43.763193 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 4, length 64 -10:01:43.766237 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 4, length 64 -10:01:44.764552 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 5, length 64 -10:01:44.767803 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 5, length 64 -``` diff --git a/doc/developer/next-hop-tracking.rst b/doc/developer/next-hop-tracking.rst new file mode 100644 index 0000000000..1d65956f8c --- /dev/null +++ b/doc/developer/next-hop-tracking.rst @@ -0,0 +1,352 @@ +Next Hop Tracking +================== + +Next hop tracking is an optimization feature that reduces the processing time +involved in the BGP bestpath algorithm by monitoring changes to the routing +table. + +Background +----------- + +Recursive routes are of the form: + +:: + + p/m --> n + [Ex: 1.1.0.0/16 --> 2.2.2.2] + +where 'n' itself is resolved through another route as follows: + +:: + + p2/m --> h, interface + [Ex: 2.2.2.0/24 --> 3.3.3.3, eth0] + +Usually, BGP routes are recursive in nature and BGP nexthops get resolved +through an IGP route. IGP usually adds its routes pointing to an interface +(these are called non-recursive routes). + +When BGP receives a recursive route from a peer, it needs to validate the +nexthop. The path is marked valid or invalid based on the reachability status +of the nexthop. Nexthop validation is also important for BGP decision process +as the metric to reach the nexthop is a parameter to best path selection +process. + +As it goes with routing, this is a dynamic process. Route to the nexthop can +change. The nexthop can become unreachable or reachable. In the current BGP +implementation, the nexthop validation is done periodically in the scanner run. +The default scanner run interval is one minute. Every minute, the scanner task +walks the entire BGP table. It checks the validity of each nexthop with Zebra +(the routing table manager) through a request and response message exchange +between BGP and Zebra process. BGP process is blocked for that duration. The +mechanism has two major drawbacks: + +- The scanner task runs to completion. That can potentially starve the other + tasks for long periods of time, based on the BGP table size and number of + nexthops. + +- Convergence around routing changes that affect the nexthops can be long + (around a minute with the default intervals). The interval can be shortened + to achieve faster reaction time, but it makes the first problem worse, with + the scanner task consuming most of the CPU resources. + +The next-hop tracking feature makes this process event-driven. It eliminates +periodic nexthop validation and introduces an asynchronous communication path +between BGP and Zebra for route change notifications that can then be acted +upon. + +Goal +---- + +Stating the obvious, the main goal is to remove the two limitations we +discussed in the previous section. The goals, in a constructive tone, +are the following: + +- **Fairness**: the scanner run should not consume an unjustly high amount of + CPU time. This should give an overall good performance and response time to + other events (route changes, session events, IO/user interface). + +- **Convergence**: BGP must react to nexthop changes instantly and provide + sub-second convergence. This may involve diverting the routes from one + nexthop to another. + +Overview of changes +------------------------ + +The changes are in both BGP and Zebra modules. The short summary is +the following: + +- Zebra implements a registration mechanism by which clients can + register for next hop notification. Consequently, it maintains a + separate table, per (VRF, AF) pair, of next hops and interested + client-list per next hop. + +- When the main routing table changes in Zebra, it evaluates the next + hop table: for each next hop, it checks if the route table + modifications have changed its state. If so, it notifies the + interested clients. + +- BGP is one such client. It registers the next hops corresponding to + all of its received routes/paths. It also threads the paths against + each nexthop structure. + +- When BGP receives a next hop notification from Zebra, it walks the + corresponding path list. It makes them valid or invalid depending + on the next hop notification. It then re-computes best path for the + corresponding destination. This may result in re-announcing those + destinations to peers. + +Design +------ + +Modules +~~~~~~~ + +The core design introduces an "nht" (next hop tracking) module in BGP +and "rnh" (recursive nexthop) module in Zebra. The "nht" module +provides the following APIs: + ++----------------------------+--------------------------------------------------+ +| Function | Action | ++============================+==================================================+ +| bgp_find_or_add_nexthop() | find or add a nexthop in BGP nexthop table | ++----------------------------+--------------------------------------------------+ +| bgp_find_nexthop() | find a nexthop in BGP nexthop table | ++----------------------------+--------------------------------------------------+ +| bgp_parse_nexthop_update() | parse a nexthop update message coming from zebra | ++----------------------------+--------------------------------------------------+ + +The "rnh" module provides the following APIs: + ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| Function | Action | ++============================+==========================================================================================================+ +| zebra_add_rnh() | add a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_delete_rnh() | delete a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_lookup_rnh() | lookup a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_add_rnh_client() | register a client for nexthop notifications against a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_remove_rnh_client() | remove the client registration for a recursive nexthop | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_evaluate_rnh_table() | (re)evaluate the recursive nexthop table (most probably because the main routing table has changed). | ++----------------------------+----------------------------------------------------------------------------------------------------------+ +| zebra_cleanup_rnh_client() | Cleanup a client from the "rnh" module data structures (most probably because the client is going away). | ++----------------------------+----------------------------------------------------------------------------------------------------------+ + +4.2. Control flow + +The next hop registration control flow is the following: + +:: + + <==== BGP Process ====>|<==== Zebra Process ====> + | + receive module nht module | zserv module rnh module + ---------------------------------------------------------------------- + | | | + bgp_update_ | | | + main() | bgp_find_or_add_ | | + | nexthop() | | + | | | + | | zserv_nexthop_ | + | | register() | + | | | zebra_add_rnh() + | | | + + +The next hop notification control flow is the following: + +:: + + <==== Zebra Process ====>|<==== BGP Process ====> + | + rib module rnh module | zebra module nht module + ---------------------------------------------------------------------- + | | | + meta_queue_ | | | + process() | zebra_evaluate_ | | + | rnh_table() | | + | | | + | | bgp_read_nexthop_ | + | | update() | + | | | bgp_parse_ + | | | nexthop_update() + | | | + + +zclient message format +~~~~~~~~~~~~~~~~~~~~~~ + +ZEBRA_NEXTHOP_REGISTER and ZEBRA_NEXTHOP_UNREGISTER messages are +encoded in the following way: + +:: + + . 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | AF | prefix len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . Nexthop prefix . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | AF | prefix len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . Nexthop prefix . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +``ZEBRA_NEXTHOP_UPDATE`` message is encoded as follows: + +:: + + . 0 1 2 3 + 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | AF | prefix len | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . Nexthop prefix getting resolved . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | metric | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | #nexthops | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | nexthop type | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . resolving Nexthop details . + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + | nexthop type | + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + . resolving Nexthop details . + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ + + +BGP data structure +~~~~~~~~~~~~~~~~~~ +Legend: + +:: + + /\ struct bgp_node: a BGP destination/route/prefix + \/ + + [ ] struct bgp_info: a BGP path (e.g. route received from a peer) + + _ + (_) struct bgp_nexthop_cache: a BGP nexthop + + /\ NULL + \/--+ ^ + | : + +--[ ]--[ ]--[ ]--> NULL + /\ : + \/--+ : + | : + +--[ ]--[ ]--> NULL + : + _ : + (_)........... + + +Zebra data structure +~~~~~~~~~~~~~~~~~~~~ + +RNH table:: + + O + / \ + O O + / \ + O O + + struct rnh + { + u_char flags; + struct route_entry *state; + struct list *client_list; + struct route_node *node; + }; + +User interface changes +~~~~~~~~~~~~~~~~~~~~~~ + +:: + + frr# show ip nht + 3.3.3.3 + resolved via kernel + via 11.0.0.6, swp1 + Client list: bgp(fd 12) + 11.0.0.10 + resolved via connected + is directly connected, swp2 + Client list: bgp(fd 12) + 11.0.0.18 + resolved via connected + is directly connected, swp4 + Client list: bgp(fd 12) + 11.11.11.11 + resolved via kernel + via 10.0.1.2, eth0 + Client list: bgp(fd 12) + + frr# show ip bgp nexthop + Current BGP nexthop cache: + 3.3.3.3 valid [IGP metric 0], #paths 3 + Last update: Wed Oct 16 04:43:49 2013 + + 11.0.0.10 valid [IGP metric 1], #paths 1 + Last update: Wed Oct 16 04:43:51 2013 + + 11.0.0.18 valid [IGP metric 1], #paths 2 + Last update: Wed Oct 16 04:43:47 2013 + + 11.11.11.11 valid [IGP metric 0], #paths 1 + Last update: Wed Oct 16 04:43:47 2013 + + frr# show ipv6 nht + frr# show ip bgp nexthop detail + + frr# debug bgp nht + frr# debug zebra nht + + 6. Sample test cases + + r2----r3 + / \ / + r1----r4 + + - Verify that a change in IGP cost triggers NHT + + shutdown the r1-r4 and r2-r4 links + + no shut the r1-r4 and r2-r4 links and wait for OSPF to come back + up + + We should be back to the original nexthop via r4 now + - Verify that a NH becoming unreachable triggers NHT + + Shutdown all links to r4 + - Verify that a NH becoming reachable triggers NHT + + no shut all links to r4 + +Future work +~~~~~~~~~~~ + +- route-policy for next hop validation (e.g. ignore default route) +- damping for rapid next hop changes +- prioritized handling of nexthop changes ((un)reachability vs. metric + changes) +- handling recursion loop, e.g:: + + 11.11.11.11/32 -> 12.12.12.12 + 12.12.12.12/32 -> 11.11.11.11 + 11.0.0.0/8 -> +- better statistics diff --git a/doc/developer/next-hop-tracking.txt b/doc/developer/next-hop-tracking.txt deleted file mode 100644 index 12ed63947b..0000000000 --- a/doc/developer/next-hop-tracking.txt +++ /dev/null @@ -1,326 +0,0 @@ -0. Introduction - -This is the design specification for next hop tracking feature in -Frr. - -1. Background - -Recursive routes are of the form: - - p/m --> n - [Ex: 1.1.0.0/16 --> 2.2.2.2] - -where 'n' itself is resolved through another route as follows: - - p2/m --> h, interface - [Ex: 2.2.2.0/24 --> 3.3.3.3, eth0] - -Usually, BGP routes are recursive in nature and BGP nexthops get -resolved through an IGP route. IGP usually adds its routes pointing to -an interface (these are called non-recursive routes). - -When BGP receives a recursive route from a peer, it needs to validate -the nexthop. The path is marked valid or invalid based on the -reachability status of the nexthop. Nexthop validation is also -important for BGP decision process as the metric to reach the nexthop -is a parameter to best path selection process. - -As it goes with routing, this is a dynamic process. Route to the -nexthop can change. The nexthop can become unreachable or -reachable. In the current BGP implementation, the nexthop validation -is done periodically in the scanner run. The default scanner run -interval is one minute. Every minute, the scanner task walks the -entire BGP table. It checks the validity of each nexthop with Zebra -(the routing table manager) through a request and response message -exchange between BGP and Zebra process. BGP process is blocked for -that duration. The mechanism has two major drawbacks: - -(1) The scanner task runs to completion. That can potentially starve - the other tasks for long periods of time, based on the BGP table - size and number of nexthops. - -(2) Convergence around routing changes that affect the nexthops can be - long (around a minute with the default intervals). The interval - can be shortened to achieve faster reaction time, but it makes the - first problem worse, with the scanner task consuming most of the - CPU resources. - -"Next hop tracking" feature makes this process event-driven. It -eliminates periodic nexthop validation and introduces an asynchronous -communication path between BGP and Zebra for route change notifications -that can then be acted upon. - -2. Goal - -Stating the obvious, the main goal is to remove the two limitations we -discussed in the previous section. The goals, in a constructive tone, -are the following: - -- fairness: the scanner run should not consume an unjustly high amount - of CPU time. This should give an overall good performance and - response time to other events (route changes, session events, - IO/user interface). - -- convergence: BGP must react to nexthop changes instantly and provide - sub-second convergence. This may involve diverting the routes from - one nexthop to another. - -3. Overview of the changes - -The changes are in both BGP and Zebra modules. The short summary is -the following: - -- Zebra implements a registration mechanism by which clients can - register for next hop notification. Consequently, it maintains a - separate table, per (VRF, AF) pair, of next hops and interested - client-list per next hop. - -- When the main routing table changes in Zebra, it evaluates the next - hop table: for each next hop, it checks if the route table - modifications have changed its state. If so, it notifies the - interested clients. - -- BGP is one such client. It registers the next hops corresponding to - all of its received routes/paths. It also threads the paths against - each nexthop structure. - -- When BGP receives a next hop notification from Zebra, it walks the - corresponding path list. It makes them valid or invalid depending - on the next hop notification. It then re-computes best path for the - corresponding destination. This may result in re-announcing those - destinations to peers. - -4. Design - -4.1. Modules - -The core design introduces an "nht" (next hop tracking) module in BGP -and "rnh" (recursive nexthop) module in Zebra. The "nht" module -provides the following APIs: - -bgp_find_or_add_nexthop() : find or add a nexthop in BGP nexthop table -bgp_find_nexthop() : find a nexthop in BGP nexthop table -bgp_parse_nexthop_update() : parse a nexthop update message coming - from zebra - -The "rnh" module provides the following APIs: - -zebra_add_rnh() : add a recursive nexthop -zebra_delete_rnh() : delete a recursive nexthop -zebra_lookup_rnh() : lookup a recursive nexthop - -zebra_add_rnh_client() : register a client for nexthop notifications - against a recursive nexthop - -zebra_remove_rnh_client(): remove the client registration for a - recursive nexthop - -zebra_evaluate_rnh_table(): (re)evaluate the recursive nexthop table - (most probably because the main routing - table has changed). - -zebra_cleanup_rnh_client(): Cleanup a client from the "rnh" module - data structures (most probably because the - client is going away). - -4.2. Control flow - -The next hop registration control flow is the following: - -<==== BGP Process ====>|<==== Zebra Process ====> - | -receive module nht module | zserv module rnh module ----------------------------------------------------------------------- - | | | -bgp_update_ | | | - main() | bgp_find_or_add_ | | - | nexthop() | | - | | | - | | zserv_nexthop_ | - | | register() | - | | | zebra_add_rnh() - | | | - - -The next hop notification control flow is the following: - -<==== Zebra Process ====>|<==== BGP Process ====> - | -rib module rnh module | zebra module nht module ----------------------------------------------------------------------- - | | | -meta_queue_ | | | - process() | zebra_evaluate_ | | - | rnh_table() | | - | | | - | | bgp_read_nexthop_ | - | | update() | - | | | bgp_parse_ - | | | nexthop_update() - | | | - - -4.3. zclient message format - -ZEBRA_NEXTHOP_REGISTER and ZEBRA_NEXTHOP_UNREGISTER messages are -encoded in the following way: - -/* - * 0 1 2 3 - * 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | AF | prefix len | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . Nexthop prefix . - * . . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . . - * . . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | AF | prefix len | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . Nexthop prefix . - * . . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - */ - -ZEBRA_NEXTHOP_UPDATE message is encoded as follows: - -/* - * 0 1 2 3 - * 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | AF | prefix len | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . Nexthop prefix getting resolved . - * . . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | metric | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | #nexthops | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | nexthop type | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . resolving Nexthop details . - * . . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * | nexthop type | - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - * . resolving Nexthop details . - * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ - */ - -4.4. BGP data structure - -Legend: - -/\ struct bgp_node: a BGP destination/route/prefix -\/ - -[ ] struct bgp_info: a BGP path (e.g. route received from a peer) - - _ -(_) struct bgp_nexthop_cache: a BGP nexthop - - - - /\ NULL - \/--+ ^ - | : - +--[ ]--[ ]--[ ]--> NULL - /\ : - \/--+ : - | : - +--[ ]--[ ]--> NULL - : - _ : - (_)............. - - -4.5. Zebra data structure - -rnh table: - - O - / \ - O O - / \ - O O - - struct rnh - { - u_char flags; - struct route_entry *state; - struct list *client_list; - struct route_node *node; - }; - -5. User interface changes - -frr# show ip nht -3.3.3.3 - resolved via kernel - via 11.0.0.6, swp1 - Client list: bgp(fd 12) -11.0.0.10 - resolved via connected - is directly connected, swp2 - Client list: bgp(fd 12) -11.0.0.18 - resolved via connected - is directly connected, swp4 - Client list: bgp(fd 12) -11.11.11.11 - resolved via kernel - via 10.0.1.2, eth0 - Client list: bgp(fd 12) - -frr# show ip bgp nexthop -Current BGP nexthop cache: - 3.3.3.3 valid [IGP metric 0], #paths 3 - Last update: Wed Oct 16 04:43:49 2013 - - 11.0.0.10 valid [IGP metric 1], #paths 1 - Last update: Wed Oct 16 04:43:51 2013 - - 11.0.0.18 valid [IGP metric 1], #paths 2 - Last update: Wed Oct 16 04:43:47 2013 - - 11.11.11.11 valid [IGP metric 0], #paths 1 - Last update: Wed Oct 16 04:43:47 2013 - -frr# show ipv6 nht -frr# show ip bgp nexthop detail - -frr# debug bgp nht -frr# debug zebra nht - -6. Sample test cases - - r2----r3 - / \ / - r1----r4 - -- Verify that a change in IGP cost triggers NHT - + shutdown the r1-r4 and r2-r4 links - + no shut the r1-r4 and r2-r4 links and wait for OSPF to come back - up - + We should be back to the original nexthop via r4 now -- Verify that a NH becoming unreachable triggers NHT - + Shutdown all links to r4 -- Verify that a NH becoming reachable triggers NHT - + no shut all links to r4 - -7. Future work - -- route-policy for next hop validation (e.g. ignore default route) -- damping for rapid next hop changes -- prioritized handling of nexthop changes ((un)reachability vs. metric - changes) -- handling recursion loop, e.g. - 11.11.11.11/32 -> 12.12.12.12 - 12.12.12.12/32 -> 11.11.11.11 - 11.0.0.0/8 -> -- better statistics diff --git a/doc/developer/workflow.rst b/doc/developer/workflow.rst index 38cf0b878d..4ab70ac3c5 100644 --- a/doc/developer/workflow.rst +++ b/doc/developer/workflow.rst @@ -1,67 +1,52 @@ -Developing for FRRouting +Process & Workflow ======================== -General note on this document ------------------------------ - -This document is "descriptive/post-factual" in that it documents -pratices that are in use; it is not "definitive/pre-factual" in -prescribing practices. +FRR is a large project developed by many different groups. This section +documents standards for code style & quality, commit messages, pull requests +and best practices that all contributors are asked to follow. -This means that when a procedure changes, it is agreed upon, then put -into practice, and then documented here. If this document doesn't match -reality, it's the document that needs to be updated, not reality. +This section is "descriptive/post-factual" in that it documents pratices that +are in use; it is not "definitive/pre-factual" in prescribing practices. This +means that when a procedure changes, it is agreed upon, then put into practice, +and then documented here. If this document doesn't match reality, it's the +document that needs to be updated, not reality. Git Structure ------------- -The master Git for FRRouting resides on Github at -`https://github.com/frrouting/frr `__ - -.. figure:: git_branches.svg - :alt: git branches continually merging to the left from 3 lanes; - float-right - - git branches continually merging to the left from 3 lanes; - float-right +The master Git for FRR resides on `Github +`__. -There is one main branch for development and a release branch for each -major release. +.. figure:: git_branches.png -New contributions are done against the head of the master branch. The CI -systems will pick up the Github Pull Requests or the new patch from -Patchwork, run some basic build and functional tests. - -For each major release (1.0, 1.1 etc) a new release branch is created -based on the master. - -There was an attempt to use a "develop" branch automatically maintained -by the CI system. This is not currently in active use, though the system -is operational. If the "develop" branch is in active use and this -paragraph is still here, this document obviously wasn't updated. +There is one main branch for development, ``master``. For each major release +(2.0, 3.0 etc) a new release branch is created based on the master. Subsequent +point releases based on a major branch are marked by tagging. Programming language, Tools and Libraries ----------------------------------------- -The core of FRRouting is written in C (gcc or clang supported) and makes +The core of FRR is written in C (gcc or clang supported) and makes use of GNU compiler extensions. A few non-essential scripts are -implemented in Perl and Python. FRRouting requires the following tools +implemented in Perl and Python. FRR requires the following tools to build distribution packages: automake, autoconf, texinfo, libtool and gawk and various libraries (i.e. libpam and libjson-c). If your contribution requires a new library or other tool, then please highlight this in your description of the change. Also make sure it’s -supported by all FRRouting platform OSes or provide a way to build +supported by all FRR platform OSes or provide a way to build without the library (potentially without the new feature) on the other platforms. -Documentation should be written in Tex (.texi) or Markdown (.md) format -with a preference for Markdown. +Documentation should be written in reStructuredText. Sphinx extensions may be +utilized but pure ReST is preferred where possible. See `Documentation +<#documentation>`__. Mailing lists ------------- -Italicized lists are private. +The FRR development group maintains multiple mailing lists for use by the +community. Italicized lists are private. +----------------------------------+--------------------------------+ | Topic | List | @@ -80,19 +65,31 @@ Italicized lists are private. Changelog ~~~~~~~~~ -The changelog will be the base for the release notes. A changelog entry -for your changes is usually not required and will be added based on your -commit messages by the maintainers. However, you are free to include an -update to the changelog with some better description. The changelog will -be the base for the release notes. +The changelog will be the base for the release notes. A changelog entry for +your changes is usually not required and will be added based on your commit +messages by the maintainers. However, you are free to include an update to the +changelog with some better description. Submitting Patches and Enhancements ----------------------------------- +FRR accepts patches from two sources: + +- Email (git format-patch) +- Github pull request + +Contributors are highly encouraged to use Github's fork-and-pr workflow. It is +easier for us to review it, test it, try it and discuss it on Github than it is +via email, thus your patch will get more attention more quickly on Github. + +The base branch for new contributions and non-critical bug fixes should be +``master``. Please ensure your pull request is based on this branch when you +submit it. + Pre-submission Checklist ~~~~~~~~~~~~~~~~~~~~~~~~ -- Format code (see `Developer's Guidelines <#developers-guidelines>`__) +- Format code (see `Code Formatting <#developers-guidelines>`__) - Verify and acknowledge license (see `License for contributions <#license-for-contributions>`__) - Ensure you have properly signed off (see `Signing @@ -115,14 +112,14 @@ Pre-submission Checklist License for contributions ~~~~~~~~~~~~~~~~~~~~~~~~~ -FRRouting is under a “GPLv2 or later” license. Any code submitted must +FRR is under a “GPLv2 or later” license. Any code submitted must be released under the same license (preferred) or any license which allows redistribution under this GPLv2 license (eg MIT License). Signing Off ~~~~~~~~~~~ -Code submitted to FRRouting must be signed off. We have the same +Code submitted to FRR must be signed off. We have the same requirements for using the signed-off-by process as the Linux kernel. In short, you must include a signed-off-by tag in every patch. @@ -142,6 +139,8 @@ to be a helpful resource. In short, when you sign off on a commit, you assert your agreement to all of the following: +:: + Developer's Certificate of Origin 1.1 By making a contribution to this project, I certify that: @@ -171,7 +170,7 @@ What do I submit my changes against? We've documented where we would like to have the different fixes applied at -https://github.com/FRRouting/frr/wiki/Where-Do-I-create-a-Pull-Request-against%3F +https://github.com/FRR/frr/wiki/Where-Do-I-create-a-Pull-Request-against%3F If you are unsure where your submission goes, look at that document or ask a project maintainer. @@ -219,12 +218,9 @@ After submitting your changes - You should automatically receive an email with the test results within less than 2 hrs of the submission. If you don’t get the - email, then check status on the github pull request (if submitted - by pull request) or on Patchwork at - https://patchwork.frrouting.org (if submitted as patch to mailing - list). + email, then check status on the Github pull request. - Please notify the development mailing list if you think something - doesn’t work. + doesn't work. - If the tests failed: @@ -253,8 +249,8 @@ After submitting your changes community members. - Your submission is done once it is merged to the master branch. -Developer's Guidelines ----------------------- +Coding Practices & Style +------------------------ Commit messages ~~~~~~~~~~~~~~~ @@ -392,13 +388,12 @@ BSD coding style applies to: Documentation ~~~~~~~~~~~~~ -FRRouting is a large and complex software project developed by many -different people over a long period of time. Without adequate -documentation, it can be exceedingly difficult to understand code -segments, APIs and other interfaces. In the interest of keeping the -project healthy and maintainable, you should make every effort to -document your code so that other people can understand what it does -without needing to closely read the code itself. +FRR is a large and complex software project developed by many different people +over a long period of time. Without adequate documentation, it can be +exceedingly difficult to understand code segments, APIs and other interfaces. +In the interest of keeping the project healthy and maintainable, you should +make every effort to document your code so that other people can understand +what it does without needing to closely read the code itself. Some specific guidelines that contributors should follow are: @@ -440,10 +435,13 @@ used for. - **For new code in ``lib/``, these guidelines are hard requirements.** If you are contributing code that adds significant user-visible -functionality or introduces a new API, please document it in ``doc/``. -Markdown and LaTeX are acceptable formats, although Markdown is -currently preferred for new documentation. This may change in the near -future. +functionality please document it in ``doc/``. If you make significant changes +to portions of the codebase covered in the Developer's Manual, please +update the relevant sections. If you add a major feature or introduce a new +API, please document the architecture and API to the best of your abilities in +the Developer's Manual. + +Documentation should be in reStructuredText. Finally, if you come across some code that is undocumented and feel like going above and beyond, document it! We absolutely appreciate and accept diff --git a/doc/ldpd-basic-test-setup.md b/doc/ldpd-basic-test-setup.md new file mode 100644 index 0000000000..b25a2b6d4b --- /dev/null +++ b/doc/ldpd-basic-test-setup.md @@ -0,0 +1,681 @@ +## Topology + +The goal of this test is to verify that the all the basic functionality +of ldpd is working as expected, be it running on Linux or OpenBSD. In +addition to that, more advanced features are also tested, like LDP +sessions over IPv6, MD5 authentication and pseudowire signaling. + +In the topology below there are 3 PE routers, 3 CE routers and one P +router (not attached to any consumer site). + +All routers have IPv4 addresses and OSPF is used as the IGP. The +three routers from the bottom of the picture, P, PE2 and PE3, are also +configured for IPv6 (dual-stack) and static IPv6 routes are used to +provide connectivity among them. + +The three CEs share the same VPLS membership. LDP is used to set up the +LSPs among the PEs and to signal the pseudowires. MD5 authentication is +used to protect all LDP sessions. + +``` + CE1 172.16.1.1/24 + + + | + +---+---+ + | PE1 | + | IOS XE| + | | + +---+---+ + | + | 10.0.1.0/24 + | + +---+---+ + | P | + +------+ IOS XR+------+ + | | | | + | +-------+ | + 10.0.2.0/24 | | 10.0.3.0/24 +2001:db8:2::/64 | | 2001:db8:3::/64 + | | + +---+---+ +---+---+ + | PE2 | | PE3 | + |OpenBSD+-------------+ Linux | + | | | | + +---+---+ 10.0.4.0/24 +---+---+ + | 2001:db8:4::/64 | + + + + 172.16.1.2/24 CE2 CE3 172.16.1.3/24 +``` + +## Configuration + +#### Linux +1 - Enable IPv4/v6 forwarding: +``` +# sysctl -w net.ipv4.ip_forward=1 +# sysctl -w net.ipv6.conf.all.forwarding=1 +``` + +2 - Enable MPLS forwarding: +``` +# modprobe mpls-router +# modprobe mpls-iptunnel +# echo 100000 > /proc/sys/net/mpls/platform_labels +# echo 1 > /proc/sys/net/mpls/conf/eth1/input +# echo 1 > /proc/sys/net/mpls/conf/eth2/input +``` + +3 - Set up the interfaces: +``` +# ip link add name lo1 type dummy +# ip link set dev lo1 up +# ip addr add 4.4.4.4/32 dev lo1 +# ip -6 addr add 4:4:4::4/128 dev lo1 +# ip link set dev eth1 up +# ip addr add 10.0.4.4/24 dev eth1 +# ip -6 addr add 2001:db8:4::4/64 dev eth1 +# ip link set dev eth2 up +# ip addr add 10.0.3.4/24 dev eth2 +# ip -6 addr add 2001:db8:3::4/64 dev eth2 +``` + +4 - Set up the bridge and pseudowire interfaces: +``` +# ip link add type bridge +# ip link set dev bridge0 up +# ip link set dev eth0 up +# ip link set dev eth0 master bridge0 +# ip link add name mpw0 type dummy +# ip link set dev mpw0 up +# ip link set dev mpw0 master bridge0 +# ip link add name mpw1 type dummy +# ip link set dev mpw1 up +# ip link set dev mpw1 master bridge0 +``` + +> NOTE: MPLS support in the Linux kernel is very recent and it still +doesn't support pseudowire interfaces. We are using here dummy interfaces +just to show how the VPLS configuration should look like in the future. + +5 - Add static IPv6 routes for the remote loopbacks: +``` +# ip -6 route add 2:2:2::2/128 via 2001:db8:3::2 +# ip -6 route add 3:3:3::3/128 via 2001:db8:4::3 +``` + +6 - Edit /etc/frr/ospfd.conf: +``` +router ospf + network 4.4.4.4/32 area 0.0.0.0 + network 10.0.3.4/24 area 0.0.0.0 + network 10.0.4.4/24 area 0.0.0.0 +! +``` + +7 - Edit /etc/frr/ldpd.conf: +``` +debug mpls ldp messages recv +debug mpls ldp messages sent +debug mpls ldp zebra +! +mpls ldp + router-id 4.4.4.4 + dual-stack cisco-interop + neighbor 1.1.1.1 password opensourcerouting + neighbor 2.2.2.2 password opensourcerouting + neighbor 3.3.3.3 password opensourcerouting + ! + address-family ipv4 + discovery transport-address 4.4.4.4 + label local advertise explicit-null + ! + interface eth2 + ! + interface eth1 + ! + ! + address-family ipv6 + discovery transport-address 4:4:4::4 + ttl-security disable + ! + interface eth2 + ! + interface eth1 + ! + ! +! +l2vpn ENG type vpls + bridge br0 + member interface eth0 + ! + member pseudowire mpw0 + neighbor lsr-id 1.1.1.1 + pw-id 100 + ! + member pseudowire mpw1 + neighbor lsr-id 3.3.3.3 + neighbor address 3:3:3::3 + pw-id 100 + ! +! +``` + +> NOTE: We have to disable ttl-security under the ipv6 address-family +in order to interoperate with the IOS-XR router. GTSM is mandatory for +LDPv6 but the IOS-XR implementation is not RFC compliant in this regard. + +8 - Run zebra, ospfd and ldpd. + +#### OpenBSD +1 - Enable IPv4/v6 forwarding: +``` +# sysctl net.inet.ip.forwarding=1 +# sysctl net.inet6.ip6.forwarding=1 +``` + +2 - Enable MPLS forwarding: +``` +# ifconfig em2 10.0.2.3/24 mpls +# ifconfig em3 10.0.4.3/24 mpls +``` + +3 - Set up the interfaces: +``` +# ifconfig lo1 alias 3.3.3.3 netmask 255.255.255.255 +# ifconfig lo1 inet6 3:3:3::3/128 +# ifconfig em2 inet6 2001:db8:2::3/64 +# ifconfig em3 inet6 2001:db8:4::3/64 +``` + +4 - Set up the bridge and pseudowire interfaces: +``` +# ifconfig bridge0 create +# ifconfig bridge0 up +# ifconfig em1 up +# ifconfig bridge0 add em1 +# ifconfig mpw0 create +# ifconfig mpw0 up +# ifconfig bridge0 add mpw0 +# ifconfig mpw1 create +# ifconfig mpw1 up +# ifconfig bridge0 add mpw1 +``` + +5 - Add static IPv6 routes for the remote loopbacks: +``` +# route -n add 4:4:4::4/128 2001:db8:4::4 +# route -n add 2:2:2::2/128 2001:db8:2::2 +``` + +6 - Edit /etc/frr/ospfd.conf: +``` +router ospf + network 10.0.2.3/24 area 0 + network 10.0.4.3/24 area 0 + network 3.3.3.3/32 area 0 +! +``` + +7 - Edit /etc/frr/ldpd.conf: +``` +debug mpls ldp messages recv +debug mpls ldp messages sent +debug mpls ldp zebra +! +mpls ldp + router-id 3.3.3.3 + dual-stack cisco-interop + neighbor 1.1.1.1 password opensourcerouting + neighbor 2.2.2.2 password opensourcerouting + neighbor 4.4.4.4 password opensourcerouting + ! + address-family ipv4 + discovery transport-address 3.3.3.3 + label local advertise explicit-null + ! + interface em3 + ! + interface em2 + ! + ! + address-family ipv6 + discovery transport-address 3:3:3::3 + ttl-security disable + ! + interface em3 + ! + interface em2 + ! + ! +! +l2vpn ENG type vpls + bridge br0 + member interface em1 + ! + member pseudowire mpw0 + neighbor lsr-id 1.1.1.1 + pw-id 100 + ! + member pseudowire mpw1 + neighbor lsr-id 4.4.4.4 + neighbor address 4:4:4::4 + pw-id 100 + ! +! +``` + +8 - Run zebra, ospfd and ldpd. + +#### Cisco routers +CE1 (IOS): +``` +interface FastEthernet0/0 + ip address 172.16.1.1 255.255.255.0 + ! +! +``` + +CE2 (IOS): +``` +interface FastEthernet0/0 + ip address 172.16.1.2 255.255.255.0 + ! +! +``` + +CE3 (IOS): +``` +interface FastEthernet0/0 + ip address 172.16.1.3 255.255.255.0 + ! +! +``` + +PE1 - IOS-XE (1): +``` +mpls ldp neighbor 2.2.2.2 password opensourcerouting +mpls ldp neighbor 3.3.3.3 password opensourcerouting +mpls ldp neighbor 4.4.4.4 password opensourcerouting +! +l2vpn vfi context VFI + vpn id 1 + member pseudowire2 + member pseudowire1 +! +bridge-domain 1 + member GigabitEthernet1 service-instance 1 + member vfi VFI +! +interface Loopback1 + ip address 1.1.1.1 255.255.255.255 +! +interface pseudowire1 + encapsulation mpls + neighbor 3.3.3.3 100 +! +interface pseudowire2 + encapsulation mpls + neighbor 4.4.4.4 100 +! +interface GigabitEthernet3 + ip address 10.0.1.1 255.255.255.0 + mpls ip +! +router ospf 1 + network 0.0.0.0 255.255.255.255 area 0 +! +``` + +P - IOS-XR (2): +``` +interface Loopback1 + ipv4 address 2.2.2.2 255.255.255.255 + ipv6 address 2:2:2::2/128 +! +interface GigabitEthernet0/0/0/0 + ipv4 address 10.0.1.2 255.255.255.0 +! +interface GigabitEthernet0/0/0/1 + ipv4 address 10.0.2.2 255.255.255.0 + ipv6 address 2001:db8:2::2/64 + ipv6 enable +! +interface GigabitEthernet0/0/0/2 + ipv4 address 10.0.3.2 255.255.255.0 + ipv6 address 2001:db8:3::2/64 + ipv6 enable +! +router static + address-family ipv6 unicast + 3:3:3::3/128 2001:db8:2::3 + 4:4:4::4/128 2001:db8:3::4 + ! +! +router ospf 1 + router-id 2.2.2.2 + address-family ipv4 unicast + area 0 + interface Loopback1 + ! + interface GigabitEthernet0/0/0/0 + ! + interface GigabitEthernet0/0/0/1 + ! + interface GigabitEthernet0/0/0/2 + ! + ! +! +mpls ldp + router-id 2.2.2.2 + neighbor + 1.1.1.1:0 password clear opensourcerouting + 3.3.3.3:0 password clear opensourcerouting + 4.4.4.4:0 password clear opensourcerouting + ! + address-family ipv4 + ! + address-family ipv6 + discovery transport-address 2:2:2::2 + ! + interface GigabitEthernet0/0/0/0 + address-family ipv4 + ! + ! + interface GigabitEthernet0/0/0/1 + address-family ipv4 + ! + address-family ipv6 + ! + ! + interface GigabitEthernet0/0/0/2 + address-family ipv4 + ! + address-family ipv6 + ! + ! +! +``` + +## Verification - Control Plane + +Using the CLI on the Linux box, the goal is to ensure that everything +is working as expected. + +First, verify that all the required adjacencies and neighborships sessions +were established: + +``` +linux# show mpls ldp discovery +Local LDP Identifier: 4.4.4.4:0 +Discovery Sources: + Interfaces: + eth1: xmit/recv + LDP Id: 3.3.3.3:0, Transport address: 3.3.3.3 + Hold time: 15 sec + LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3 + Hold time: 15 sec + eth2: xmit/recv + LDP Id: 2.2.2.2:0, Transport address: 2.2.2.2 + Hold time: 15 sec + LDP Id: 2.2.2.2:0, Transport address: 2:2:2::2 + Hold time: 15 sec + Targeted Hellos: + 4.4.4.4 -> 1.1.1.1: xmit/recv + LDP Id: 1.1.1.1:0, Transport address: 1.1.1.1 + Hold time: 45 sec + 4:4:4::4 -> 3:3:3::3: xmit/recv + LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3 + Hold time: 45 sec + +linux# show mpls ldp neighbor +Peer LDP Identifier: 1.1.1.1:0 + TCP connection: 4.4.4.4:40921 - 1.1.1.1:646 + Session Holdtime: 180 sec + State: OPERATIONAL; Downstream-Unsolicited + Up time: 00:06:02 + LDP Discovery Sources: + IPv4: + Targeted Hello: 1.1.1.1 + +Peer LDP Identifier: 2.2.2.2:0 + TCP connection: 4:4:4::4:52286 - 2:2:2::2:646 + Session Holdtime: 180 sec + State: OPERATIONAL; Downstream-Unsolicited + Up time: 00:06:02 + LDP Discovery Sources: + IPv4: + Interface: eth2 + IPv6: + Interface: eth2 + +Peer LDP Identifier: 3.3.3.3:0 + TCP connection: 4:4:4::4:60575 - 3:3:3::3:646 + Session Holdtime: 180 sec + State: OPERATIONAL; Downstream-Unsolicited + Up time: 00:05:57 + LDP Discovery Sources: + IPv4: + Interface: eth1 + IPv6: + Targeted Hello: 3:3:3::3 + Interface: eth1 +``` + +Note that the neighborships with the P and PE2 routers were established +over IPv6, since this is the default behavior for dual-stack LSRs, as +specified in RFC 7552. If desired, the **dual-stack transport-connection +prefer ipv4** command can be used to establish these sessions over IPv4 +(the command should be applied an all routers). + +Now, verify that there's a remote label for each PE address: +``` +linux# show mpls ldp binding +1.1.1.1/32 + Local binding: label: 20 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 imp-null + 2.2.2.2 24000 + 3.3.3.3 20 +2.2.2.2/32 + Local binding: label: 21 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 18 + 2.2.2.2 imp-null + 3.3.3.3 21 +3.3.3.3/32 + Local binding: label: 22 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 21 + 2.2.2.2 24003 + 3.3.3.3 imp-null +4.4.4.4/32 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 22 + 2.2.2.2 24001 + 3.3.3.3 22 +10.0.1.0/24 + Local binding: label: 23 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 imp-null + 2.2.2.2 imp-null + 3.3.3.3 23 +10.0.2.0/24 + Local binding: label: 24 + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 20 + 2.2.2.2 imp-null + 3.3.3.3 imp-null +10.0.3.0/24 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 19 + 2.2.2.2 imp-null + 3.3.3.3 24 +10.0.4.0/24 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 1.1.1.1 23 + 2.2.2.2 24002 + 3.3.3.3 imp-null +2:2:2::2/128 + Local binding: label: 18 + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 imp-null + 3.3.3.3 18 +3:3:3::3/128 + Local binding: label: 19 + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 24007 +4:4:4::4/128 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 24006 + 3.3.3.3 19 +2001:db8:2::/64 + Local binding: label: - + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 imp-null + 3.3.3.3 imp-null +2001:db8:3::/64 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 2.2.2.2 imp-null +2001:db8:4::/64 + Local binding: label: imp-null + Remote bindings: + Peer Label + ----------------- --------- + 3.3.3.3 imp-null +``` + +Check if the pseudowires are up: +``` +linux# show l2vpn atom vc +Interface Peer ID VC ID Name Status +--------- --------------- ---------- ---------------- ---------- +mpw1 3.3.3.3 100 ENG UP +mpw0 1.1.1.1 100 ENG UP +``` + +Check the label bindings of the pseudowires: +``` +linux# show l2vpn atom binding + Destination Address: 1.1.1.1, VC ID: 100 + Local Label: 25 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 + Remote Label: 16 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 + Destination Address: 3.3.3.3, VC ID: 100 + Local Label: 26 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 + Remote Label: 26 + Cbit: 1, VC Type: Ethernet, GroupID: 0 + MTU: 1500 +``` + +## Verification - Data Plane + +Verify that all the exchanged label mappings were installed in zebra: +``` +linux# show mpls table + Inbound Outbound + Label Type Nexthop Label +-------- ------- --------------- -------- + 17 LDP 2001:db8:3::2 3 + 19 LDP 2001:db8:3::2 24005 + 20 LDP 10.0.3.2 24000 + 21 LDP 10.0.3.2 3 + 22 LDP 10.0.3.2 24001 + 23 LDP 10.0.3.2 3 + 24 LDP 10.0.3.2 3 + 25 LDP 10.0.3.2 3 + +linux# show ip route ldp +Codes: K - kernel route, C - connected, S - static, R - RIP, + O - OSPF, I - IS-IS, B - BGP, P - PIM, A - Babel, L - LDP, + > - selected route, * - FIB route + +L>* 1.1.1.1/32 [0/0] via 10.0.3.2, eth2 label 24000 +L>* 3.3.3.3/32 [0/0] via 10.0.3.2, eth2 label 24001 +``` + +Verify that all the exchanged label mappings were installed in the kernel: +``` +$ ip -M ro +17 via inet6 2001:db8:3::2 dev eth2 proto zebra +19 as to 24005 via inet6 2001:db8:3::2 dev eth2 proto zebra +20 as to 24000 via inet 10.0.3.2 dev eth2 proto zebra +21 via inet 10.0.3.2 dev eth2 proto zebra +22 as to 24001 via inet 10.0.3.2 dev eth2 proto zebra +23 via inet 10.0.3.2 dev eth2 proto zebra +24 via inet 10.0.3.2 dev eth2 proto zebra +25 via inet 10.0.3.2 dev eth2 proto zebra +$ +$ ip route | grep mpls +1.1.1.1 encap mpls 24000 via 10.0.3.2 dev eth2 proto zebra metric 20 +3.3.3.3 encap mpls 24001 via 10.0.3.2 dev eth2 proto zebra metric 20 +``` + +Now ping PE1's loopback using lo1's address as a source address: +``` +$ ping -c 5 -I 4.4.4.4 1.1.1.1 +PING 1.1.1.1 (1.1.1.1) from 4.4.4.4 : 56(84) bytes of data. +64 bytes from 1.1.1.1: icmp_seq=1 ttl=253 time=3.02 ms +64 bytes from 1.1.1.1: icmp_seq=2 ttl=253 time=3.13 ms +64 bytes from 1.1.1.1: icmp_seq=3 ttl=253 time=3.19 ms +64 bytes from 1.1.1.1: icmp_seq=4 ttl=253 time=3.07 ms +64 bytes from 1.1.1.1: icmp_seq=5 ttl=253 time=3.27 ms + +--- 1.1.1.1 ping statistics --- +5 packets transmitted, 5 received, 0% packet loss, time 4005ms +rtt min/avg/max/mdev = 3.022/3.140/3.278/0.096 ms +``` + +Verify that the ICMP echo request packets are leaving with the MPLS +label advertised by the P router. Also, verify that the ICMP echo reply +packets are arriving with an explicit-null MPLS label: +``` +# tcpdump -n -i eth2 mpls and icmp +tcpdump: verbose output suppressed, use -v or -vv for full protocol decode +listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes +10:01:40.758771 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 1, length 64 +10:01:40.761777 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 1, length 64 +10:01:41.760343 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 2, length 64 +10:01:41.763448 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 2, length 64 +10:01:42.761758 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 3, length 64 +10:01:42.764924 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 3, length 64 +10:01:43.763193 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 4, length 64 +10:01:43.766237 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 4, length 64 +10:01:44.764552 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 5, length 64 +10:01:44.767803 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 5, length 64 +```