pim: fix order of vxlan mroutes cleanup when pimd is shutdown
1. vxlan instance cleanup needs to be done before the upstream entries are
force-flushed.
2. also vxlan callbacks need to be ignored post instance-cleanup.
bgpd: lock the tenant-vrf associated with the l2-vni
The l2vni (bgpevpn instance) was maintaining a back pointer to the
tenant vrf without locking it. This would result in bgp_terminate crashing
as the tenant-vrf is released before the underlay-vrf (vpn->bgp_vrf->l2vnis
is NULL). Call stack -
BGP: [bt 3] /lib/libfrr.so.0(listnode_delete+0x11) [0x7f041c967f51]
BGP: [bt 4] /usr/lib/frr/bgpd(bgp_evpn_free+0x26) [0x55e3428eea46]
BGP: [bt 5] /lib/libfrr.so.0(hash_iterate+0x4a) [0x7f041c95f00a]
BGP: [bt 6] /usr/lib/frr/bgpd(bgp_evpn_cleanup+0x22) [0x55e3428f0a72]
BGP: [bt 7] /usr/lib/frr/bgpd(bgp_free+0x180) [0x55e342955f50]
PIM: vxlan SG (*,239.1.1.111) term mroute-up del
BGP: [bt 8] /usr/lib/frr/bgpd(bgp_delete+0x43a) [0x55e342959d7a]
BGP: [bt 9] /usr/lib/frr/bgpd(sigint+0xee) [0x55e3428d6a5e]
This was resulting in static analyzer warnings for subsequent usage
of the same pointer -
pimd/pim_vxlan.c:962:36: warning: Access to field 'info' results in a
dereference of a null pointer (loaded from variable 'ifp')
pim_ifp = (struct pim_interface *)ifp->info;
^~~~~~~~~
1 warning generated.
The MLAG component on the switch is expected to provide some
properties (such as peerlink-rif) to bootstrap the anycast-VTEP
functionality. The final interface for this is being defined as
a part of the pim-mlag functionality.
This commit provides a hidden command to test the anycast-VTEP
functionality independent of the MLAG component.
pimd: display commands for the pim-vxlan-sg database and worklist
Sample output:
root@TORS1:~# vtysh -c "show ip pim vxlan-groups"
Codes: I -> installed
Source Group Input Output Flags
27.0.0.7 239.1.1.101 lo I
* 239.1.1.100 - ipmr-lo I
* 239.1.1.101 - ipmr-lo I
27.0.0.7 239.1.1.100 lo I
root@TORS1:~#
root@TORS1:~# vtysh -c "show ip pim vxlan-work"
Codes: I -> installed
Source Group Input Flags
27.0.0.7 239.1.1.100 lo I
PS: note the worklist dump is a hidden command
1. peerlink-rif as OIF in origination mroutes -
Hosts are multi-homed to the anycast-VTEP pair and can send BUM traffic to
either switch. But the RP would have only joined one MLAG switch for
pulling down the MDT. To make that work we add the peerlink/ISL as
an OIF to origination mroutes (TORC11<=>TORC12 is an anycast VTEP pair) -
root@TORC11:~# ip mr |grep "(36.0.0.9, 239.1.1.100)"
(36.0.0.9, 239.1.1.100) Iif: peerlink-3.4094 Oifs: peerlink-3.4094 uplink-1
root@TORC11:~#
root@TORC12:~# ip mr |grep "(36.0.0.9, 239.1.1.100)"
(36.0.0.9, 239.1.1.100) Iif: peerlink-3.4094 Oifs: peerlink-3.4094
root@TORC12:~#
2. VTEP-PIP as register source -
TORC11 and TORC12 share the same anycast VTEP IP (36.0.0.9 in the above
example). And that is the source registered by both VTEPs for all the BUM
mcast-groups. However to allow the pim register start machine to close
the SIP in the register-pkt's IP header must be set to an unique IP address.
This is the VTEP PIP.
1. special handling of term device in orig mroutes -
The multicast-vxlan termination device ipmr-lo is added to the (*, G)
mroute -
(0.0.0.0, 239.1.1.100) Iif: uplink-1 Oifs: uplink-1 ipmr-lo
This means that it will be inherited into all the SG entries including the
origination mroute. However we cannot terminate the traffic we originate
so some special handling is needed to exclude the termination device
in the origination entries -
27.0.0.7, 239.1.1.100) Iif: lo Oifs: uplink-1
2. special handling of term device on the MLAG pair -
Both MLAG switches pull down BUM-MDT traffic but only one (the DF) can
terminate the traffic. The non-DF must not exclude the termination device
from the MFC to prevent dups to the overlay.
DF -
root@TORC11:~# ip mr |grep "(0.0.0.0, 239.1.1.100)"
(0.0.0.0, 239.1.1.100) Iif: uplink-1 Oifs: uplink-1 ipmr-lo State: resolved
root@TORC11:~#
non-DF -
root@TORC12:~# ip mr |grep "(0.0.0.0, 239.1.1.100)"
(0.0.0.0, 239.1.1.100) Iif: uplink-1 Oifs: uplink-1 State: resolved
root@TORC12:~#
An interface needs to be designated as "termination device" and added to
the termination mroute's OIL. This is used by kernel and ASIC backends
to vxlan-decaps matching flows.
The default termination device is expected to have the prefix (start
sub-string) "ipmr-lo". This can be made configurable if needed -
root@TORS1:~# ip -d link show ipmr-lo
28: ipmr-lo: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/ether 12:5a:ae:74:51:a2 brd ff:ff:ff:ff:ff:ff promiscuity 0
dummy addrgenmode eui64
root@TORS1:~# ip mr
This commit includes the changes to enable pim implicitly on the device
and set it up as the vxlan-term device per-pim-instance.
pimd: add termination mroutes for each vxlan multicast tunnels
To terminate a multicast VxLAN tunnel entry we setup a mroute with
ipmr-lo in the OIL -
(0.0.0.0, 239.1.1.100) Iif: uplink-1 Oifs: uplink-1 ipmr-lo
This is done by the vxlan component that add ipmr-lo as a local
member to termination SG entries. In addition termination entries
are also subject to MLAG DF election on the anycast VxLAN-AA setup.
pimd: MLAG flag defintions in the PIM upstream entries
Two flags have been introduced per-upstream entry -
1. XXX_MLAG_VXLAN - This indicates that MLAG DF (designated-forwarded)
election is needed on the entry. In the case of pim-evpn this flag is set
for termination (*, G) entries and will be inherited by the (S, G) entries
that are created as a result of SPT switchover on the G.
2. XXX_MLAG_NON_DF - This is set on entries that have lost the
DF election. Such entries are primarily used for blackholing traffic on
one of the MLAG switches. On a hardware accelerated switch this blackholing
happens in the ASIC preventing (non-needed) traffic hitting the CPU.
pimd: register local VTEP-IP for each BUM MDT via NULL registers
For multicast vxlan tunnels we register the local VTEP-IP independent
of the prescence of BUM traffic i.e. we prime the pump. This
is acheived via NULL registers.
VxLAN orig entries with upstream in a PIM_REG_JOIN state are linked to
a work list for periodic NULL register transmission. Once the SPT setup
is complete the upstream-entry moves to a PIM_REG_PRUNE state and is
remved from the VxLAN work list.
pimd: add peerlink-rif to the origination-mroute's OIL
In a PIM MLAG setup (say L11<->L12 is the anycast VTEP pair) the RP
can choose to join either MLAG switch as the anycast VTEP-IP is
present on both. Let's say the RP joins L11.
Hosts are dual connected to L11<->L12 and can send traffic to either
switch. Let's say a host sends broadcast traffic to L12; now L12
must encapsulate and send the traffic toward L11. To do that the
origination-mroute on L12 must include the ISL in its OIL -
(36.0.0.9, 239.1.1.100) Iif: peerlink-3.4094 Oifs: peerlink-3.4094
L11 has a similar mroute -
(36.0.0.9, 239.1.1.100) Iif: peerlink-3.4094 Oifs: peerlink-3.4094 uplink-1
This mroute is used to rx traffic on peerlink-3.4094 and send it out of
uplink-1. Note that this mroute also includes the peerlink-rif in its
OIL. Explicit removal of IIF from OIL is done by the kernel (and other
dataplanes) to prevent traffic looping.
pimd: support for vxlan origination-upstream entries
For every (local-vtep-ip, bum-mcast-grp) registered by evpn an origination
mroute is setup by pimd. The purpose of this mroute is to forward vxlan
encapsulated BUM -
Sample mroute (single VTEP):
(27.0.0.7, 239.1.1.100) Iif: lo Oifs: uplink-1
Sample mroute (anycast VTEP):
(36.0.0.9, 239.1.1.100) Iif: peerlink-3.4094\
Oifs: peerlink-3.4094 uplink-1
This commit is part-1 of orignation mroute setup and includes setup
of upstream entries with vxlan properties.
pimd: header changes to cache MLAG information needed for pim-vxlan
This information will come in from a MLAG component. Hidden commands
will also be provided for testing the interface independent of the
separate MLAG component.
PS: It is possible that this cache will be merged with the base
pim-mlag datastructures once they are available.
pimd: vxlan definitions for creation origination and terminatiom mroutes
pim vxlan component will create upstream entries and OIFs for
multicast VxLAN tunnel origination and termination in single-VTEP
and anycast-VTEP (MLAG) setups.
pimd: API for enabling pim on the vxlan term device ipmr-lo
ipmr-lo is a dummy netdev with no additional addressing requirements -
root@TORS1:~# ip -d link show ipmr-lo
28: ipmr-lo: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN mode DEFAULT group default
link/ether 12:5a:ae:74:51:a2 brd ff:ff:ff:ff:ff:ff promiscuity 0
dummy addrgenmode eui64
root@TORS1:~#
This device is used by pim-vxlan to signify multicast-vxlan-tunnel
termination.
In an anycast VTEP setup the peerlink-rif (ISL) is added as a OIF to the
tunnel origination mroute. A new OIF protocol, VxLAN, has been added to
allow that functionalty.
pimd: definition of pim-evpn origination and termination devices
Two devices have special significance to multicast VxLAN tunnels -
1. tunnel origination device -
This device is used as the source device to vxlan-encapsulate BUM traffic.
In the case of the default-vrf this is lo. And in the case of non-default
VRF this is vrf-net-device. This patchset is limited to default-VRF
underlay so all subsequent references of origination-dev are to lo. But it
is possible in the future to extend support to non-default VRFs.
Sample origination mroute on single-VTEP:
(27.0.0.7, 239.1.1.100) Iif: lo Oifs: uplink-1
In the case of MLAG we need to mroute traffic form the MLAG-peer so
we force the IIF to the ISL.
Sample origination mroute on MLAG-VTEP:
(36.0.0.9, 239.1.1.100) Iif: peerlink-3.4094 Oifs: peerlink-3.4094 uplink-1
2. tunnel termination device -
This device is used in the OIL to indicate that packets matching the flow
must be vxlan terminated and overlay packets subsequently forward to the
tenants. A special device has been created for this purpose called ipmr-lo.
This is a simple dummy interface from the kernel perspective which has
special siginficance only to pimd which implicitly enabled pim on the
device and adds it to the termination mroutes.
Sample termination mroute:
(0.0.0.0, 239.1.1.100) Iif: uplink-1 Oifs: uplink-1 ipmr-lo
PS: currently we default the termination device name to "ipmr-lo" but in
the future it is possible to provide a config command to set the
termination device.
pimd: per-SG control to allow any router to register itself as source
In a VxLAN-AA setup both the anycast VTEPS can send VxLAN encapsulated
traffic. This is despite the fact that the it is not-DR on the IIF
associated with the originating mroute.
pimd: provide a per-SG control to disabled register encapsulation of data
In a MLAG setup both of the VTEPs can rx and reg-encapsulate BUM traffic
toward the RP. To prevent these duplicates we need a mechanism to disable
register encaps on specific mroutes.
This is specifically needed to allow pim-evpn mroutes in the MLAG setup -
(36.0.0.11, 239.1.1.100) Iif: peerlink.4094 Oifs: uplink-1, peerlink.4094
I could have gone the other way and disabled PIM_ENFORCE_LOOPFREE_MFC but
that opens the door too wide. Relaxing the checks for mlag-specific mroutes
seemed like the safer choice.
This commit provides the infrastructure to relax checks on a per-mroute
basis.
pimd: provide a mechanism to pin the IIF for an SG entry
In the case of vxlan origination entries IIF is set to -
1. lo for single VTEPs
2. MLAG-ISL for VTEPs multihomed via MLAG.
This commit creates the necessary infrastructure by -
1. allowing the IIF to be set statically (without RPF lookup)
2. and by preventing next-hop-tracking registration
PS: Note that I have skipped additional checks in pim_upstream_del
intentionally i.e. an attempt will be made to remove nexthop-tracking
for the upstream entry (with STATIC_IIF) which will fail because of the
up-entry not being in the nh's hash table. Ideally we should maintain
a nh pointer in the up-entry to prevent this unnecessary processing.
In the abscence of that I wanted to avoid spraying STATIC_IIF checks
all over.
pimd: provide an api to force stop kat on an upstream entry
In the case of pim vxlan we create and keep upstream entries alive
in the abscence of traffic. So we need a mechanism to purge entries
abruptly on vxlan SG delete without having to wait for the entry
to age out.
These are again just the infrastructure changes needed for it.
pimd: provide an upstream control to prevent KAT expiry
For vxlan BUM MDTs we prime the pump and register the local-VTEP-ip
as source even before the first BUM packet is rxed. This commit provides
the infrastructure changes needed for that.
zebra sends (S, G) and (*, G) entries for BUM mcast groups to pimd. This
commit includes the changes to handle the notifications and trigger the
creation of (S, G) base cache in pimd.
pimd: initial infrastructure to maintain VxLAN SG database
These entries will be used over the subsequent commits for
1. vxlan-tunnel-termination handling - setup MDT to rx VxLAN encapsulated
BUM traffic.
2. vxlan-tunnel-origination handling - register local-vtep-ip as a
multicast source.
lib, zebra: changes to propagate vxlan mcast SG entries to pimd
These updates act as triggers to pimd to -
1. join the MDT for rxing VxLAN encapsulated BUM traffic
2. register the local-vtep-ip as a source for the MDT
zebra: maintain mcast tunnel origination and termination SG entries
Each multicast tunnel is associated with a -
1. Tunnel origination mroute that is used for forwarding the
VxLAN encapsulated flow -
S - local VTEP-IP
G - BUM mcast-group
2. And a tunnel termination entry -
S - * (any remote VTEP)
G - BUM mcast-group
Multiple L2 VNIs can share the same BUM mcast group (and local-VTEP-IP).
Zebra maintains an mcast (SG) hash table to pass this info to pimd for
subsequent MDT setup.
zebra: install flood FDB entry only if the remote VTEP asked for HER
Remote VTEPs advertise the flood mode via IMET and the ingress VTEP
needs to perform head-end-replication of BUM packets to it only if the
PMSI tunnel type is set to ingress-replication. If a type-3 route is not
rxed or rxed with a mode other than ingress-replication we can skip
installation of the flood fdb entry for that L2-VNI. In that case the
remote VTEP is either not interested in BUM traffic or is using a
"static-config" based replication mode like PIM.
Sample output with HER -
=======================
root@TORS1:~# vtysh -c "show evpn vni 1000" |grep "Remote\|flood"
Remote VTEPs for this VNI:
27.0.0.8 flood: HER
root@TORS1:~#
Sample output with PIM-SM -
=========================
root@TORS2:~# vtysh -c "show evpn vni 1000" |grep "Remote\|flood"
Remote VTEPs for this VNI:
27.0.0.7 flood: -
root@TORS2:~#
bgpd: propagate flood mode to zebra based on the tunnel-type in the IMET route
IMET/type-3 routes are used by VTEPs to advertise the flood mode for BUM
traffic via the PMSI tunnel attribute. If a type-3 route is not rxed from
a remote-VTEP we default to "no-head-end-rep" for that remote-VTEP. In such
cases static-config such as PIM is likely used for BUM flooding.
bgpd: suppress IMET route generation if flood mode is PIM-SM
IMET route is optional if the flood mode is PIM-SM and serves
no functional purpose. So this change limits type-3 route generation
to flood-mode=head-end-replication.
If PIM-SM if used for BUM flooding the multicast group address can be
configured per-vxlan-device. BGP receives this config from zebra via
the L2 VNI add/update.
zebra: header changes for l2 vni bum-mcast-grp handling
The multicast group ip address for BUM traffic is configurable per-l2-vni.
One way to configure that is to setup a vxlan device that per-l2-vni and
specify the address against that vxlan device -
root@TORS1:~# vtysh -c "show interface vx-1000" |grep -i vxlan
Interface Type Vxlan
VxLAN Id 1000 VTEP IP: 27.0.0.15 Access VLAN Id 1000 Mcast 239.1.1.100
root@TORS1:~# vtysh -c "show evpn vni 1000" |grep Mcast
Mcast group: 239.1.1.100
root@TORS1:~#
Don Slice [Mon, 15 Apr 2019 18:27:00 +0000 (18:27 +0000)]
zebra: stop sending invalid nexthops to clients
Found that zebra_rnh_apply_nht_rmap would set the
NEXTHOP_FLAG_ACTIVE if not blocked by the route-map, even
if the flag was not active prior to the check. This fix
changes the flag used to denote the nexthop is filtered so
that proper active state can be retained. Additionally,
found two cases where we would send invalid nexthops via
send_client, which would also cause this crash. All three
fixed in this commit.
Signed-off-by: Don Slice <dslice@cumulusnetworks.com>
Donald Sharp [Tue, 16 Apr 2019 13:07:12 +0000 (09:07 -0400)]
zebra: Double check is not necessary in nexthop_active_update
The nexthop_active_update command looks at each individual
nexthop and decides if it has changed. If any nexthop
has changed we will set the re->status to ROUTE_ENTRY_CHANGED
and ROUTE_ENTRY_NEXTHOPS_CHANGED.
Additionally the test for old_nh_num != curr_active
makes no sense because suppose we have several events
we are processing at the same time and a total ecmp
of 16 but 14 are active at the start and 14 are active
at the end but different interfaces are up or down.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Donald Sharp [Tue, 16 Apr 2019 12:37:24 +0000 (08:37 -0400)]
lib, zebra: Remove unused flag
The NEXTHOP_FLAG_FILTERED went away when we started treating
static routes like every other route in the system. This was
a special case for handling static route code that just didn't
get finished cleaning up.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Donald Sharp [Tue, 16 Apr 2019 12:09:56 +0000 (08:09 -0400)]
zebra: nexthop_active_update does not need set
We are effectively calling nexthop_active_update() on every
route entry being processed for installation at least 2 times.
This is a bit ridiculous. We need to resolve the nexthops
when we know a route has changed in some manner, so do so.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Donald Sharp [Thu, 18 Apr 2019 00:47:44 +0000 (20:47 -0400)]
tests: bgp_l3vpn_to_bgp_vrf were bailing to quickly
The tests are not coming up consistently on my test box. Add a bit of wait
time to test to allow normal bgp when the first attempt doesn't come up.
Especially since bgp timeouts are 120 seconds with non datacenter compiles.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Certain operations, like removing non-presence containers or
modifying list keys, are not considered to be valid from the
perspective of the northbound layer. This is because we want to
implement a minimum set of northbound configuration callbacks and
use them to process all possible configuration changes.
The removal of a np-container [1], for example, can be processed by
calling the "delete" callback of all of its child nodes (recursion
is used for np-container child nodes). Similarly, the modification
of a list key can be processed as if the corresponding list entry
was removed and readded with updated key values. This strategy saves
us the burden of implementing lots of extra configuration callbacks.
That said, the nb_operation_is_valid() function shouldn't be used
for anything other than checking which callbacks are valid for
which YANG nodes. Using it in the nb_candidate_edit() function
is inappropriate as we want as much flexibility as possible when
editing a candidate configuration. We should allow CLI commands,
for example, to remove np-containers (the northbound layer will then
figure out which callbacks need to be called when this candidate
is committed). Remove the check.
[1] We can't do the same for presence containers since they have a
"create" callback associated with them.
lib: move zlog() prototype back to the public logging API
zlog() should be part of the public logging API as it's useful in
the cases where the logging priority isn't known at compile time
(i.e. it depends on a variable).
lib: don't initialize the northbound database in the unit tests
Move call to nb_db_init() from nb_init() to frr_init() so that only
the FRR daemons will initialize the northbound database. This should
fix a few warnings when running some unit tests.
lib: rework management of user pointers in the northbound layer
Introduce a hash table to keep track of user pointers associated
to configuration entries. The previous strategy was to embed
the user pointers inside libyang data nodes, but this solution
incurred a substantial performance overhead. The user pointers
embedded in candidate configurations could be lost while the
configuration was being edited, so they needed to be regenerated
before the candidate could be committed. This was done by the
nb_candidate_restore_priv_pointers() function, which was extremely
expensive for large configurations. The new hash table solves this
performance problem.
The yang_dnode_[gs]et_entry() functions were renamed and moved from
yang.[ch] to northbound.[ch], which is a more appropriate place
for them. This patch also introduces the nb_running_unset_entry()
function, the counterpart of nb_running_set_entry() (unsetting
user pointers was done automatically before, now it needs to be
done manually).
As a consequence of these changes, we shouldn't need support for
libyang private pointers anymore (-DENABLE_LYD_PRIV=ON). But it's
probably a good idea to keep requiring this feature as we might
need it in the future for other things (e.g. disable configuration
settings without removing them).
David Lamparter [Tue, 16 Apr 2019 19:33:06 +0000 (21:33 +0200)]
ospfd: make ECMP nexthop order deterministic
The order of ECMP nexthops currently depends on whatever order the
pqueue code returns the vertices in, which is essentially random since
they compare as equal. While this shouldn't cause issues normally, it
is nondeterministic and causes the ldp-topo1 test to fail when the
ordering comes up different. Also, nondeterministic behaviour is not a
nice thing to have here in general.
Just sort by nexthop address; realistic numbers of ECMP nexthops should
hopefully not make this a performance issue. (Also, nexthops should be
hot in the caches here.)
Signed-off-by: David Lamparter <equinox@opensourcerouting.org>
Donald Sharp [Wed, 17 Apr 2019 03:15:56 +0000 (23:15 -0400)]
ospf6d: listhead returns a listnode *
The ospf6_route_get_first_nh_index function call calls
listhead which returns a (listnode *) but we are casting
it to a (struct ospf6_nexthop *) and away we go.
Fixes: #4142
Found By: Kwind Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
projects / mirror / frr.git / log