lib: introduce configuration back-off timer for YANG-modeled commands
When using the default CLI mode, the northbound layer needs to create
a separate transaction to process each YANG-modeled command since
they are supposed to be applied immediately (there's no candidate
configuration nor the "commit" command like in the transactional
CLI). The problem is that configuration transactions have an overhead
associated to them, in big part because of the use of some heavy
libyang functions like `lyd_validate()` and `lyd_diff()`. As of
now this overhead is substantial and doesn't scale well when large
numbers of transactions need to be performed in sequence.
As an example, loading 50k prefix-lists using a single transaction
takes about 2 seconds on a modern CPU. Loading the same 50k
prefix-lists using 50k transactions can take more than an hour
to complete (which is unacceptable by any standard). To fix this
problem, some heavy optimization work needs to be done on libyang and
on the FRR northbound itself too (e.g. perform partial configuration
diffs whenever possible). This, however, should be a long term
effort since these optimizations shouldn't be trivial to implement
and we're far from having the performance numbers we need.
In the meanwhile, this commit introduces a simple but efficient
workaround to alleviate the issue. In short, a new back-off timer
was introduced in the CLI to monitor and detect when too many
YANG-modeled commands are being received at the same time. When
a certain threshold is reached (100 YANG-modeled commands within
one second), the northbound starts to group all subsequent commands
into a single large transaction, which allows them to be processed
much faster (e.g. seconds and not hours). It's essentially a
protection mechanism that creates dynamically-sized transactions
when necessary to prevent performance issues from happening. This
mechanism is enabled both when parsing configuration files and when
reading commands from a terminal.
The downside of this optimization is that, if several YANG-modeled
commands are grouped into the same transaction and at least one of
them fails, the whole transaction is rejected. This is undesirable
since users don't expect transactional behavior when that's not
enabled explicitly. To minimize this issue, the CLI will log all
commands that were rejected whenever that happens, to make the
user aware of what happened and have enough information to fix
the problem. Commands that fail due to parsing errors or CLI-level
validations in general are rejected separately.
Again, this proposed workaround is intended to be temporary. The
goal is to provided a quick fix to issues like #6658 while we work
on better long-term solutions.
DEFPY_YANG will allow the CLI to identify which commands are
YANG-modeled or not before executing them. This is going to be
useful for the upcoming configuration back-off timer work that
needs to commit pending configuration changes before executing a
command that isn't YANG-modeled.
Mark Stapp [Wed, 29 Jul 2020 20:57:37 +0000 (16:57 -0400)]
tests: Avoid top ecmp route_scale test case when memory limited
Address-sanitizer runs in the CI appear to require more
memory than is available (at present), so skip the top
x32 route_scale testcase when running with <4G of ram.
Mark Stapp [Mon, 20 Jul 2020 18:51:16 +0000 (14:51 -0400)]
tests: rework route_scale topotest
Make some changes to the route-scale topotest, in view of
issue #6734. Table-drive the test to eliminate some
repeated code. Assert and fail if a step in the progression
of scale fails. Wait a little longer between checking the show
output - it's costly to generate that output at scale. Add a
memleak testcase.
The `INET_ADDRSTRLEN` is 16 and is only enough to format an IPv4 address.
So when there is a prefix (`/xx`), the debug output may get truncated.
Use `PREFIX2STR_BUFFER` macro instead to fix the issue.
tests: Skipping evpn_type5_test_topo1 tests from CI runs
1. evpn_type5_test_topo1 tests started failing in CI for all Ubuntu 18.04 machine,
which are having kernel version: 5.4.0-42-generic
2. We will enable these tests once issue is found and fixed.
Donald Sharp [Mon, 27 Jul 2020 15:09:16 +0000 (11:09 -0400)]
tests: Remove 'log monitor' from tests
The `log monitor' command is a no-op and actually
outputs a `this doesn't do anything` warning. Let's remove
this cli line from our tests as that don't do anything and
people will look at these configs for guidance.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Donald Sharp [Mon, 27 Jul 2020 11:10:41 +0000 (07:10 -0400)]
bgpd: Deref after null check in bgp_evpn_vty.c
Coverity has noticed that we are using bgp_evpn after
we have already NULL checked it one time. Add an assert
to make Coverity happy here, if we get to this point
something terrible has happened.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
causes clang 11 to be unhappy, with length of field warnings.
Modify the offending code to compile properly using that compiler.
I've tested against clang 11 and gcc 9.3
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Russell Bryant [Fri, 24 Jul 2020 00:26:54 +0000 (20:26 -0400)]
bgpd: Add missing newline to a log message
While checking my BGP debugging settings at the console, I noticed
this message was missing a newline. Add it to be consistent with the
other similar messages.
Signed-off-by: Russell Bryant <rbryant@redhat.com>
Tore Anderson [Wed, 22 Jul 2020 15:32:18 +0000 (17:32 +0200)]
tools: do not silently ignore errors when loading config during startup
Drop the `-n` (`--noerror`) flag from the `vtysh -b` invocation called by the
init script responsible for starting FRR. This ensures that errors in the
configuration file is propagated to the administrator, and prevents a node from
entering a production network while running an essentially undefined
configuration (a behaviour that I can personally attest to has the potential to
cause disastrous network outages - documented in more detail in Cumulus
Networks CS#12791).
Silently ignoring errors also leads to the rather odd behaviour that starting
FRR will ostensibly succeed, while reloading it immediately after - without
changing the configuration - will fail. This is due to the fact that the `-n`
flag is not used while reloading.
The use of the `-n` flag appears to have been introduced without any
explanation in commit 858aa29c6862ed2390baee53b6fc9f54e65246e2 by @donaldsharp.
Looking at the commit message, I suspect that it was not an intentional change.
It seems more likely to me that it was just meant to be used during testing and
development, but ended up being committed to master by accident.
Chirag Shah [Mon, 6 Jul 2020 18:13:54 +0000 (11:13 -0700)]
zebra: vrf disable clean up evpn rmac nxthp cache
In networking restart event, l3vni (vxlan) interface followed by
associated vrf interfaces go down/deleted.
L3vni (oper) down event (from zebra to bgp) triggers to
clean up/un-import evpn routes (one-by-one) from the vrf table,
zebra internally removes the route entry from nexthop and RMAC hash.
When all the routes references in nexthop and RMAC db removed,
both (nexthop/rmac) are suppose to be uninstalled from the
bridge fdb and neigh table.
While evpn routes removal in progress, a vrf disable event removes
l3vni to its vrf association.
Subsequent bgp to evpn routes removal does not clean up thus evpn routes
reference to nexthop and RMAC remains in zebra hash.
bridge fdb and neigh tables are flushed out since networking restart brings down
all interfaces which results in flush of fdb and neigh tables.
By product is the zebra does not install nexthop and rmac when routes are re-imported
into vrf in VNI/VRF up event.
The fix is in vrf disable event to flush all l3vni's nexthop and rmac db.
Mark Stapp [Fri, 17 Jul 2020 17:10:29 +0000 (13:10 -0400)]
zebra: improve vty, simplify some primary/backup code
Improve vty output for routes and lsps with backups, including
json. Simplify or correct some code that uses both primary and
backup nexthops in dplane, nht.
Igor Ryzhov [Mon, 20 Jul 2020 15:15:49 +0000 (18:15 +0300)]
vtysh: rework DEFPY processing in extract.pl
Currently, all DEFPY commands are translated into one-liners in
vtysh_cmd.c. After the patch, DEFPY commands are correctly indented just
like DEFUN/ALIAS commands.
Update LDP to process received pw-status in received order.
Update LDP to save pw-status regardless of whether the PW is configured.
When the PW is configured, LDP checks for any saved PW pw-status.
David Lamparter [Sun, 19 Jul 2020 09:14:48 +0000 (11:14 +0200)]
lib: fix TLS log buffer on NetBSD
... this didn't work on NetBSD. Like, at all. It returns a positive
error code from posix_fallocate() and then we bang our head against a
brick wall trying to write to the mmap'd buffer.
Signed-off-by: David Lamparter <equinox@diac24.net>
Rafael Zalamena [Sun, 19 Jul 2020 18:27:56 +0000 (15:27 -0300)]
lib,yang: merge cisco/zebra access list styles
Merge the cisco style access list with zebra's logic so we can mix both
types of rules while keeping the commands.
With this the cisco style limitation of having 'destination-*' only for
specific number ranges no longer exist for users of YANG/northbound (the
CLI still has this limitation).
Signed-off-by: Rafael Zalamena <rzalamena@opensourcerouting.org>
Mark Stapp [Tue, 30 Jun 2020 19:52:37 +0000 (15:52 -0400)]
lib,sharpd,zebra: initial support for multiple backup nexthops
Initial changes to support a nexthop with multiple backups. Lib
changes to hold a small array in each primary, zapi message
changes to support sending multiple backups, and daemon
changes to show commands to support multiple backups. The config
input for multiple backup indices is not present here.
Don Slice [Tue, 14 Jul 2020 14:39:06 +0000 (14:39 +0000)]
tools: fix more frr-reload vrf static errors
Reported that in certain config changes, a static intended for the
default table would be duplicated into a vrf context. Determined
that we still weren't keeping or adding the exit-vrf command when
necessary to keep the contexts straight. Added logic to look for
the failing circumstances and add or remove the delete of the
exit-vrf command as needed.
Philippe Guibert [Thu, 16 Apr 2020 13:18:24 +0000 (15:18 +0200)]
zebra: do not free iptable context, once created
This commit avoids freeing the iptable context, once created. the case
where there is an error when reading zapi stream simply needs to free
the zpi context.
Donald Sharp [Fri, 17 Jul 2020 14:07:17 +0000 (10:07 -0400)]
bgpd: Remove memset from bgp_zebra_announce
During perf testing of receiving and installing 7.5 million
routes into zebra it was noticed that memset in bgp_zebra_announce
was taking ~11% of the runtime. With this change bgp_zebra_announce
now no longer has any appreciable time spent in memset as reported
by perf. In addition bgp_zebra_announce run time in perf was
reduced by a composite amount.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
projects / mirror / frr.git / log