Chirag Shah [Fri, 8 Dec 2017 17:33:53 +0000 (09:33 -0800)]
ospfd: prevent passive interface cmd crash
Current OSPF VRF configuration are allow pre-provisining even if
VRF is not configured. In such case ospf->vrf_id would VRF_UNKNOWN,
when passive interface configuration done under such ospf instance,
it would lookup all vrf_device and try to create ifp with unknown
vrf_id.
for passive interface config command lookup ifp for vrf_id is within range.
Donald Sharp [Thu, 7 Dec 2017 23:59:54 +0000 (18:59 -0500)]
ldpd: Switch over to new debug style
When compiling ldpd on a mac, there exists a #define MSG_SEND
which conflicts with a define in ldp_debug.h.
During discussion about this we decided that it would be
better to remove the macro massaging that was going on and
to just call our own #define for it.
Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Chirag Shah [Mon, 4 Dec 2017 22:08:23 +0000 (14:08 -0800)]
ospfd: fix crash no router ospf/show running
no router ospf removes default ospf instance,
if there are other non-default vrf instance present
with interface level configuration. Lookup ospf instance
for ifp->vrf_id, if ospf instnace present use that
to access 'instance id'.
Ticket: CM-19078
Testing Done:
run no router ospf and show running config along with other
non-default vrf aware ospf configurations.
Signed-off-by: Chirag Shah <chirag@cumulusnetworks.com>
Chirag Shah [Mon, 27 Nov 2017 21:59:19 +0000 (13:59 -0800)]
ospfd: Display all vrf aware interface config
OSPF interface specific configuration can be done independent
of router ospf [vrf x] global config.
In cases where ospf interface non default vrf configuration
is done prior to 'router ospf vrf x', show running-config
would not display such configuration.
To display configuration now walk all vrfs and interface list
and only display where OSPF configure params are set.
Ticket:CM-18952
Testing Done:
Tried ospf interface specific configuration with VRF,
where router ospf vrf x is not present.
Signed-off-by: Chirag Shah <chirag@cumulusnetworks.com>
Donald Sharp [Fri, 1 Dec 2017 16:49:13 +0000 (11:49 -0500)]
bgpd: Allow Address-Family activation to work in certain states
If we are in OpenSent or OpenConfirm peer state and we receive a new
address-family activation, we would end up ignoring the new activation
and not tell our peer about it. You could notice this by seeing
the fact that a 'show bgp neighbor' command returns a 'Not in
any update group' for a particular family.
This modifies the code such that we now notice that we are in
either OpenSent or OpenConfirm state and reset the peer to
allow us to send them the new capability.
Ticket: CM-19021 Signed-off-by: Donald Sharp <sharpd@cumulusnetworks.com>
Quentin Young [Thu, 30 Nov 2017 19:11:12 +0000 (14:11 -0500)]
bgpd: intelligently adjust coalesce timer
The subgroup coalesce timer controls how long updates to a particular
subgroup are delayed in order to allow additional peers to join the
subgroup. Presently the timer value is 200 ms. Increase it to 1 second
and adjust up as peers are configured, with an upper cap at 10s.
This cuts convergence time by a factor of 3 at large scale (300+ peers,
1000+ prefixes per peer).
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 13 Nov 2017 22:59:04 +0000 (17:59 -0500)]
bgpd: turn off keepalives when sending NOTIFY
This is necessary because otherwise between the time we wipe the output
buffer and the time we push the NOTIFY onto it, the KA generation thread
could have pushed a KEEPALIVE in the middle.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 13 Nov 2017 08:18:49 +0000 (03:18 -0500)]
bgpd: yield more when generating UPDATEs
In the same vein as the round-robin input commit, this re-adds logic for
limiting the amount of time spent generating UPDATEs per generation
cycle. Missed this when shifting around wpkt_quanta; prior to MT it
limited both calls to write() as well as UPDATE generation.
Quentin Young [Fri, 10 Nov 2017 21:42:49 +0000 (16:42 -0500)]
bgpd: restore packet input limit
Unfortunately, batching input processing severely impacts BGP initial
convergence times. As a consequence of the way update-groups were
implemented, advancing the state of the routing table based on prefixes
learned from one peer prior to all (or at least most) peers establishing
connections will cause us to start generating outbound UPDATEs, which is
a very expensive operation at present. This intensive processing starves
out bgp_accept(), delaying connection of additional peers. When
additional peers do connect the problem gets worse and worse, yielding
approximately exponential growth in convergence time dependent on both
peering and prefix counts. This behavior is present pre-multithreading
as well, but batched input exacerbates it.
Round-robin input processing marginally harms convergence times for
small topologies but should allow much larger topologies to function
within reasonable performance thresholds.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Tue, 7 Nov 2017 07:49:54 +0000 (02:49 -0500)]
bgpd: schedule process packet as timer
Different places scheduling the same thread should use the same
semantics and thread type. Additionally providing the back reference
here makes sure we only schedule the job once and avoids flooding the
event queue with jobs to process an empty buffer.
Quentin Young [Mon, 6 Nov 2017 19:15:36 +0000 (14:15 -0500)]
bgpd: re-add write trigger logic
Apparently I didn't fully understand how subgroup packets make their way
out to individual peers. Turns out (on the base branch) we just busy
poll while waiting for packets to make their way onto subgroup queues.
While this needs to be fixed in the future, for now readding this logic
fixes performance issues with convergence.
Quentin Young [Mon, 6 Nov 2017 06:41:27 +0000 (01:41 -0500)]
bgpd: properly set peer->last_update
Instead of checking whether the post-write number of updates sent was
greater than the pre-write number of updates sent, it was comparing post
to zero. In effect this meant every time we wrote a packet it was
counted as an update for route advertisement timer purposes.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 6 Nov 2017 05:33:46 +0000 (00:33 -0500)]
bgpd: schedule packet job after connection xfer
During initial session establishment, bgpd performs a "connection
transfer" to a new peer struct if the connection was initiated passively
(i.e. by the remote peer). With the addition of buffered input and a
reorganized packet processor, the following race condition manifests:
1. Remote peer initiates a connection. After exchanging OPEN messages,
we send them a KEEPALIVE. They send us a KEEPALIVE followed by
10,000 UPDATE messages. The I/O thread pushes these onto our local
peer's input buffer and schedules a packet processing job on the
main thread.
2. The packet job runs and processes the KEEPALIVE, which completes the
handshake on our end. As part of transferring to ESTABLISHED we
transfer all peer state to a new struct, as mentioned. Upon returning
from the KEEPALIVE processing routing, the peer context we had has
now been destroyed. We notice this and stop processing. Meanwhile
10k UPDATE messages are sitting on the input buffer.
3. N seconds later, the remote peer sends us a KEEPALIVE. The I/O thread
schedules another process job, which finds 10k UPDATEs waiting for
it. Convergence is achieved, but has been delayed by the value of the
KEEPALIVE timer.
The racey part is that if the remote peer takes a little bit of time to
send UPDATEs after KEEPALIVEs -- somewhere on the order of a few hundred
milliseconds -- we complete the transfer successfully and the packet
processing job is scheduled on the new peer upon arrival of the UPDATE
messages. Yuck.
The solution is to schedule a packet processing job on the new peer
struct after transferring state.
Lengthy commit message in case someone has to debug similar problems in
the future...
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 3 Nov 2017 18:47:56 +0000 (14:47 -0400)]
bgpd: transfer raw input buffer to new peer
During initial session establishment, bgpd performs a "connection
transfer" to a new peer struct if the connection was initiated passively
(i.e. by the remote peer). With the addition of buffered input, I forgot
to transfer the raw input buffer to the new peer. This resulted in
infrequent failures during session handshaking whereby half of a packet
would be thrown away in the middle of a read causing us to send a NOTIFY
for an unsynchronized header. Usually the transfer coincided with a
clean input buffer, hence why it only showed up once in a while.
Quentin Young [Mon, 25 Sep 2017 02:18:15 +0000 (22:18 -0400)]
bgpd: fix bgp active open
At some point when rearranging FSM code, bgpd lost the ability to
perform active opens because it was only paying attention to POLLIN and
not POLLOUT, when the latter is used to signify a successful connection
in the active case.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 12 Jun 2017 21:16:40 +0000 (21:16 +0000)]
bgpd: be more promiscuous with updgrp packets
Slightly incorrect trigger for generating update group packets. In order
to match semantics of previous bgp_write() we need to trigger
update-group packet generation after every write operation, even if no
packets were written. Of course if we're tearing down the session we can
still skip this operation.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 12 Jun 2017 20:20:50 +0000 (20:20 +0000)]
bgpd: re-add update-group write triggers
Removed in earlier version where the I/O pthread busy-waited for packets
to be posted to an output queue. Now that it's poll()-based, it's
necessary once again. Although this time we can say what we're actually
doing instead of a side effect of a write job.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 12 Jun 2017 02:53:42 +0000 (02:53 +0000)]
bgpd: misc fsm fixes
* Keepalive on/off calls are necessary in certain cases due to screwy
fsm flow not turning them on after transferring a passive peer
connection in peer_xfer_conn
* Missed a case bgp_event_update() that resulted in a return code of -1
instead of BGP_Stop, which confuses the packet processing routine
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Sat, 10 Jun 2017 01:01:56 +0000 (01:01 +0000)]
bgpd: fix bgp_packet.c / bgp_fsm.c organization
Despaghettification of bgp_packet.c and bgp_fsm.c
Sometimes we call bgp_event_update() inline packet parsing.
Sometimes we post events instead.
Sometimes we increment packet counters in the FSM.
Sometimes we do it in packet routines.
Sometimes we update EOR's in FSM.
Sometimes we do it in packet routines.
Fix the madness.
bgp_process_packet() is now the centralized place to:
- Update message counters
- Execute FSM events in response to incoming packets
FSM events are now executed directly from this function instead of being
queued on the thread_master. This is to ensure that the FSM contains the
proper state after each packet is parsed. Otherwise there could be race
conditions where two packets are parsed in succession without the
appropriate FSM update in between, leading to session closure due to
receiving inappropriate messages for the current FSM state.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Thu, 8 Jun 2017 21:14:18 +0000 (21:14 +0000)]
bgpd: small i/o threading improvements
* Start bit flags at 1, not 2
* Make run-flags atomic for i/o thread
* Remove work_cond mutex, it should no longer be necessary
* Add asserts to ensure proper ordering in bgp_connect()
* Use true/false with booleans, not 1/0
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 5 Jun 2017 20:14:47 +0000 (20:14 +0000)]
bgpd: atomize write-quanta, add read-quanta
bgpd supports setting a write-quanta that serves as a hint on how many
packets to write per I/O cycle. Now that input is buffered, it makes
sense to add the equivalent parameter for how many packets are processed
per cycle. This is *not* how many packets are read off the wire per I/O
cycle; rather it is how many packets are processed from the input buffer
in a given cycle after having been read off the wire and sanitized.
Since these values must be used from multiple threads, they have also
been made atomic.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 2 Jun 2017 01:52:39 +0000 (01:52 +0000)]
bgpd: batched i/o
Instead of reading a packet header and the rest of the packet in two
separate i/o cycles, instead read a chunk of data at one time and then
parse as many packets as possible out of the chunk.
Also changes bgp_packet.c to batch process packets.
To avoid thrashing on useless mutex locks, the scheduling call for
bgp_process_packet has been changed to always succeed at the cost of no
longer being cancel-able. In this case this is acceptable; following the
pattern of other event-based callbacks, an additional check in
bgp_process_packet to ignore stray events is sufficient. Before deleting
the peer all events are cleared which provides the requisite ordering.
XXX: chunk hardcoded to 5, should use something similar to wpkt_quanta
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Tue, 2 May 2017 00:37:45 +0000 (00:37 +0000)]
bgpd: implement buffered reads
* Move and modify all network input related code to bgp_io.c
* Add a real input buffer to `struct peer`
* Move connection initialization to its own thread.c task instead of
piggybacking off of bgp_read()
* Tons of little fixups
Primary changes are in bgp_packet.[ch], bgp_io.[ch], bgp_fsm.[ch].
Changes made elsewhere are almost exclusively refactoring peer->ibuf to
peer->curr since peer->ibuf is now the true FIFO packet input buffer
while peer->curr represents the packet currently being processed by the
main pthread.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Tue, 18 Apr 2017 18:11:43 +0000 (18:11 +0000)]
bgpd: move bgp i/o to a separate source file
After implement threading, bgp_packet.c was serving the double purpose
of consolidating packet parsing functionality and handling actual I/O
operations. This is somewhat messy and difficult to understand. I've
thus moved all code and data structures for handling threaded packet
writes to bgp_io.[ch].
Although bgp_io.[ch] only handles writes at the moment to keep the noise
on this commit series down, for organization purposes, it's probably
best to move bgp_read() and its trappings into here as well and
restructure that code so that read()'s happen in the pthread and packet
processing happens on the main thread.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Thu, 6 Apr 2017 23:45:57 +0000 (23:45 +0000)]
bgpd: correctly schedule select() at session startup
On TCP connection failure during session setup, bgp_stop() checks
whether peer->t_read is non-null to know whether or not to unschedule
select() on peer->fd before calling close() on it. Using the API exposed
by thread.c instead of bgpd's wrapper macro BGP_READ_ON() results in
this thread value never being set, which causes bgp_stop() to skip the
cancellation of select() before calling close(). Subsequent calls to
select() on that fd crash the daemon.
Use the macro instead.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Thu, 6 Apr 2017 01:09:33 +0000 (01:09 +0000)]
bgpd: transfer packets from peer stub to actual peer
During transition from OpenConfirm -> Established, we wipe the peer stub's
output buffer. Because thread.c prioritizes I/O operations over regular
background threads and events, in a single threaded environment this ordering
meant that the output buffer would be happily empty at wipe time. In MT-land,
this convenient coincidence is no longer true; thus we need to make sure that
any packets remaining on the peer stub get transferred over to the peer proper.
Also removes misleading comment indicating that bgp_establish() sends a
keepalive packet. It does not.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Changes all synchronization primitives to be dynamically allocated. This
should help catch any subtle errors in pthread lifecycles.
This change also pre-initializes synchronization primitives before
threads begin to run, eliminating a potential race condition that
probably would have caused a segfault on startup on a very fast box.
Also changes mutex and condition variable allocations to use
MTYPE_PTHREAD and updates tests to do the proper initializations.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Mon, 27 Mar 2017 19:47:23 +0000 (19:47 +0000)]
bgpd: remove unused `struct thread` from peer
* Remove t_write
* Remove t_keepalive
These have been replaced by pthreads and are no longer needed. Since
some code looks at these values to determine if the threads are
scheduled, also add a new bitfield to store the same information.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Wed, 8 Mar 2017 23:16:15 +0000 (23:16 +0000)]
lib, bgpd: implement pthread lifecycle management
Removes the WiP shim and implements proper thread lifecycle management.
* Declare necessary pthread_t's in bgp_master
* Define new MTYPE in lib/thread.c for pthreads
* Allocate and free BGP's pthreads appropriately
* Terminate and join threads appropriately
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Thu, 5 Jan 2017 23:13:16 +0000 (23:13 +0000)]
bgpd: put BGP keepalives in a pthread
This patch, in tandem with moving packet writes into a dedicated kernel
thread, fixes session flaps caused by long-running internal operations
starving the (old) userspace write thread.
BGP keepalives are now produced by a kernel thread and placed onto the
peer's output queue. These are then consumed by the write thread. Both
of these tasks are concurrent with the rest of bgpd, obviating the
session flaps described above.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Fri, 24 Mar 2017 19:05:56 +0000 (19:05 +0000)]
bgpd: move bgp_connect_check() to bgp_fsm.c
Prior to this change, after initiating a nonblocking connection to the
remote peer bgpd would call both BGP_READ_ON and BGP_WRITE_ON on the
peer's socket. This resulted in a call to select(), so that when some
event (either a connection success or failure) occurred on the socket,
one of bgp_read() or bgp_write() would run. At the beginning of each of
those functions was a hook into bgp_connect_check(), which checked the
socket status and issued the correct connection event onto the BGP FSM.
This code is better suited for bgp_fsm.c. Placing it there avoids
scheduling packet reads or writes when we don't know if the socket has
established a connection yet, and the specific functionality is a better
fit for the responsibility scope of this unit.
This change also helps isolate the responsibilities of the
packet-writing kernel thread.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
Quentin Young [Wed, 22 Mar 2017 17:13:23 +0000 (17:13 +0000)]
bgpd: move update group processing to main thread
Prior to this change, packets generated for update groups were taken off
of the (independent) buffer for the update group, reformatted for the
specific peer under question and sent off inline with bgp_write(). Since
the operations of this code path can include the merging and pruning of
subgroups and are too large to safely synchronize, this change moves
that logic to execute after each tick of the write thread.
Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>
projects / mirror / frr.git / log