bgpd: intelligently adjust coalesce timer

The subgroup coalesce timer controls how long updates to a particular
subgroup are delayed in order to allow additional peers to join the
subgroup. Presently the timer value is 200 ms. Increase it to 1 second
and adjust up as peers are configured, with an upper cap at 10s.

This cuts convergence time by a factor of 3 at large scale (300+ peers,
1000+ prefixes per peer).

Signed-off-by: Quentin Young <qlyoung@cumulusnetworks.com>

1 files changed, 21 insertions, 1 deletions

diff --git a/bgpd/bgp_updgrp.h b/bgpd/bgp_updgrp.h
index 97afe7a11a..e941fecb61 100644
--- a/bgpd/bgp_updgrp.h
+++ b/bgpd/bgp_updgrp.h
@@ -29,7 +29,27 @@
 
 #include "bgp_advertise.h"
 
-#define BGP_DEFAULT_SUBGROUP_COALESCE_TIME 200
+/*
+ * The following three heuristic constants determine how long advertisement to
+ * a subgroup will be delayed after it is created. The intent is to allow
+ * transient changes in peer state (primarily session establishment) to settle,
+ * so that more peers can be grouped together and benefit from sharing
+ * advertisement computations with the subgroup.
+ *
+ * These values have a very large impact on initial convergence time; any
+ * changes should be accompanied by careful performance testing at all scales.
+ *
+ * The coalesce time 'C' for a new subgroup within a particular BGP instance
+ * 'B' with total number of known peers 'P', established or not, is computed as
+ * follows:
+ *
+ * C = MIN(BGP_MAX_SUBGROUP_COALESCE_TIME,
+ *         BGP_DEFAULT_SUBGROUP_COALESCE_TIME +
+ *         (P*BGP_PEER_ADJUST_SUBGROUP_COALESCE_TIME))
+ */
+#define BGP_DEFAULT_SUBGROUP_COALESCE_TIME 1000
+#define BGP_MAX_SUBGROUP_COALESCE_TIME 10000
+#define BGP_PEER_ADJUST_SUBGROUP_COALESCE_TIME 50
 
 #define PEER_UPDGRP_FLAGS                                                      \
 	(PEER_FLAG_LOCAL_AS_NO_PREPEND | PEER_FLAG_LOCAL_AS_REPLACE_AS)