diff options
Diffstat (limited to 'doc/developer')
32 files changed, 1008 insertions, 140 deletions
diff --git a/doc/developer/building-frr-for-archlinux.rst b/doc/developer/building-frr-for-archlinux.rst index f62add5963..e589a9f724 100644 --- a/doc/developer/building-frr-for-archlinux.rst +++ b/doc/developer/building-frr-for-archlinux.rst @@ -11,7 +11,7 @@ Installing Dependencies git autoconf automake libtool make cmake pcre readline texinfo \ pkg-config pam json-c bison flex python-pytest \ c-ares python systemd python2-ipaddress python-sphinx \ - systemd-libs net-snmp perl libcap + systemd-libs net-snmp perl libcap libelf .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-centos6.rst b/doc/developer/building-frr-for-centos6.rst index b730a5ee32..5d3be492de 100644 --- a/doc/developer/building-frr-for-centos6.rst +++ b/doc/developer/building-frr-for-centos6.rst @@ -45,7 +45,8 @@ Add packages: sudo yum install git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig \ - json-c-devel pam-devel flex epel-release c-ares-devel libcap-devel + json-c-devel pam-devel flex epel-release c-ares-devel libcap-devel \ + elfutils-libelf-devel Install newer version of bison (CentOS 6 package source is too old) from CentOS 7: diff --git a/doc/developer/building-frr-for-centos7.rst b/doc/developer/building-frr-for-centos7.rst index eb97150d67..8d0aea943c 100644 --- a/doc/developer/building-frr-for-centos7.rst +++ b/doc/developer/building-frr-for-centos7.rst @@ -21,7 +21,8 @@ Add packages: sudo yum install git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig \ json-c-devel pam-devel bison flex pytest c-ares-devel \ - python-devel systemd-devel python-sphinx libcap-devel + python-devel systemd-devel python-sphinx libcap-devel \ + elfutils-libelf-devel .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-centos8.rst b/doc/developer/building-frr-for-centos8.rst index 75beb53378..77fe489358 100644 --- a/doc/developer/building-frr-for-centos8.rst +++ b/doc/developer/building-frr-for-centos8.rst @@ -14,7 +14,8 @@ Add packages: sudo dnf install --enablerepo=PowerTools git autoconf pcre-devel \ automake libtool make readline-devel texinfo net-snmp-devel pkgconfig \ groff pkgconfig json-c-devel pam-devel bison flex python2-pytest \ - c-ares-devel python2-devel systemd-devel libcap-devel + c-ares-devel python2-devel systemd-devel libcap-devel \ + elfutils-libelf-devel .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-debian8.rst b/doc/developer/building-frr-for-debian8.rst index c12bf46f8d..51dd07c42a 100644 --- a/doc/developer/building-frr-for-debian8.rst +++ b/doc/developer/building-frr-for-debian8.rst @@ -18,7 +18,7 @@ Add packages: sudo apt-get install git autoconf automake libtool make \ libreadline-dev texinfo libjson-c-dev pkg-config bison flex python3-pip \ libc-ares-dev python3-dev python3-sphinx build-essential libsystemd-dev \ - libsnmp-dev libcap-dev + libsnmp-dev libcap-dev libelf-dev Install newer pytest (>3.0) from pip diff --git a/doc/developer/building-frr-for-debian9.rst b/doc/developer/building-frr-for-debian9.rst index f976b9f49a..919b010314 100644 --- a/doc/developer/building-frr-for-debian9.rst +++ b/doc/developer/building-frr-for-debian9.rst @@ -11,7 +11,7 @@ Add packages: sudo apt-get install git autoconf automake libtool make \ libreadline-dev texinfo libjson-c-dev pkg-config bison flex \ libc-ares-dev python3-dev python3-pytest python3-sphinx build-essential \ - libsnmp-dev libsystemd-dev libcap-dev + libsnmp-dev libsystemd-dev libcap-dev libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-fedora.rst b/doc/developer/building-frr-for-fedora.rst index 4ab59490fd..5fecd8a826 100644 --- a/doc/developer/building-frr-for-fedora.rst +++ b/doc/developer/building-frr-for-fedora.rst @@ -14,7 +14,8 @@ Installing Dependencies sudo dnf install git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig json-c-devel \ pam-devel python3-pytest bison flex c-ares-devel python3-devel \ - python3-sphinx perl-core patch systemd-devel libcap-devel + python3-sphinx perl-core patch systemd-devel libcap-devel \ + elfutils-libelf-devel .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-opensuse.rst b/doc/developer/building-frr-for-opensuse.rst index 5ed714a67e..4e886e9c25 100644 --- a/doc/developer/building-frr-for-opensuse.rst +++ b/doc/developer/building-frr-for-opensuse.rst @@ -13,7 +13,8 @@ Installing Dependencies zypper in git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig libjson-c-devel\ pam-devel python3-pytest bison flex c-ares-devel python3-devel\ - python3-Sphinx perl patch systemd-devel libcap-devel libyang-devel + python3-Sphinx perl patch systemd-devel libcap-devel libyang-devel \ + libelf-devel Building & Installing FRR ------------------------- diff --git a/doc/developer/building-frr-for-ubuntu1404.rst b/doc/developer/building-frr-for-ubuntu1404.rst index cc54415266..2711e92b6f 100644 --- a/doc/developer/building-frr-for-ubuntu1404.rst +++ b/doc/developer/building-frr-for-ubuntu1404.rst @@ -14,7 +14,7 @@ Installing Dependencies git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ libc-ares-dev python3-dev python3-sphinx install-info build-essential \ - libsnmp-dev perl libcap-dev + libsnmp-dev perl libcap-dev libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-ubuntu1604.rst b/doc/developer/building-frr-for-ubuntu1604.rst index 63c6f8648c..2cb9536f9b 100644 --- a/doc/developer/building-frr-for-ubuntu1604.rst +++ b/doc/developer/building-frr-for-ubuntu1604.rst @@ -14,7 +14,8 @@ Installing Dependencies git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ libc-ares-dev python3-dev libsystemd-dev python-ipaddress python3-sphinx \ - install-info build-essential libsystemd-dev libsnmp-dev perl libcap-dev + install-info build-essential libsystemd-dev libsnmp-dev perl libcap-dev \ + libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-ubuntu1804.rst b/doc/developer/building-frr-for-ubuntu1804.rst index 9d85957d88..eb3991c139 100644 --- a/doc/developer/building-frr-for-ubuntu1804.rst +++ b/doc/developer/building-frr-for-ubuntu1804.rst @@ -14,7 +14,8 @@ Installing Dependencies git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ libc-ares-dev python3-dev libsystemd-dev python-ipaddress python3-sphinx \ - install-info build-essential libsystemd-dev libsnmp-dev perl libcap-dev + install-info build-essential libsystemd-dev libsnmp-dev perl libcap-dev \ + libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-ubuntu2004.rst b/doc/developer/building-frr-for-ubuntu2004.rst index f7f8c63e5b..ffc05a6841 100644 --- a/doc/developer/building-frr-for-ubuntu2004.rst +++ b/doc/developer/building-frr-for-ubuntu2004.rst @@ -15,7 +15,7 @@ Installing Dependencies pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ libc-ares-dev python3-dev libsystemd-dev python-ipaddress python3-sphinx \ install-info build-essential libsystemd-dev libsnmp-dev perl \ - libcap-dev python2 + libcap-dev python2 libelf-dev Note that Ubuntu 20 no longer installs python 2.x, so it must be installed explicitly. Ensure that your system has a symlink named @@ -27,7 +27,7 @@ ubuntu apt repositories; in order to install it: .. code-block:: shell - curl https://bootstrap.pypa.io/get-pip.py --output get-pip.py + curl https://bootstrap.pypa.io/2.7/get-pip.py --output get-pip.py sudo python2 ./get-pip.py # And verify the installation diff --git a/doc/developer/cross-compiling.rst b/doc/developer/cross-compiling.rst index 339e00c921..3bf78f7633 100644 --- a/doc/developer/cross-compiling.rst +++ b/doc/developer/cross-compiling.rst @@ -189,7 +189,7 @@ later on to FRR. One may get burned when compiling gRPC if the ``protoc`` version on the build machine differs from the version of ``protoc`` being linked to during a gRPC build. The error messages from this defect look like: -.. code-block:: terminal +.. code-block:: shell gens/src/proto/grpc/channelz/channelz.pb.h: In member function ‘void grpc::channelz::v1::ServerRef::set_name(const char*, size_t)’: gens/src/proto/grpc/channelz/channelz.pb.h:9127:64: error: ‘EmptyDefault’ is not a member of ‘google::protobuf::internal::ArenaStringPtr’ diff --git a/doc/developer/hooks.rst b/doc/developer/hooks.rst index 10fe6b9c43..b37a4aeea7 100644 --- a/doc/developer/hooks.rst +++ b/doc/developer/hooks.rst @@ -15,13 +15,13 @@ Example: :caption: mydaemon.h #include "hook.h" - DECLARE_HOOK(some_update_event, (struct eventinfo *info), (info)) + DECLARE_HOOK(some_update_event, (struct eventinfo *info), (info)); .. code-block:: c :caption: mydaemon.c #include "mydaemon.h" - DEFINE_HOOK(some_update_event, (struct eventinfo *info), (info)) + DEFINE_HOOK(some_update_event, (struct eventinfo *info), (info)); ... hook_call(some_update_event, info); @@ -110,9 +110,9 @@ Definition .. code-block:: c - DECLARE_HOOK(foo, (), ()) - DECLARE_HOOK(bar, (int arg), (arg)) - DECLARE_HOOK(baz, (const void *x, in_addr_t y), (x, y)) + DECLARE_HOOK(foo, (), ()); + DECLARE_HOOK(bar, (int arg), (arg)); + DECLARE_HOOK(baz, (const void *x, in_addr_t y), (x, y)); .. c:macro:: DEFINE_HOOK(name, arglist, passlist) diff --git a/doc/developer/images/PCEPlib_design.jpg b/doc/developer/images/PCEPlib_design.jpg Binary files differnew file mode 100644 index 0000000000..41aada3774 --- /dev/null +++ b/doc/developer/images/PCEPlib_design.jpg diff --git a/doc/developer/images/PCEPlib_internal_deps.jpg b/doc/developer/images/PCEPlib_internal_deps.jpg Binary files differnew file mode 100644 index 0000000000..8380021b31 --- /dev/null +++ b/doc/developer/images/PCEPlib_internal_deps.jpg diff --git a/doc/developer/images/PCEPlib_socket_comm.jpg b/doc/developer/images/PCEPlib_socket_comm.jpg Binary files differnew file mode 100644 index 0000000000..3d62a462f2 --- /dev/null +++ b/doc/developer/images/PCEPlib_socket_comm.jpg diff --git a/doc/developer/images/PCEPlib_threading_model.jpg b/doc/developer/images/PCEPlib_threading_model.jpg Binary files differnew file mode 100644 index 0000000000..afe91c2407 --- /dev/null +++ b/doc/developer/images/PCEPlib_threading_model.jpg diff --git a/doc/developer/images/PCEPlib_threading_model_frr_infra.jpg b/doc/developer/images/PCEPlib_threading_model_frr_infra.jpg Binary files differnew file mode 100644 index 0000000000..5648a9d788 --- /dev/null +++ b/doc/developer/images/PCEPlib_threading_model_frr_infra.jpg diff --git a/doc/developer/images/PCEPlib_timers.jpg b/doc/developer/images/PCEPlib_timers.jpg Binary files differnew file mode 100644 index 0000000000..a178ee9c6f --- /dev/null +++ b/doc/developer/images/PCEPlib_timers.jpg diff --git a/doc/developer/index.rst b/doc/developer/index.rst index 8e7913419f..46fd8f612e 100644 --- a/doc/developer/index.rst +++ b/doc/developer/index.rst @@ -19,4 +19,5 @@ FRRouting Developer's Guide zebra vtysh path + pceplib link-state diff --git a/doc/developer/lists.rst b/doc/developer/lists.rst index 28b21533c0..86db788c0e 100644 --- a/doc/developer/lists.rst +++ b/doc/developer/lists.rst @@ -140,7 +140,7 @@ The common setup pattern will look like this: #include <typesafe.h> - PREDECL_XXX(Z) + PREDECL_XXX(Z); struct item { int otherdata; struct Z_item mylistitem; @@ -149,20 +149,20 @@ The common setup pattern will look like this: struct Z_head mylisthead; /* unsorted: */ - DECLARE_XXX(Z, struct item, mylistitem) + DECLARE_XXX(Z, struct item, mylistitem); /* sorted, items that compare as equal cannot be added to list */ int compare_func(const struct item *a, const struct item *b); - DECLARE_XXX_UNIQ(Z, struct item, mylistitem, compare_func) + DECLARE_XXX_UNIQ(Z, struct item, mylistitem, compare_func); /* sorted, items that compare as equal can be added to list */ int compare_func(const struct item *a, const struct item *b); - DECLARE_XXX_NONUNIQ(Z, struct item, mylistitem, compare_func) + DECLARE_XXX_NONUNIQ(Z, struct item, mylistitem, compare_func); /* hash tables: */ int compare_func(const struct item *a, const struct item *b); uint32_t hash_func(const struct item *a); - DECLARE_XXX(Z, struct item, mylistitem, compare_func, hash_func) + DECLARE_XXX(Z, struct item, mylistitem, compare_func, hash_func); ``XXX`` is replaced with the name of the data structure, e.g. ``SKIPLIST`` or ``ATOMLIST``. The ``DECLARE_XXX`` invocation can either occur in a `.h` diff --git a/doc/developer/logging.rst b/doc/developer/logging.rst index cf3aa8d17f..a35e60619c 100644 --- a/doc/developer/logging.rst +++ b/doc/developer/logging.rst @@ -71,6 +71,10 @@ Extensions +-----------+--------------------------+----------------------------------------------+ | ``%pI6`` | ``struct in6_addr *`` | ``fe80::1234`` | +-----------+--------------------------+----------------------------------------------+ +| ``%pIA`` | ``struct ipaddr *`` | ``1.2.3.4`` | +| | | | +| | | ``fe80::1234`` | ++-----------+--------------------------+----------------------------------------------+ | ``%pFX`` | ``struct prefix *`` | ``fe80::1234/64`` | +-----------+--------------------------+----------------------------------------------+ | ``%pSG4`` | ``struct prefix_sg *`` | ``(*,1.2.3.4)`` | diff --git a/doc/developer/memtypes.rst b/doc/developer/memtypes.rst index 952b316ea0..08dad7fb68 100644 --- a/doc/developer/memtypes.rst +++ b/doc/developer/memtypes.rst @@ -15,15 +15,15 @@ Example: .. code-block:: c :caption: mydaemon.h - DECLARE_MGROUP(MYDAEMON) - DECLARE_MTYPE(MYNEIGHBOR) + DECLARE_MGROUP(MYDAEMON); + DECLARE_MTYPE(MYNEIGHBOR); .. code-block:: c :caption: mydaemon.c - DEFINE_MGROUP( MYDAEMON, "My daemon's memory") - DEFINE_MTYPE( MYDAEMON, MYNEIGHBOR, "Neighbor entry") - DEFINE_MTYPE_STATIC(MYDAEMON, MYNEIGHBORNAME, "Neighbor name") + DEFINE_MGROUP( MYDAEMON, "My daemon's memory"); + DEFINE_MTYPE( MYDAEMON, MYNEIGHBOR, "Neighbor entry"); + DEFINE_MTYPE_STATIC(MYDAEMON, MYNEIGHBORNAME, "Neighbor name"); struct neigh *neighbor_new(const char *name) { diff --git a/doc/developer/modules.rst b/doc/developer/modules.rst index 02330ddfe4..e95f8a1b4a 100644 --- a/doc/developer/modules.rst +++ b/doc/developer/modules.rst @@ -76,7 +76,7 @@ Basic boilerplate: .version = "0.0", .description = "my module", .init = module_init, - ) + ); The ``frr_late_init`` hook will be called after the daemon has finished its other startup and is about to enter the main event loop; this is the diff --git a/doc/developer/pceplib.rst b/doc/developer/pceplib.rst new file mode 100644 index 0000000000..774617dfe5 --- /dev/null +++ b/doc/developer/pceplib.rst @@ -0,0 +1,781 @@ +.. _pceplib: + +******* +PCEPlib +******* + +Overview +======== + +The PCEPlib is a PCEP implementation library that can be used by either a PCE +or PCC. + +Currently, only the FRR pathd has been implemented as a PCC with the PCEPlib. +The PCEPlib is able to simultaneously connect to multiple PCEP peers and can +maintain persistent PCEP connections. + + +PCEPlib compliance +================== + +The PCEPlib implements version 1 of the PCEP protocol, according to `RFC 5440 <https://tools.ietf.org/html/rfc5440>`_. + +Additionally, the PCEPlib implements the following PCEP extensions: + +- `RFC 8281 <https://tools.ietf.org/html/rfc8281>`_ PCE initiated for PCE-Initiated LSP Setup +- `RFC 8231 <https://tools.ietf.org/html/rfc8231>`_ Extensions for Stateful PCE +- `RFC 8232 <https://tools.ietf.org/html/rfc8232>`_ Optimizations of Label Switched Path State Synchronization Procedures for a Stateful PCE +- `RFC 8282 <https://tools.ietf.org/html/rfc8282>`_ Extensions to PCEP for Inter-Layer MPLS and GMPLS Traffic Engineering +- `RFC 8408 <https://tools.ietf.org/html/rfc8408>`_ Conveying Path Setup Type in PCE Communication Protocol (PCEP) Messages +- `draft-ietf-pce-segment-routing-07 <https://tools.ietf.org/html/draft-ietf-pce-segment-routing-07>`_, + `draft-ietf-pce-segment-routing-16 <https://tools.ietf.org/html/draft-ietf-pce-segment-routing-16>`_, + `RFC 8664 <https://tools.ietf.org/html/rfc8664>`_ Segment routing protocol extensions +- `RFC 7470 <https://tools.ietf.org/html/rfc7470>`_ Conveying Vendor-Specific Constraints +- `Draft-ietf-pce-association-group-10 <https://tools.ietf.org/html/draft-ietf-pce-association-group-10>`_ + Establishing Relationships Between Sets of Label Switched Paths +- `Draft-barth-pce-segment-routing-policy-cp-04 <https://tools.ietf.org/html/draft-barth-pce-segment-routing-policy-cp-04>`_ + Segment Routing Policy Candidate Paths + + +PCEPlib Architecture +==================== + +The PCEPlib is comprised of the following modules, each of which will be +detailed in the following sections. + +- **pcep_messages** + - PCEP messages, objects, and TLVs implementations + +- **pcep_pcc** + - PCEPlib public PCC API with a sample PCC binary + +- **pcep_session_logic** + - PCEP Session handling + +- **pcep_socket_comm** + - Socket communications + +- **pcep_timers** + - PCEP timers + +- **pcep_utils** + - Internal utilities used by the PCEPlib modules. + +The interaction of these modules can be seen in the following diagram. + +PCEPlib Architecture: + +.. image:: images/PCEPlib_design.jpg + + +PCEP Session Logic library +-------------------------- + +The PCEP Session Logic library orchestrates calls to the rest of the PCC libraries. + +PCEP Session Logic library responsibilities: + +- Handle messages received from "PCEP Socket Comm" +- Create and manage "PCEP Session" objects +- Set timers and react to timer expirations +- Manage counters + +The PCEP Session Logic library will have 2 main triggers controlled by a +pthread condition variable: + +- Timer expirations - ``on_timer_expire()`` callback +- Messages received from PCEP SocketComm - ``message_received()`` callback + +The counters are created and managed using the ``pcep_utils/pcep_utils_counters.h`` +counters library. The following are the different counter groups managed: + +- **COUNTER_SUBGROUP_ID_RX_MSG** +- **COUNTER_SUBGROUP_ID_TX_MSG** +- **COUNTER_SUBGROUP_ID_RX_OBJ** +- **COUNTER_SUBGROUP_ID_TX_OBJ** +- **COUNTER_SUBGROUP_ID_RX_SUBOBJ** +- **COUNTER_SUBGROUP_ID_TX_SUBOBJ** +- **COUNTER_SUBGROUP_ID_RX_RO_SR_SUBOBJ** +- **COUNTER_SUBGROUP_ID_TX_RO_SR_SUBOBJ** +- **COUNTER_SUBGROUP_ID_RX_TLV** +- **COUNTER_SUBGROUP_ID_TX_TLV** +- **COUNTER_SUBGROUP_ID_EVENT** + +The counters can be obtained and reset as explained later in the PCEPlib PCC API. + +PCEP Socket Comm library +------------------------ + +PCEP communication can be configured to be handled internally in this simple +library. When this library is instantiated by the PCEP Session Logic, callbacks +are provided to handle received messages and error conditions. + +The following diagram illustrates how the library works. + +PCEPlib Socket Comm: + +.. image:: images/PCEPlib_socket_comm.jpg + + +PCEP Timers library +------------------- + +Timers can be configured to be handled internally by this library. When this +library is instantiated by the PCEP Session Logic, callbacks are provided to +ha:0 +ndle timer expirations. The following timers are implemented and handled, +according to `RFC 5440 <https://tools.ietf.org/html/rfc5440>`_. + +- Open KeepWait (fixed at 60 seconds) + - Set once the PCC sends an Open, and if it expires before receiving a KeepAlive or PCErr, then the PCC should send a PCErr and close the TCP connection + +- Keepalive timer + - How often the PCC should send Keepalive messages to the PCE (and vice-versa) + - The timer will be reset after any message is sent: any message serves as a Keepalive + +- DeadTimer + - If no messages are received before expiration, the session is declared as down + - Reset everytime any message is received + +- PCReq request timer + - How long the PCC waits for the PCE to reply to PCReq messages. + +PCEPlib Timers: + +.. image:: images/PCEPlib_timers.jpg + + +PCEP Messages library +--------------------- + +The PCEP Messages library has all of the implemented PCEP messages, objects, +TLVs, and related functionality. + +The following header files can be used for creating and handling received PCEP +entities. + +- pcep-messages.h +- pcep-objects.h +- pcep-tlvs.h + + +PCEP Messages ++++++++++++++ + +The following PCEP messages can be created and received: + +- ``struct pcep_message* pcep_msg_create_open(...);`` +- ``struct pcep_message* pcep_msg_create_open_with_tlvs(...);`` +- ``struct pcep_message* pcep_msg_create_request(...);`` +- ``struct pcep_message* pcep_msg_create_request_ipv6(...);`` +- ``struct pcep_message* pcep_msg_create_reply(...);`` +- ``struct pcep_message* pcep_msg_create_close(...);`` +- ``struct pcep_message* pcep_msg_create_error(...);`` +- ``struct pcep_message* pcep_msg_create_error_with_objects(...);`` +- ``struct pcep_message* pcep_msg_create_keepalive(...);`` +- ``struct pcep_message* pcep_msg_create_report(...);`` +- ``struct pcep_message* pcep_msg_create_update(...);`` +- ``struct pcep_message* pcep_msg_create_initiate(...);`` + +Refer to ``pcep_messages/include/pcep-messages.h`` and the API section +below for more details. + + +PCEP Objects +++++++++++++ + +The following PCEP objects can be created and received: + +- ``struct pcep_object_open* pcep_obj_create_open(...);`` +- ``struct pcep_object_rp* pcep_obj_create_rp(...);`` +- ``struct pcep_object_notify* pcep_obj_create_notify(...);`` +- ``struct pcep_object_nopath* pcep_obj_create_nopath(...);`` +- ``struct pcep_object_association_ipv4* pcep_obj_create_association_ipv4(...);`` +- ``struct pcep_object_association_ipv6* pcep_obj_create_association_ipv6(...);`` +- ``struct pcep_object_endpoints_ipv4* pcep_obj_create_endpoint_ipv4(...);`` +- ``struct pcep_object_endpoints_ipv6* pcep_obj_create_endpoint_ipv6(...);`` +- ``struct pcep_object_bandwidth* pcep_obj_create_bandwidth(...);`` +- ``struct pcep_object_metric* pcep_obj_create_metric(...);`` +- ``struct pcep_object_lspa* pcep_obj_create_lspa(...);`` +- ``struct pcep_object_svec* pcep_obj_create_svec(...);`` +- ``struct pcep_object_error* pcep_obj_create_error(...);`` +- ``struct pcep_object_close* pcep_obj_create_close(...);`` +- ``struct pcep_object_srp* pcep_obj_create_srp(...);`` +- ``struct pcep_object_lsp* pcep_obj_create_lsp(...);`` +- ``struct pcep_object_vendor_info* pcep_obj_create_vendor_info(...);`` +- ``struct pcep_object_ro* pcep_obj_create_ero(...);`` +- ``struct pcep_object_ro* pcep_obj_create_rro(...);`` +- ``struct pcep_object_ro* pcep_obj_create_iro(...);`` +- ``struct pcep_ro_subobj_ipv4* pcep_obj_create_ro_subobj_ipv4(...);`` +- ``struct pcep_ro_subobj_ipv6* pcep_obj_create_ro_subobj_ipv6(...);`` +- ``struct pcep_ro_subobj_unnum* pcep_obj_create_ro_subobj_unnum(...);`` +- ``struct pcep_ro_subobj_32label* pcep_obj_create_ro_subobj_32label(...);`` +- ``struct pcep_ro_subobj_asn* pcep_obj_create_ro_subobj_asn(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_nonai(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv4_node(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv6_node(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv4_adj(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_ipv6_adj(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_unnumbered_ipv4_adj(...);`` +- ``struct pcep_ro_subobj_sr* pcep_obj_create_ro_subobj_sr_linklocal_ipv6_adj(...);`` + +Refer to ``pcep_messages/include/pcep-objects.h`` and the API section +below for more details. + + +PCEP TLVs ++++++++++ + +The following PCEP TLVs (Tag, Length, Value) can be created and received: + +- Open Object TLVs + - ``struct pcep_object_tlv_stateful_pce_capability* pcep_tlv_create_stateful_pce_capability(...);`` + - ``struct pcep_object_tlv_lsp_db_version* pcep_tlv_create_lsp_db_version(...);`` + - ``struct pcep_object_tlv_speaker_entity_identifier* pcep_tlv_create_speaker_entity_id(...);`` + - ``struct pcep_object_tlv_path_setup_type* pcep_tlv_create_path_setup_type(...);`` + - ``struct pcep_object_tlv_path_setup_type_capability* pcep_tlv_create_path_setup_type_capability(...);`` + - ``struct pcep_object_tlv_sr_pce_capability* pcep_tlv_create_sr_pce_capability(...);`` + +- LSP Object TLVs + - ``struct pcep_object_tlv_ipv4_lsp_identifier* pcep_tlv_create_ipv4_lsp_identifiers(...);`` + - ``struct pcep_object_tlv_ipv6_lsp_identifier* pcep_tlv_create_ipv6_lsp_identifiers(...);`` + - ``struct pcep_object_tlv_symbolic_path_name* pcep_tlv_create_symbolic_path_name(...);`` + - ``struct pcep_object_tlv_lsp_error_code* pcep_tlv_create_lsp_error_code(...);`` + - ``struct pcep_object_tlv_rsvp_error_spec* pcep_tlv_create_rsvp_ipv4_error_spec(...);`` + - ``struct pcep_object_tlv_rsvp_error_spec* pcep_tlv_create_rsvp_ipv6_error_spec(...);`` + - ``struct pcep_object_tlv_nopath_vector* pcep_tlv_create_nopath_vector(...);`` + - ``struct pcep_object_tlv_vendor_info* pcep_tlv_create_vendor_info(...);`` + - ``struct pcep_object_tlv_arbitrary* pcep_tlv_create_tlv_arbitrary(...);`` + +- SRPAG (SR Association Group) TLVs + - ``struct pcep_object_tlv_srpag_pol_id *pcep_tlv_create_srpag_pol_id_ipv4(...);`` + - ``struct pcep_object_tlv_srpag_pol_id *pcep_tlv_create_srpag_pol_id_ipv6(...);`` + - ``struct pcep_object_tlv_srpag_pol_name *pcep_tlv_create_srpag_pol_name(...);`` + - ``struct pcep_object_tlv_srpag_cp_id *pcep_tlv_create_srpag_cp_id(...);`` + - ``struct pcep_object_tlv_srpag_cp_pref *pcep_tlv_create_srpag_cp_pref(...);`` + +Refer to ``pcep_messages/include/pcep-tlvs.h`` and the API section +below for more details. + + +PCEP PCC +-------- + +This module has a Public PCC API library (explained in detail later) and a +sample PCC binary. The APIs in this library encapsulate other PCEPlib libraries +for simplicity. With this API, the PCEPlib PCC can be started and stopped, and +the PCEPlib event queue can be accessed. The PCEP Messages library is not +encapsulated, and should be used directly. + + +Internal Dependencies +--------------------- + +The following diagram illustrates the internal PCEPlib library dependencies. + +PCEPlib internal dependencies: + +.. image:: images/PCEPlib_internal_deps.jpg + + +External Dependencies +--------------------- + +Originally the PCEPlib was based on the open source `libpcep project <https://www.acreo.se/open-software-libpcep>`_, +but that dependency has been reduced to just one source file (pcep-tools.[ch]). + + +PCEPlib Threading model +----------------------- + +The PCEPlib can be run in stand-alone mode whereby a thread is launched for +timers and socket comm, as is illustrated in the following diagram. + +PCEPlib Threading model: + +.. image:: images/PCEPlib_threading_model.jpg + +The PCEPlib can also be configured to use an external timers and socket +infrastructure like the FRR threads and tasks. In this case, no internal +threads are launched for timers and socket comm, as is illustrated in the +following diagram. + +PCEPlib Threading model with external infra: + +.. image:: images/PCEPlib_threading_model_frr_infra.jpg + + +Building +-------- + +The autotools build system is used and integrated with the frr build system. + +Testing +------- + +The Unit Tests for an individual library are executed with the ``make check`` +command. The Unit Test binary will be written to the project ``build`` directory. +All Unit Tests are executed with Valgrind, and any memory issues reported by +Valgrind will cause the Unit Test to fail. + + +PCEPlib PCC API +=============== + +The following sections describe the PCEPlib PCC API. + + +PCEPlib PCC Initialization and Destruction +------------------------------------------ + +The PCEPlib can be initialized to handle memory, timers, and socket comm +internally in what is called stand-alone mode, or with an external +infrastructure, like FRR. + +PCEPlib PCC Initialization and Destruction in stand-alone mode +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +PCEPlib PCC initialization and destruction functions: + +- ``bool initialize_pcc();`` +- ``bool initialize_pcc_wait_for_completion();`` +- ``bool destroy_pcc();`` + +The PCC can be initialized with either ``initialize_pcc()`` or +``initialize_pcc_wait_for_completion()``. + +- ``initialize_pcc_wait_for_completion()`` blocks until ``destroy_pcc()`` + is called from a separate pthread. +- ``initialize_pcc()`` is non-blocking and will be stopped when + ``destroy_pcc()`` is called. + +Both initialize functions will launch 3 pthreads: + +- 1 Timer pthread +- 1 SocketComm pthread +- 1 SessionLogic pthread + +When ``destroy_pcc()`` is called, all pthreads will be stopped and all +resources will be released. + +All 3 functions return true upon success, and false otherwise. + +PCEPlib PCC Initialization and Destruction with FRR infrastructure +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + +PCEPlib PCC initialization and destruction functions: + +- ``bool initialize_pcc_infra(struct pceplib_infra_config *infra_config);`` +- ``bool destroy_pcc();`` + +The ``pceplib_infra_config`` struct has the following fields: + +- **void *pceplib_infra_mt** + - FRR Memory type pointer for infra related memory management + +- **void *pceplib_messages_mt** + - FRR Memory type pointer for PCEP messages related memory management + +- **pceplib_malloc_func mfunc** + - FRR malloc function pointer + +- **pceplib_calloc_func cfunc** + - FRR calloc function pointer + +- **pceplib_realloc_func rfunc** + - FRR realloc function pointer + +- **pceplib_strdup_func sfunc** + - FRR strdup function pointer + +- **pceplib_free_func ffunc** + - FRR free function pointer + +- **void *external_infra_data** + - FRR data used by FRR timers and sockets infrastructure + +- **ext_timer_create timer_create_func** + - FRR timer create function pointer + +- **ext_timer_cancel timer_cancel_func** + - FRR timer cancel function pointer + +- **ext_socket_write socket_write_func** + - FRR socket write function pointer, indicating fd is ready to be written to + +- **ext_socket_read socket_read_func** + - FRR socket write function pointer, indicating fd is ready to be read from + + +PCEPlib PCC configuration +------------------------- + +PCEPlib PCC configuratoin functions: + +- ``pcep_configuration *create_default_pcep_configuration();`` +- ``void destroy_pcep_configuration(pcep_configuration *config);`` + +A ``pcep_configuration`` object with default values is created with +``create_default_pcep_configuration()``. These values can be tailored to +specific use cases. + +Created ``pcep_configuration`` objects are destroyed with +``destroy_pcep_configuration()``. + + +PCEPlib PCC configuration paramaters +++++++++++++++++++++++++++++++++++++ + +The ``pcep_configuration`` object is defined in ``pcep_session_logic/include/pcep_session_logic.h`` +The attributes in the ``pcep_configuration`` object are detailed as follows. + +PCEP Connection parameters: + +- **dst_pcep_port** + - Defaults to 0, in which case the default PCEP TCP destination port + 4189 will be used. + - Set to use a specific PCEP TCP destination port. + +- **src_pcep_port** + - Defaults to 0, in which case the default PCEP TCP source port + 4189 will be used. + - Set to use a specific PCEP TCP source port. + +- **Source IP** + - Defaults to IPv4 INADDR_ANY + - Set **src_ip.src_ipv4** and **is_src_ipv6=false** to set the source IPv4. + - Set **src_ip.src_ipv6** and **is_src_ipv6=true** to set the source IPv6. + +- **socket_connect_timeout_millis** + - Maximum amount of time to wait to connect to the PCE TCP socket + before failing, in milliseconds. + +PCEP Versioning: + +- **pcep_msg_versioning->draft_ietf_pce_segment_routing_07** + - Defaults to false, in which case draft 16 versioning will be used. + - Set to true to use draft 07 versioning. + +PCEP Open Message Parameters: + +- **keep_alive_seconds** + - Sent to PCE in PCEP Open Msg + - Recommended value = 30, Minimum value = 1 + - Disabled by setting value = 0 + +- **dead_timer_seconds** + - Sent to PCE in PCEP Open Msg + - Recommended value = 4 * keepalive timer value + +- Supported value ranges for PCEP Open Message received from the PCE + - **min_keep_alive_seconds**, **max_keep_alive_seconds** + - **min_dead_timer_seconds**, **max_dead_timer_seconds** + +- **request_time_seconds** + - When a PCC sends a PcReq to a PCE, the amount of time a PCC will + wait for a PcRep reply from the PCE. + +- **max_unknown_requests** + - If a PCC/PCE receives PCRep/PCReq messages with unknown requests + at a rate equal or greater than MAX-UNKNOWN-REQUESTS per minute, + the PCC/PCE MUST send a PCEP CLOSE message. + - Recommended value = 5 + +- **max_unknown_messages** + - If a PCC/PCE receives unrecognized messages at a rate equal or + greater than MAX-UNKNOWN-MESSAGES per minute, the PCC/PCE MUST + send a PCEP CLOSE message + - Recommended value = 5 + +Stateful PCE Capability TLV configuration parameters (RFC 8231, 8232, 8281, and +draft-ietf-pce-segment-routing-16): + +- **support_stateful_pce_lsp_update** + - If this flag is true, then a Stateful PCE Capability TLV will + be added to the PCEP Open object, with the LSP Update Capability + U-flag set true. + - The rest of these parameters are used to configure the Stateful + PCE Capability TLV + +- **support_pce_lsp_instantiation** + - Sets the I-flag true, indicating the PCC allows instantiation + of an LSP by a PCE. + +- **support_include_db_version** + - Sets the S-bit true, indicating the PCC will include the + LSP-DB-VERSION TLV in each LSP object. See lsp_db_version below. + +- **support_lsp_triggered_resync** + - Sets the T-bit true, indicating the PCE can trigger resynchronization + of LSPs at any point in the life of the session. + +- **support_lsp_delta_sync** + - Sets the D-bit true, indicating the PCEP speaker allows incremental + (delta) State Synchronization. + +- **support_pce_triggered_initial_sync** + - Sets the F-bit true, indicating the PCE SHOULD trigger initial (first) + State Synchronization + +LSP DB Version TLV configuration parameters: + +- **lsp_db_version** + - If this parameter has a value other than 0, and the above + support_include_db_version flag is true, then an LSP DB + Version TLV will be added to the PCEP Open object. + - This parameter should only be set if LSP-DB survived a restart + and is available. + - This value will be copied over to the pcep_session upon initialization. + +SR PCE Capability sub-TLV configuration parameters (draft-ietf-pce-segment-routing-16): + +- **support_sr_te_pst** + - If this flag is true, then an SR PCE Capability sub-TLV will be + added to a Path Setup type Capability TLV, which will be added + to the PCEP Open object. + - The PST used in the Path Setup type Capability will be 1, + indicating the Path is setup using Segment Routing Traffic Engineering. + +Only set the following fields if the **support_sr_te_pst** flag is true. + +- **pcc_can_resolve_nai_to_sid** + - Sets the N-flag true, indicating that the PCC is capable of resolving + a Node or Adjacency Identifier to a SID + +- **max_sid_depth** + - If set other than 0, then the PCC imposes a limit on the Maximum + SID depth. + - If this parameter is other than 0, then the X bit will be true, + and the parameter value will be set in the MSD field. + + +PCEPlib PCC connections +----------------------- + +PCEPlib PCC connect and disconnect functions: + +- ``pcep_session *connect_pce(pcep_configuration *config, struct in_addr *pce_ip);`` +- ``pcep_session *connect_pce_ipv6(pcep_configuration *config, struct in6_addr *pce_ip);`` +- ``void disconnect_pce(pcep_session *session);`` + +When connecting to a PCE, a ``pcep_session`` will be returned on success, NULL +otherwise. + +Refer to the above PCC configuration parameters section for setting the source +and destination PCEP TCP ports, and the source IP address and version. + + +PCEP Messages, Objects, and TLVs +-------------------------------- + +The PCEP messages, objects, and TLVs created in the PCEPlib are high-level API +structures, meaning they need to be encoded before being sent on-the-wire, and +the raw data received needs to be decoded into these structures. This makes +using these objects much easier for the library consumer, since they do not +need to know the detailed raw format of the PCEP entities. + + +PCEP Messages ++++++++++++++ + +Received messages (in the ``pcep_event`` explained below) are of type +``pcep_message``, which have the following fields: + +- ``struct pcep_message_header *msg_header;`` + - Defines the PCEP version and message type + +- ``double_linked_list *obj_list;`` + - A double linked list of the message objects + - Each entry is a pointer to a ``struct pcep_object_header``, and + using the ``object_class`` and ``object_type`` fields, the pointer + can be cast to the appropriate object structure to access the + rest of the object fields + +- ``uint8_t *encoded_message;`` + - This field is only populated for received messages or once the + ``pcep_encode_message()`` function has been called on the message. + - This field is a pointer to the raw PCEP data for the entire + message, including all objects and TLVs. + +- ``uint16_t encoded_message_length;`` + - This field is only populated for received messages or once the + ``pcep_encode_message()`` function has been called on the message. + - This field is the length of the entire raw message, including + all objects and TLVs. + - This field is in host byte order. + + +PCEP Objects +++++++++++++ + +A PCEP message has a double linked list of pointers to ``struct pcep_object_header`` +structures, which have the following fields: + +- ``enum pcep_object_classes object_class;`` +- ``enum pcep_object_types object_type;`` +- ``bool flag_p;`` + - PCC Processing rule bit: When set, the object MUST be taken into + account, when cleared the object is optional + +- ``bool flag_i;`` + - PCE Ignore bit: indicates to a PCC whether or not an optional + object was processed + +- ``double_linked_list *tlv_list;`` + - A double linked list of the object TLVs + - Each entry is a pointer to a ``struct pcep_object_tlv_header``, and + using the TLV type field, the pointer can be cast to the + appropriate TLV structure to access the rest of the TLV fields + +- ``uint8_t *encoded_object;`` + - This field is only populated for received objects or once the + ``pcep_encode_object()`` (called by ``pcep_encode_message()``) + function has been called on the object. + - Pointer into the encoded_message field (from the pcep_message) + where the raw object PCEP data starts. + +- ``uint16_t encoded_object_length;`` + - This field is only populated for received objects or once the + ``pcep_encode_object()`` (called by ``pcep_encode_message()``) + function has been called on the object. + - This field is the length of the entire raw TLV + - This field is in host byte order. + +The object class and type can be used to cast the ``struct pcep_object_header`` +pointer to the appropriate object structure so the specific object fields can +be accessed. + + +PCEP TLVs ++++++++++ + +A PCEP object has a double linked list of pointers to ``struct pcep_object_tlv_header`` +structures, which have the following fields: + +- ``enum pcep_object_tlv_types type;`` +- ``uint8_t *encoded_tlv;`` + - This field is only populated for received TLVs or once the + ``pcep_encode_tlv()`` (called by ``pcep_encode_message()``) + function has been called on the TLV. + - Pointer into the encoded_message field (from the pcep_message) + where the raw TLV PCEP data starts. + +- ``uint16_t encoded_tlv_length;`` + - This field is only populated for received TLVs or once the + ``pcep_encode_tlv()`` (called by ``pcep_encode_message()``) + function has been called on the TLV. + - This field is the length of the entire raw TLV + - This field is in host byte order. + + +Memory management ++++++++++++++++++ + +Any of the PCEPlib Message Library functions that receive a pointer to a +``double_linked_list``, ``pcep_object_header``, or ``pcep_object_tlv_header``, +transfer the ownership of the entity to the PCEPlib. The memory will be freed +internally when the encapsulating structure is freed. If the memory for any of +these is freed by the caller, then there will be a double memory free error +when the memory is freed internally in the PCEPlib. + +Any of the PCEPlib Message Library functions that receive either a pointer to a +``struct in_addr`` or ``struct in6_addr`` will allocate memory for the IP +address internally and copy the IP address. It is the responsibility of the +caller to manage the memory for the IP address passed into the PCEPlib Message +Library functions. + +For messages received via the event queue (explained below), the message will +be freed when the event is freed by calling ``destroy_pcep_event()``. + +When sending messages, the message will be freed internally in the PCEPlib when +the ``send_message()`` ``pcep_pcc`` API function when the ``free_after_send`` flag +is set true. + +To manually delete a message, call the ``pcep_msg_free_message()`` function. +Internally, this will call ``pcep_obj_free_object()`` and ``pcep_obj_free_tlv()`` +appropriately. + + +Sending a PCEP Report message +----------------------------- + +This section shows how to send a PCEP Report messages from the PCC to the PCE, +and serves as an example of how to send other messages. Refer to the sample +PCC binary located in ``pcep_pcc/src/pcep_pcc.c`` for code examples os sending +a PCEP Report message. + +The Report message must have at least an SRP, LSP, and ERO object. + +The PCEP Report message objects are created with the following APIs: + +- ``struct pcep_object_srp *pcep_obj_create_srp(...);`` +- ``struct pcep_object_lsp *pcep_obj_create_lsp(...);`` +- ``struct pcep_object_ro *pcep_obj_create_ero(...);`` + - Create ero subobjects with the ``pcep_obj_create_ro_subobj_*(...);`` functions + +PCEP Report message is created with the following API: + +- ``struct pcep_header *pcep_msg_create_report(double_linked_list *report_object_list);`` + +A PCEP report messages is sent with the following API: + +- ``void send_message(pcep_session *session, pcep_message *message, bool free_after_send);`` + + +PCEPlib Received event queue +---------------------------- + +PCEP events and messages of interest to the PCEPlib consumer will be stored +internally in a message queue for retrieval. + +The following are the event types: + +- **MESSAGE_RECEIVED** +- **PCE_CLOSED_SOCKET** +- **PCE_SENT_PCEP_CLOSE** +- **PCE_DEAD_TIMER_EXPIRED** +- **PCE_OPEN_KEEP_WAIT_TIMER_EXPIRED** +- **PCC_CONNECTED_TO_PCE** +- **PCC_CONNECTION_FAILURE** +- **PCC_PCEP_SESSION_CLOSED** +- **PCC_RCVD_INVALID_OPEN** +- **PCC_SENT_INVALID_OPEN** +- **PCC_RCVD_MAX_INVALID_MSGS** +- **PCC_RCVD_MAX_UNKOWN_MSGS** + +The following PCEP messages will not be posted on the message queue, as they +are handled internally in the library: + +- **Open** +- **Keep Alive** +- **Close** + +Received event queue API: + +- ``bool event_queue_is_empty();`` + - Returns true if the queue is empty, false otherwise + +- ``uint32_t event_queue_num_events_available();`` + - Return the number of events on the queue, 0 if empty + +- ``struct pcep_event *event_queue_get_event();`` + - Return the next event on the queue, NULL if empty + - The ``message`` pointer will only be non-NULL if ``event_type`` + is ``MESSAGE_RECEIVED`` + +- ``void destroy_pcep_event(struct pcep_event *event);`` + - Free the PCEP Event resources, including the PCEP message if present + + +PCEPlib Counters +---------------- + +The PCEPlib counters are managed in the ``pcep_session_logic`` library, and can +be accessed in the ``pcep_session_counters`` field of the ``pcep_session`` structure. +There are 2 API functions to manage the counters: + +- ``void dump_pcep_session_counters(pcep_session *session);`` + - Dump all of the counters to the logs + +- ``void reset_pcep_session_counters(pcep_session *session);`` + diff --git a/doc/developer/scripting.rst b/doc/developer/scripting.rst index b0413619ab..708f65ff7d 100644 --- a/doc/developer/scripting.rst +++ b/doc/developer/scripting.rst @@ -271,7 +271,7 @@ has). For example, here is the encoder function for ``struct prefix``: This function pushes a single value onto the Lua stack. It is a table whose equivalent in Lua is: -.. code-block:: +.. code-block:: c { ["network"] = "1.2.3.4/24", ["prefixlen"] = 24, ["family"] = 2 } diff --git a/doc/developer/subdir.am b/doc/developer/subdir.am index f7e4486ef0..d16420c7e7 100644 --- a/doc/developer/subdir.am +++ b/doc/developer/subdir.am @@ -57,6 +57,7 @@ dev_RSTFILES = \ doc/developer/tracing.rst \ doc/developer/testing.rst \ doc/developer/topotests-snippets.rst \ + doc/developer/topotests-markers.rst \ doc/developer/topotests.rst \ doc/developer/workflow.rst \ doc/developer/xrefs.rst \ diff --git a/doc/developer/topotests.rst b/doc/developer/topotests.rst index 93d81548b2..4f58ee335b 100644 --- a/doc/developer/topotests.rst +++ b/doc/developer/topotests.rst @@ -60,6 +60,7 @@ following steps will get you there on Ubuntu 20.04. .. code:: shell + apt install libsnmp-dev apt install snmpd snmp apt install snmp-mibs-downloader download-mibs @@ -388,11 +389,19 @@ This is the recommended test writing routine: - Format the new code using `black <https://github.com/psf/black>`_ - Create a Pull Request -.. Note:: +Some things to keep in mind: + +- BGP tests MUST use generous convergence timeouts - you must ensure + that any test involving BGP uses a convergence timeout of at least + 130 seconds. +- Topotests are run on a range of Linux versions: if your test + requires some OS-specific capability (like mpls support, or vrf + support), there are test functions available in the libraries that + will help you determine whether your test should run or be skipped. +- Avoid including unstable data in your test: don't rely on link-local + addresses or ifindex values, for example, because these can change + from run to run. - BGP tests MUST use generous convergence timeouts - you must ensure - that any test involving BGP uses a convergence timeout of at least - 130 seconds. Topotest File Hierarchy """"""""""""""""""""""" @@ -795,7 +804,7 @@ Requirements: - Use `black <https://github.com/psf/black>`_ code formatter before creating a pull request. This ensures we have a unified code style. - Mark test modules with pytest markers depending on the daemons used during the - tests (s. Markers) + tests (see :ref:`topotests-markers`) Tips: diff --git a/doc/developer/tracing.rst b/doc/developer/tracing.rst index 71f7b4ac49..ae4d621a8e 100644 --- a/doc/developer/tracing.rst +++ b/doc/developer/tracing.rst @@ -62,113 +62,113 @@ use your tracer as usual. To see available USDT probes:: - readelf -n /usr/lib/frr/bgpd + readelf -n /usr/lib/frr/bgpd Example:: - root@host ~> readelf -n /usr/lib/frr/bgpd - - Displaying notes found in: .note.ABI-tag - Owner Data size Description - GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag) - OS: Linux, ABI: 3.2.0 - - Displaying notes found in: .note.gnu.build-id - Owner Data size Description - GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) - Build ID: 4f42933a69dcb42a519bc459b2105177c8adf55d - - Displaying notes found in: .note.stapsdt - Owner Data size Description - stapsdt 0x00000045 NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: packet_read - Location: 0x000000000045ee48, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-96(%rbp) 8@-104(%rbp) - stapsdt 0x00000047 NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: open_process - Location: 0x000000000047c43b, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-224(%rbp) 2@-226(%rbp) - stapsdt 0x00000049 NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: update_process - Location: 0x000000000047c4bf, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-208(%rbp) 2@-210(%rbp) - stapsdt 0x0000004f NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: notification_process - Location: 0x000000000047c557, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-192(%rbp) 2@-194(%rbp) - stapsdt 0x0000004c NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: keepalive_process - Location: 0x000000000047c5db, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-176(%rbp) 2@-178(%rbp) - stapsdt 0x0000004a NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: refresh_process - Location: 0x000000000047c673, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-160(%rbp) 2@-162(%rbp) - stapsdt 0x0000004d NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: capability_process - Location: 0x000000000047c6f7, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-144(%rbp) 2@-146(%rbp) - stapsdt 0x0000006f NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: output_filter - Location: 0x000000000048e33a, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-144(%rbp) 8@-152(%rbp) 4@-156(%rbp) 4@-160(%rbp) 8@-168(%rbp) - stapsdt 0x0000007d NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: process_update - Location: 0x0000000000491f10, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-800(%rbp) 8@-808(%rbp) 4@-812(%rbp) 4@-816(%rbp) 4@-820(%rbp) 8@-832(%rbp) - stapsdt 0x0000006e NT_STAPSDT (SystemTap probe descriptors) - Provider: frr_bgp - Name: input_filter - Location: 0x00000000004940ed, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 - Arguments: 8@-144(%rbp) 8@-152(%rbp) 4@-156(%rbp) 4@-160(%rbp) 8@-168(%rbp) + root@host ~> readelf -n /usr/lib/frr/bgpd + + Displaying notes found in: .note.ABI-tag + Owner Data size Description + GNU 0x00000010 NT_GNU_ABI_TAG (ABI version tag) + OS: Linux, ABI: 3.2.0 + + Displaying notes found in: .note.gnu.build-id + Owner Data size Description + GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring) + Build ID: 4f42933a69dcb42a519bc459b2105177c8adf55d + + Displaying notes found in: .note.stapsdt + Owner Data size Description + stapsdt 0x00000045 NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: packet_read + Location: 0x000000000045ee48, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-96(%rbp) 8@-104(%rbp) + stapsdt 0x00000047 NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: open_process + Location: 0x000000000047c43b, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-224(%rbp) 2@-226(%rbp) + stapsdt 0x00000049 NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: update_process + Location: 0x000000000047c4bf, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-208(%rbp) 2@-210(%rbp) + stapsdt 0x0000004f NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: notification_process + Location: 0x000000000047c557, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-192(%rbp) 2@-194(%rbp) + stapsdt 0x0000004c NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: keepalive_process + Location: 0x000000000047c5db, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-176(%rbp) 2@-178(%rbp) + stapsdt 0x0000004a NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: refresh_process + Location: 0x000000000047c673, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-160(%rbp) 2@-162(%rbp) + stapsdt 0x0000004d NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: capability_process + Location: 0x000000000047c6f7, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-144(%rbp) 2@-146(%rbp) + stapsdt 0x0000006f NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: output_filter + Location: 0x000000000048e33a, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-144(%rbp) 8@-152(%rbp) 4@-156(%rbp) 4@-160(%rbp) 8@-168(%rbp) + stapsdt 0x0000007d NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: process_update + Location: 0x0000000000491f10, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-800(%rbp) 8@-808(%rbp) 4@-812(%rbp) 4@-816(%rbp) 4@-820(%rbp) 8@-832(%rbp) + stapsdt 0x0000006e NT_STAPSDT (SystemTap probe descriptors) + Provider: frr_bgp + Name: input_filter + Location: 0x00000000004940ed, Base: 0x00000000005a09d2, Semaphore: 0x0000000000000000 + Arguments: 8@-144(%rbp) 8@-152(%rbp) 4@-156(%rbp) 4@-160(%rbp) 8@-168(%rbp) To see available LTTng probes, run the target, create a session and then:: - lttng list --userspace | grep frr + lttng list --userspace | grep frr Example:: - root@host ~> lttng list --userspace | grep frr - PID: 11157 - Name: /usr/lib/frr/bgpd - frr_libfrr:route_node_get (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:list_sort (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:list_delete_node (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:list_remove (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:list_add (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:memfree (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:memalloc (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:frr_pthread_stop (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:frr_pthread_run (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) - frr_libfrr:thread_call (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:thread_cancel_async (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:thread_cancel (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:schedule_write (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:schedule_read (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:schedule_event (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:schedule_timer (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:hash_release (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:hash_insert (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_libfrr:hash_get (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:output_filter (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:input_filter (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:process_update (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:packet_read (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:refresh_process (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:capability_process (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:notification_process (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:update_process (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:keepalive_process (loglevel: TRACE_INFO (6)) (type: tracepoint) - frr_bgp:open_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + root@host ~> lttng list --userspace | grep frr + PID: 11157 - Name: /usr/lib/frr/bgpd + frr_libfrr:route_node_get (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_sort (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_delete_node (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_remove (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:list_add (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:memfree (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:memalloc (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:frr_pthread_stop (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:frr_pthread_run (loglevel: TRACE_DEBUG_LINE (13)) (type: tracepoint) + frr_libfrr:thread_call (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:thread_cancel_async (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:thread_cancel (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_write (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_read (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_event (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:schedule_timer (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:hash_release (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:hash_insert (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_libfrr:hash_get (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:output_filter (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:input_filter (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:process_update (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:packet_read (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:refresh_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:capability_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:notification_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:update_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:keepalive_process (loglevel: TRACE_INFO (6)) (type: tracepoint) + frr_bgp:open_process (loglevel: TRACE_INFO (6)) (type: tracepoint) When using LTTng, you can also get zlogs as trace events by enabling the ``lttng_ust_tracelog:*`` event class. diff --git a/doc/developer/workflow.rst b/doc/developer/workflow.rst index 71e2b00448..abdbea5a9c 100644 --- a/doc/developer/workflow.rst +++ b/doc/developer/workflow.rst @@ -1229,9 +1229,9 @@ towards making documentation easier to use, write and maintain. CLI Commands ^^^^^^^^^^^^ -When documenting CLI please use a combination of the ``.. index::`` and -``.. clicmd::`` directives. For example, the command :clicmd:`show pony` would -be documented as follows: +When documenting CLI please use the ``.. clicmd::`` directive. This directive +will format the command and generate index entries automatically. For example, +the command :clicmd:`show pony` would be documented as follows: .. code-block:: rest @@ -1252,6 +1252,7 @@ be documented as follows: hjw |_>|> /_] // /_] /_] + When documented this way, CLI commands can be cross referenced with the ``:clicmd:`` inline markup like so: @@ -1262,8 +1263,27 @@ When documented this way, CLI commands can be cross referenced with the This is very helpful for users who want to quickly remind themselves what a particular command does. -When documenting a cli that has a ``no`` form, please do not include -the ``no`` in the ``.. index::`` line. +When documenting a cli that has a ``no`` form, please do not include the ``no`` +form. I.e. ``no show pony`` would not be documented anywhere. Since most +commands have ``no`` forms, users should be able to infer these or get help +from vtysh's completions. + +When documenting commands that have lots of possible variants, just document +the single command in summary rather than enumerating each possible variant. +E.g. for ``show pony [foo|bar]``, do not: + +.. code-block:: rest + + .. clicmd:: show pony + .. clicmd:: show pony foo + .. clicmd:: show pony bar + +Do: + +.. code-block:: rest + + .. clicmd:: show pony [foo|bar] + Configuration Snippets ^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/developer/xrefs.rst b/doc/developer/xrefs.rst index 6a0794d41b..e8e07dfe1d 100644 --- a/doc/developer/xrefs.rst +++ b/doc/developer/xrefs.rst @@ -20,8 +20,6 @@ To verify xrefs have been included in a binary or dynamic library, run ``readelf -n binary``. For individual object files, it's ``readelf -S object.o | grep xref_array`` instead. -An extraction tool will be added in a future commit. - Structure and contents ---------------------- @@ -168,3 +166,50 @@ entry point. for C++ code when compiled by GCC. A workaround is present for runtime functionality, but to extract the xrefs from a C++ source file, it needs to be built with clang (or a future fixed version of GCC) instead. + +Extraction tool +--------------- + +The FRR source contains a matching tool to extract xref data from compiled ELF +binaries in ``python/xrelfo.py``. This tool uses CPython extensions +implemented in ``clippy`` and must therefore be executed with that. + +``xrelfo.py`` processes input from one or more ELF file (.o, .so, executable), +libtool object (.lo, .la, executable wrapper script) or JSON (output from +``xrelfo.py``) and generates an output JSON file. During standard FRR build, +it is invoked on all binaries and libraries and the result is combined into +``frr.json``. + +ELF files from any operating system, CPU architecture and endianness can be +processed on any host. Any issues with this are bugs in ``xrelfo.py`` +(or clippy's ELF code.) + +``xrelfo.py`` also performs some sanity checking, particularly on log +messages. The following options are available: + +.. option:: -o OUTPUT + + Filename to write JSON output to. As a convention, a ``.xref`` filename + extension is used. + +.. option:: -Wlog-format + + Performs extra checks on log message format strings, particularly checks + for ``\t`` and ``\n`` characters (which should not be used in log messages). + +.. option:: -Wlog-args + + Generates cleanup hints for format string arguments where + :c:func:`printfrr()` extensions could be used, e.g. replacing ``inet_ntoa`` + with ``%pI4``. + +.. option:: --profile + + Runs the Python profiler to identify hotspots in the ``xrelfo.py`` code. + +``xrelfo.py`` uses information about C structure definitions saved in +``python/xrefstructs.json``. This file is included with the FRR sources and +only needs to be regenerated when some of the ``struct xref_*`` definitions +are changed (which should be almost never). The file is written by +``python/tiabwarfo.py``, which uses ``pahole`` to extract the necessary data +from DWARF information. |
