diff options
Diffstat (limited to 'doc/developer')
| -rw-r--r-- | doc/developer/mgmtd-dev.rst | 172 | ||||
| -rw-r--r-- | doc/developer/northbound/advanced-topics.rst | 95 | ||||
| -rw-r--r-- | doc/developer/northbound/architecture.rst | 55 | ||||
| -rw-r--r-- | doc/developer/northbound/demos.rst | 3 | ||||
| -rw-r--r-- | doc/developer/northbound/links.rst | 3 | ||||
| -rw-r--r-- | doc/developer/northbound/northbound.rst | 2 | ||||
| -rw-r--r-- | doc/developer/northbound/operational-data-rpcs-and-notifications.rst | 18 | ||||
| -rw-r--r-- | doc/developer/northbound/plugins-sysrepo.rst | 3 | ||||
| -rw-r--r-- | doc/developer/northbound/ppr-basic-test-topology.rst | 17 | ||||
| -rw-r--r-- | doc/developer/northbound/ppr-mpls-basic-test-topology.rst | 17 | ||||
| -rw-r--r-- | doc/developer/northbound/retrofitting-configuration-commands.rst | 160 | ||||
| -rw-r--r-- | doc/developer/northbound/transactional-cli.rst | 150 | ||||
| -rw-r--r-- | doc/developer/northbound/yang-module-translator.rst | 36 | ||||
| -rw-r--r-- | doc/developer/northbound/yang-tools.rst | 12 |
14 files changed, 469 insertions, 274 deletions
diff --git a/doc/developer/mgmtd-dev.rst b/doc/developer/mgmtd-dev.rst index d611f9c11b..e557d3a84a 100644 --- a/doc/developer/mgmtd-dev.rst +++ b/doc/developer/mgmtd-dev.rst @@ -12,7 +12,7 @@ MGMTD Development ================= Overview -^^^^^^^^ +-------- ``mgmtd`` (Management Daemon) is a new centralized management daemon for FRR. @@ -33,9 +33,55 @@ each daemon. ``mgmtd`` currently provides the CLI interface for each daemon that has been converted to it, but in the future RESTCONF and NETCONF servers can easily be added as *front-ends* to mgmtd to support those protocols as well. +Conversion Status +^^^^^^^^^^^^^^^^^ + +Fully Converted To MGMTD +"""""""""""""""""""""""" + +- lib/distribute +- lib/filter +- lib/if_rmap +- lib/routemap +- ripd +- staticd +- zebra (* - partial) + +Converted To Northbound +""""""""""""""""""""""" +- bfdd +- lib/affinitymap +- lib/if +- pathd +- pbrd +- pimd +- ripngd + +Converted To Northbound With Issues +""""""""""""""""""""""""""""""""""" +- eigrp +- isisd + +Unconverted +""""""""""" +- babel +- bgpd +- ldpd +- lib/event +- lib/keychain +- lib/log_vty +- lib/nexthop_group +- lib/zlog_5424_cli +- nhrpd +- ospfd +- ospf6d +- pceplib +- qdb +- sharpd +- vrrpd Converting A Daemon to MGMTD -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +---------------------------- A daemon must first be transitioned to the new :ref:`northbound` interface if that has not already been done (see :ref:`nb-retrofit` for how to do this). Once this @@ -43,7 +89,7 @@ is done a few simple steps are all that is required move the daemon over to ``mgmtd`` control. Overview of Changes -------------------- +^^^^^^^^^^^^^^^^^^^ Adding support for a *northbound* converted daemon involves very little work. It requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support. @@ -51,31 +97,78 @@ requires enabling *frontend* (CLI and YANG) and *backend* (YANG) support. Front-End Interface: -1. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.yang.c``). -2. Add CLI handler file[s] to ``mgmtd/subdir.am`` (e.g., ``staticd/static_vty.c``). -3. [if needed] Exclude (#ifndef) non-configuration CLI handlers from CLI source - file (e.g., inside ``staticd/static_vty.c``). -4. [otherwise] Remove CLI handler file from SOURCES in daemon (e.g in :file:`staticd/subdir.am`) -5. Add YANG module description into array defined in ``mgmtd/mgmt_main.c`` (see :ref:`mgmtd-config-write`). -6. Initialize the CLI handlers inside :code:`mgmt_vty_init` in :file:`mgmtd/mgmt_vty.c`. -7. Direct ``vtysh`` to send CLI commands to ``mgmtd`` by modifying +#. Add YANG module file to ``mgmtd/subdir.am`` (e.g., ``yang/frr-staticd.yang.c``). + +#. Add CLI handler file[s] to ``mgmtd/subdir.am``. The `subdir.am` variable to + use is indicated in the next 2 steps. + + #. [if needed] Exclude (:code:`#ifndef`) non-configuration CLI handlers from + CLI source file (e.g., inside :file:`staticd/static_vty.c`) and add the + file to :code:`nodist_mgmtd_libmgmt_be_nb_la_SOURCES` in + :file:`mgmtd/subdir.am`. + + #. [otherwise] Remove CLI handler file from _SOURCES variable in the daemon + :file:`subdir.am` file (e.g in :file:`staticd/subdir.am`) and add to + :code:`mgmtd_libmgmtd_a_SOURCES` in :file:`mgmtd/subdir.am`. + +#. In order to have mgmtd try and load existing per-daemon config files, add + the daemon to the :code:`mgmt_daemons` array in :file:`lib/vty.c`. With the + official release of the mgmtd code FRR is no longer supporting per daemon log + files but it will take a while before all of the topotest is converted. + +#. In the daemon's :code:`struct frr_daemon_info` (i.e., inside it's + :code:`FRR_DAEMON_INFO()`) set the `.flags` bit `FRR_NO_SPLIT_CONFIG`. This + will keep the daemon from trying to read it's per-daemon config file as mgmtd + will now be doing this. + +#. Add the daemon's YANG module description[s] into the array + :code:`mgmt_yang_modules` defined in :file:`mgmtd/mgmt_main.c` (see + :ref:`mgmtd-config-write`). Make sure that all YANG modules that the daemon + uses are present in the mgmtd list. To find this list look in the daemon's + equivalent yang module array variable. + +#. Initialize the CLI handlers inside :code:`mgmt_vty_init` in :file:`mgmtd/mgmt_vty.c`. + +#. Direct ``vtysh`` to send CLI commands to ``mgmtd`` by modifying ``vtysh/vtysh.h``. At the top of this file each daemon has a bit ``#define``'d (e.g., ``#define VTYSH_STATICD 0x08000``) below this there are groupings, replace all the uses of the daemons bit with ``VTYSH_MGMTD`` instead so that the CLI commands get properly routed to ``mgmtd`` rather than the daemon now. + #. Remove initialization (and installation) of library CLI routines. These will + correspond with the VTYSH removals from the last step i.e.,: + + - change access_list_init() to access_list_init_new(false) and remove from + VTYSH_ACL_CONFIG (leave in VTYSH_ACL_SHOW). + - remove if_cmd_init_default() => remove from VTYSH_INTERFACE_SUBSET + - remove if_cmd_init() => remove from VTYSH_INTERFACE_SUBSET + - change route_map_init() to route_map_init_new(false) and remove from + VTYSH_ROUTE_MAP_CONFIG (leave in VTYSH_ROUTE_MAP_SHOW). + - remove vrf_cmd_init(NULL) => remove from VTYSH_INTERFACE_SUBSET + ... + Back-End Interface: -8. In ``mgmtd/mgmt_be_adapter.c`` add xpath prefix mappings to a one or both +#. In the daemon's main file initialize the BE client library. You add a global + `struct mgmt_be_client *mgmt_be_client` near the daemons `event_loop *master` + variable. Then where the daemon used to initialize it's CLI/VTY code replace + that with the client initialization by calling `mgmt_be_client_create`. + Likewise in the daemon's sigint cleanup code, operational walks should be + canceled with a call to `nb_oper_cancel_all_walks`, and then the BE client + should be destroyed with a call to `mgmt_be_client_destroy` and to be safe + NULL out the global `mgmt_be_client` variable. + +#. In ``mgmtd/mgmt_be_adapter.c`` add xpath prefix mappings to a one or both mapping arrays (``be_client_config_xpaths`` and ``be_client_oper_xpaths``) to direct ``mgmtd`` to send config and oper-state requests to your daemon. NOTE: make sure to include library supported xpaths prefixes as well (e.g., - "/frr-interface:lib"). - + "/frr-interface:lib"). A good way to figure these paths out are to look in + each of the YANG modules that the daemon uses and include each of their paths + in the array. Add YANG and CLI into MGMTD ---------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ As an example here is the addition made to ``mgmtd/subdir.am`` for adding ``staticd`` support. @@ -108,7 +201,7 @@ An here is the addition to the modules array in ``mgmtd/mgmt_main.c``: CLI Config and Show Handlers ----------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The daemon's CLI handlers for configuration (which having been converted to the :ref:`northbound` now simply generate YANG changes) will be linked directly into @@ -157,7 +250,7 @@ are present in a single file (e.g. a ``xxx_vty.c`` or ``xxx_cli.c`` file) then .. _mgmtd-config-write: CLI Config Write Handlers (:code:`cli_show`) --------------------------------------------- +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To support writing out the CLI configuration file the northbound API defines a 2 callbacks (:code:`cli_show` and :code:`cli_show_end`). Pointers to these @@ -172,10 +265,10 @@ the *backend* config and oper-state callbacks (e.g., :code:`create`, So you will need to define 2 :code:`struct frr_yang_module_info` arrays. -1. The existing array remains in the same place in the daemon, but with all the +#. The existing array remains in the same place in the daemon, but with all the :code:`cli_show` handlers removed. -2. The removed :code:`cli_show` handlers should be added to a new +#. The removed :code:`cli_show` handlers should be added to a new :code:`struct frr_yang_module_info` array. This second array should be included in the same file that includes that actual function pointed to by the the :code:`cli_show` callbacks (i.e., the file is compiled into @@ -184,8 +277,44 @@ So you will need to define 2 :code:`struct frr_yang_module_info` arrays. This new :code:`struct frr_yang_module_info` array is the one to be included in mgmtd in `mgmt_yang_modules` inside ``mgmtd/mgmt_main.c``. +Back-End Client Connection +^^^^^^^^^^^^^^^^^^^^^^^^^^ + +In order for your daemon to communicate with mgmtd you need to initialize the +backend client library. You normally do this where you used to initialize your +CLI/VTY code. + +.. code-block:: c + + ... + struct event_loop *master; + + static struct mgmt_be_client *mgmt_be_client; + ... + + int main(int argc, char **argv) + { + ... + rip_init(); + rip_if_init(); + mgmt_be_client = mgmt_be_client_create("ripd", NULL, 0, master); + +Likewise the client should be cleaned up in the daemon cleanup routine. + +.. code-block:: c + + /* SIGINT handler. */ + static void sigint(void) + { + zlog_notice("Terminating on signal"); + ... + nb_oper_cancel_all_walks(); + mgmt_be_client_destroy(mgmt_be_client); + mgmt_be_client = NULL; + + Back-End XPATH mappings ------------------------ +^^^^^^^^^^^^^^^^^^^^^^^ In order for ``mgmtd`` to direct configuration to your daemon you need to add some XPATH mappings to ``mgmtd/mgmt_be_adapter.c``. These XPATHs determine which @@ -228,9 +357,8 @@ not conditionalized b/c it should always be present): [MGMTD_BE_CLIENT_ID_ZEBRA] = zebra_oper_xpaths, }; - MGMTD Internals -^^^^^^^^^^^^^^^ +--------------- This section will describe the internal functioning of ``mgmtd``, for now a couple diagrams are included to aide in source code perusal. diff --git a/doc/developer/northbound/advanced-topics.rst b/doc/developer/northbound/advanced-topics.rst index bee29a95a9..edfc10bcbd 100644 --- a/doc/developer/northbound/advanced-topics.rst +++ b/doc/developer/northbound/advanced-topics.rst @@ -1,3 +1,11 @@ +Advanced Topics +=============== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 + Auto-generated CLI commands ~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -34,13 +42,16 @@ CLI on a separate program The flexible design of the northbound architecture opens the door to move the CLI to a separate program in the long-term future. Some -advantages of doing so would be: \* Treat the CLI as just another -northbound client, instead of having CLI commands embedded in the -binaries of all FRR daemons. \* Improved robustness: bugs in CLI -commands (e.g. null-pointer dereferences) or in the CLI code itself -wouldn’t affect the FRR daemons. \* Foster innovation by allowing other -CLI programs to be implemented, possibly using higher level programming -languages. +advantages of doing so would be: + +* Treat the CLI as just another northbound client, instead of having CLI + commands embedded in the binaries of all FRR daemons. + +* Improved robustness: bugs in CLI commands (e.g. null-pointer dereferences) or + in the CLI code itself wouldn’t affect the FRR daemons. + +* Foster innovation by allowing other CLI programs to be implemented, possibly + using higher level programming languages. The problem, however, is that the northbound retrofitting process will convert only the CLI configuration commands and EXEC commands in a first @@ -232,40 +243,42 @@ vtysh support As explained in the [[Transactional CLI]] page, all commands introduced by the transactional CLI are not yet available in *vtysh*. This needs to be addressed in the short term future. Some challenges for doing that -work include: \* How to display configurations (running, candidates and -rollbacks) in a more clever way? The implementation of the -``show running-config`` command in *vtysh* is not something that should -be followed as an example. A better idea would be to fetch the desired -configuration from all daemons (encoded in JSON for example), merge them -all into a single ``lyd_node`` variable and then display the combined -configurations from this variable (the configuration merges would -transparently take care of combining the shared configuration objects). -In order to be able to manipulate the JSON configurations, *vtysh* will -need to load the YANG modules from all daemons at startup (this might -have a minimal impact on startup time). The only issue with this -approach is that the ``cli_show()`` callbacks from all daemons are -embedded in their binaries and thus not accessible externally. It might -be necessary to compile these callbacks on a separate shared library so -that they are accessible to *vtysh* too. Other than that, displaying the -combined configurations in the JSON/XML formats should be -straightforward. \* With the current design, transaction IDs are -per-daemon and not global across all FRR daemons. This means that the -same transaction ID can represent different transactions on different -daemons. Given this observation, how to implement the -``rollback configuration`` command in *vtysh*? The easy solution would -be to add a ``daemon WORD`` argument to specify the context of the -rollback, but per-daemon rollbacks would certainly be confusing and -convoluted to end users. A better idea would be to attack the root of -the problem: change configuration transactions to be global instead of -being per-daemon. This involves a bigger change in the northbound -architecture, and would have implications on how transactions are stored -in the SQL database (daemon-specific and shared configuration objects -would need to have their own tables or columns). \* Loading -configuration files in the JSON or XML formats will be tricky, as -*vtysh* will need to know which sections of the configuration should be -sent to which daemons. *vtysh* will either need to fetch the YANG -modules implemented by all daemons at runtime or obtain this information -at compile-time somehow. +work include: + +* How to display configurations (running, candidates and rollbacks) in a more + clever way? The implementation of the ``show running-config`` command in + *vtysh* is not something that should be followed as an example. A better idea + would be to fetch the desired configuration from all daemons (encoded in JSON + for example), merge them all into a single ``lyd_node`` variable and then + display the combined configurations from this variable (the configuration + merges would transparently take care of combining the shared configuration + objects). In order to be able to manipulate the JSON configurations, *vtysh* + will need to load the YANG modules from all daemons at startup (this might + have a minimal impact on startup time). The only issue with this approach is + that the ``cli_show()`` callbacks from all daemons are embedded in their + binaries and thus not accessible externally. It might be necessary to compile + these callbacks on a separate shared library so that they are accessible to + *vtysh* too. Other than that, displaying the combined configurations in the + JSON/XML formats should be straightforward. + +* With the current design, transaction IDs are per-daemon and not global across + all FRR daemons. This means that the same transaction ID can represent + different transactions on different daemons. Given this observation, how to + implement the ``rollback configuration`` command in *vtysh*? The easy solution + would be to add a ``daemon WORD`` argument to specify the context of the + rollback, but per-daemon rollbacks would certainly be confusing and convoluted + to end users. A better idea would be to attack the root of the problem: change + configuration transactions to be global instead of being per-daemon. This + involves a bigger change in the northbound architecture, and would have + implications on how transactions are stored in the SQL database + (daemon-specific and shared configuration objects would need to have their own + tables or columns). + +* Loading configuration files in the JSON or XML formats will be tricky, as + *vtysh* will need to know which sections of the configuration should be sent + to which daemons. *vtysh* will either need to fetch the YANG modules + implemented by all daemons at runtime or obtain this information at + compile-time somehow. Detecting type mismatches at compile-time ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/developer/northbound/architecture.rst b/doc/developer/northbound/architecture.rst index e571971c7f..5fd89c32f1 100644 --- a/doc/developer/northbound/architecture.rst +++ b/doc/developer/northbound/architecture.rst @@ -1,3 +1,6 @@ +Architecture +============ + Introduction ------------ @@ -42,30 +45,34 @@ and `CoAP <https://www.ietf.org/archive/id/draft-vanderstok-core-comi-11.txt>`__. In addition to being management-protocol independent, some other -advantages of using YANG in FRR are listed below: \* Have a formal -contract between FRR and application developers (management clients). A -management client that has access to the FRR YANG models knows about all -existing configuration options available for use. This information can -be used to auto-generate user-friendly interfaces like Web-UIs, custom -CLIs and even code bindings for several different programming languages. -Using `PyangBind <https://github.com/robshakir/pyangbind>`__, for -example, it’s possible to generate Python class hierarchies from YANG -models and use these classes to instantiate objects that mirror the -structure of the YANG modules and can be serialized/deserialized using -different encoding formats. \* Support different encoding formats for -instance data. Currently only JSON and XML are supported, but -`GPB <https://developers.google.com/protocol-buffers/>`__ and -`CBOR <http://cbor.io/>`__ are other viable options in the long term. -Additional encoding formats can be implemented in the *libyang* library -for optimal performance, or externally by translating data to/from one -of the supported formats (with a performance penalty). \* Have a formal -mechanism to introduce backward-incompatible changes based on `semantic -versioning <http://www.openconfig.net/docs/semver/>`__ (not part of the -YANG standard, which allows backward-compatible module updates only). \* -Provide seamless support to the industry-standard NETCONF/RESTCONF -protocols as alternative management APIs. If FRR configuration/state -data is modeled using YANG, supporting YANG-based protocols like NETCONF -and RESTCONF is much easier. +advantages of using YANG in FRR are listed below: + +* Have a formal contract between FRR and application developers (management + clients). A management client that has access to the FRR YANG models knows + about all existing configuration options available for use. This information + can be used to auto-generate user-friendly interfaces like Web-UIs, custom + CLIs and even code bindings for several different programming languages. Using + `PyangBind <https://github.com/robshakir/pyangbind>`__, for example, it’s + possible to generate Python class hierarchies from YANG models and use these + classes to instantiate objects that mirror the structure of the YANG modules + and can be serialized/deserialized using different encoding formats. + +* Support different encoding formats for instance data. Currently only JSON and + XML are supported, but `GPB + <https://developers.google.com/protocol-buffers/>`__ and `CBOR + <http://cbor.io/>`__ are other viable options in the long term. Additional + encoding formats can be implemented in the *libyang* library for optimal + performance, or externally by translating data to/from one of the supported + formats (with a performance penalty). + +* Have a formal mechanism to introduce backward-incompatible changes based on + `semantic versioning <http://www.openconfig.net/docs/semver/>`__ (not part of + the YANG standard, which allows backward-compatible module updates only). + +* Provide seamless support to the industry-standard NETCONF/RESTCONF protocols + as alternative management APIs. If FRR configuration/state data is modeled + using YANG, supporting YANG-based protocols like NETCONF and RESTCONF is much + easier. As important as shifting to a model-driven management paradigm, the new northbound architecture also introduces the concept of configuration diff --git a/doc/developer/northbound/demos.rst b/doc/developer/northbound/demos.rst index 876bd25222..8a0f6ad6b2 100644 --- a/doc/developer/northbound/demos.rst +++ b/doc/developer/northbound/demos.rst @@ -1,3 +1,6 @@ +Demos +===== + Transactional CLI ----------------- diff --git a/doc/developer/northbound/links.rst b/doc/developer/northbound/links.rst index e80374c57b..6cec176f8b 100644 --- a/doc/developer/northbound/links.rst +++ b/doc/developer/northbound/links.rst @@ -1,3 +1,6 @@ +Links +===== + RFCs ~~~~ diff --git a/doc/developer/northbound/northbound.rst b/doc/developer/northbound/northbound.rst index 7dddf06460..c5f4e2f7e6 100644 --- a/doc/developer/northbound/northbound.rst +++ b/doc/developer/northbound/northbound.rst @@ -11,11 +11,11 @@ Northbound API transactional-cli retrofitting-configuration-commands operational-data-rpcs-and-notifications - plugins-sysrepo advanced-topics yang-tools yang-module-translator demos links + plugins-sysrepo ppr-basic-test-topology ppr-mpls-basic-test-topology diff --git a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst index 554bc17c80..5cb70ca6f1 100644 --- a/doc/developer/northbound/operational-data-rpcs-and-notifications.rst +++ b/doc/developer/northbound/operational-data-rpcs-and-notifications.rst @@ -1,3 +1,11 @@ +Operational Data, RPCs and Notifications +======================================== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 + Operational data ~~~~~~~~~~~~~~~~ @@ -330,10 +338,12 @@ CLI can take too long, potentially long enough to the point of triggering some protocol timeouts and bringing sessions down. To avoid this kind of problem, northbound clients are encouraged to do -one of the following: \* Create a separate pthread for handling requests -to fetch operational data. \* Iterate over YANG lists and leaf-lists -asynchronously, returning a maximum number of elements per time instead -of returning all elements in one shot. +one of the following: + +* Create a separate pthread for handling requests to fetch operational data. + +* Iterate over YANG lists and leaf-lists asynchronously, returning a maximum + number of elements per time instead of returning all elements in one shot. In order to handle both cases correctly, the ``get_next`` callbacks need to use locks to prevent the YANG lists from being modified while they diff --git a/doc/developer/northbound/plugins-sysrepo.rst b/doc/developer/northbound/plugins-sysrepo.rst index 186c3a0177..1797157573 100644 --- a/doc/developer/northbound/plugins-sysrepo.rst +++ b/doc/developer/northbound/plugins-sysrepo.rst @@ -1,3 +1,6 @@ +Plugins Sysrepo (old) +===================== + Installation ------------ diff --git a/doc/developer/northbound/ppr-basic-test-topology.rst b/doc/developer/northbound/ppr-basic-test-topology.rst index a680ed7dfa..4929c9b285 100644 --- a/doc/developer/northbound/ppr-basic-test-topology.rst +++ b/doc/developer/northbound/ppr-basic-test-topology.rst @@ -1,15 +1,10 @@ -Table of Contents -~~~~~~~~~~~~~~~~~ +IS-IS PPR Basic +=============== -- `Software <#software>`__ -- `Topology <#topology>`__ -- `Configuration <#configuration>`__ - - - `CLI <#configuration-cli>`__ - - `YANG <#configuration-yang>`__ - -- `Verification - Control Plane <#verification-cplane>`__ -- `Verification - Forwarding Plane <#verification-fplane>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 Software ~~~~~~~~ diff --git a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst index cedb795da9..aceec5fa2f 100644 --- a/doc/developer/northbound/ppr-mpls-basic-test-topology.rst +++ b/doc/developer/northbound/ppr-mpls-basic-test-topology.rst @@ -1,15 +1,10 @@ -Table of Contents -~~~~~~~~~~~~~~~~~ +IS-IS PPR Basic MPLS +==================== -- `Software <#software>`__ -- `Topology <#topology>`__ -- `Configuration <#configuration>`__ - - - `CLI <#configuration-cli>`__ - - `YANG <#configuration-yang>`__ - -- `Verification - Control Plane <#verification-cplane>`__ -- `Verification - Forwarding Plane <#verification-fplane>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 Software ~~~~~~~~ diff --git a/doc/developer/northbound/retrofitting-configuration-commands.rst b/doc/developer/northbound/retrofitting-configuration-commands.rst index 6ccc578a0d..4772610856 100644 --- a/doc/developer/northbound/retrofitting-configuration-commands.rst +++ b/doc/developer/northbound/retrofitting-configuration-commands.rst @@ -2,7 +2,15 @@ .. _nb-retrofit: Retrofitting Configuration Commands ------------------------------------ +=================================== + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 2 + +Retrofitting process +-------------------- This page explains how to convert existing CLI configuration commands to the new northbound model. This documentation is meant to be the primary @@ -10,9 +18,6 @@ reference for developers working on the northbound retrofitting process. We’ll show several examples taken from the ripd northbound conversion to illustrate some concepts described herein. -Retrofitting process --------------------- - Step 1: writing a YANG module ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -33,34 +38,41 @@ possible to facilitate the process of writing module translators using the [[YANG module translator]]. As an example, the frr-ripd YANG module incorporated several parts of the IETF RIP YANG module. The repositories below contain big collections of YANG models that might be used as a -reference: \* https://github.com/YangModels/yang \* -https://github.com/openconfig/public +reference: + +* https://github.com/YangModels/yang + +* https://github.com/openconfig/public When writing a YANG module, it’s highly recommended to follow the guidelines from `RFC 6087 <https://tools.ietf.org/html/rfc6087>`__. In general, most commands should be modeled fairly easy. Here are a few -guidelines specific to authors of FRR YANG models: \* Use -presence-containers or lists to model commands that change the CLI node -(e.g. ``router rip``, ``interface eth0``). This way, if the -presence-container or list entry is removed, all configuration options -below them are removed automatically (exactly like the CLI behaves when -a configuration object is removed using a *no* command). This -recommendation is orthogonal to the `YANG authoring guidelines for -OpenConfig -models <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ -where the use of presence containers is discouraged. OpenConfig YANG -models however were not designed to replicate the behavior of legacy CLI -commands. \* When using YANG lists, be careful to identify what should -be the key leaves. In the ``offset-list WORD <in|out> (0-16) IFNAME`` -command, for example, both the direction (``<in|out>``) and the -interface name should be the keys of the list. This can be only known by -analyzing the data structures used to store the commands. \* For -clarity, use non-presence containers to group leaves that are associated -to the same configuration command (as we’ll see later, this also -facilitate the process of writing ``cli_show`` callbacks). \* YANG -leaves of type *enumeration* should define explicitly the value of each -*enum* option based on the value used in the FRR source code. \* Default -values should be taken from the source code whenever they exist. +guidelines specific to authors of FRR YANG models: + +* Use presence-containers or lists to model commands that change the CLI node + (e.g. ``router rip``, ``interface eth0``). This way, if the presence-container + or list entry is removed, all configuration options below them are removed + automatically (exactly like the CLI behaves when a configuration object is + removed using a *no* command). This recommendation is orthogonal to the `YANG + authoring guidelines for OpenConfig models + <https://github.com/openconfig/public/blob/master/doc/openconfig_style_guide.md>`__ + where the use of presence containers is discouraged. OpenConfig YANG models + however were not designed to replicate the behavior of legacy CLI commands. + +* When using YANG lists, be careful to identify what should be the key leaves. + In the ``offset-list WORD <in|out> (0-16) IFNAME`` command, for example, both + the direction (``<in|out>``) and the interface name should be the keys of the + list. This can be only known by analyzing the data structures used to store + the commands. + +* For clarity, use non-presence containers to group leaves that are associated + to the same configuration command (as we’ll see later, this also facilitate + the process of writing ``cli_show`` callbacks). + +* YANG leaves of type *enumeration* should define explicitly the value of each + *enum* option based on the value used in the FRR source code. + +* Default values should be taken from the source code whenever they exist. Some commands are more difficult to model and demand the use of more advanced YANG constructs like *choice*, *when* and *must* statements. @@ -729,15 +741,17 @@ Configuration options are edited individually ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Several CLI commands edit multiple configuration options at the same -time. Some examples taken from ripd: \* -``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` - -*/frr-ripd:ripd/instance/timers/flush-interval* - -*/frr-ripd:ripd/instance/timers/holddown-interval* - -*/frr-ripd:ripd/instance/timers/update-interval* \* -``distance (1-255) A.B.C.D/M [WORD]`` - -*/frr-ripd:ripd/instance/distance/source/prefix* - -*/frr-ripd:ripd/instance/distance/source/distance* - -*/frr-ripd:ripd/instance/distance/source/access-list* +time. Some examples taken from ripd: + +* ``timers basic (5-2147483647) (5-2147483647) (5-2147483647)`` + * */frr-ripd:ripd/instance/timers/flush-interval* + * */frr-ripd:ripd/instance/timers/holddown-interval* + * */frr-ripd:ripd/instance/timers/update-interval* + +* ``distance (1-255) A.B.C.D/M [WORD]`` + * */frr-ripd:ripd/instance/distance/source/prefix* + * */frr-ripd:ripd/instance/distance/source/distance* + * */frr-ripd:ripd/instance/distance/source/access-list* In the new northbound model, there’s one or more separate callbacks for each configuration option. This usually has implications when converting @@ -1037,16 +1051,23 @@ changing the candidate configuration. the northbound callbacks are not involved). Other important details to keep in mind while rewriting the CLI -commands: \* ``nb_cli_cfg_change()`` returns CLI errors codes -(e.g. ``CMD_SUCCESS``, ``CMD_WARNING``), so the return value of this -function can be used as the return value of CLI commands. \* Calls to -``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted to -calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t -necessary anymore and can be removed: ``VTY_DECLVAR_CONTEXT``, -``VTY_DECLVAR_CONTEXT_SUB``, ``VTY_GET_CONTEXT`` and -``VTY_CHECK_CONTEXT``. The ``nb_cli_cfg_change()`` functions uses the -``VTY_CHECK_XPATH`` macro to check if the data node being edited still -exists before doing anything else. +commands: + +* ``nb_cli_cfg_change()`` returns CLI errors codes (e.g. ``CMD_SUCCESS``, + ``CMD_WARNING``), so the return value of this function can be used as the + return value of CLI commands. + +* Calls to ``VTY_PUSH_CONTEXT`` and ``VTY_PUSH_CONTEXT_SUB`` should be converted + to calls to ``VTY_PUSH_XPATH``. Similarly, the following macros aren’t + necessary anymore and can be removed: + + * ``VTY_DECLVAR_CONTEXT`` + * ``VTY_DECLVAR_CONTEXT_SUB`` + * ``VTY_GET_CONTEXT`` + * ``VTY_CHECK_CONTEXT``. + + The ``nb_cli_cfg_change()`` functions uses the ``VTY_CHECK_XPATH`` macro to + check if the data node being edited still exists before doing anything else. The examples below provide additional details about how the conversion should be done. @@ -1788,10 +1809,13 @@ Implementation of the ``cli_show`` callback: } This is the most complex ``cli_show`` callback we have in ripd. Its -complexity comes from the following: \* The -``ip rip authentication mode ...`` command changes two YANG leaves at -the same time. \* Part of the command should be hidden when the -``show_defaults`` parameter is set to false. +complexity comes from the following: + +* The ``ip rip authentication mode ...`` command changes two YANG leaves at the + same time. + +* Part of the command should be hidden when the ``show_defaults`` parameter is + set to false. This is the behavior we want to implement: @@ -1841,19 +1865,27 @@ As mentioned in the fourth step, the northbound retrofitting process can happen gradually over time, since both “old” and “new” commands can coexist without problems. Once all commands from a given daemon were converted, we can proceed to the consolidation step, which consists of -the following: \* Remove the vty configuration lock, which is enabled by -default in all daemons. Now multiple users should be able to edit the -configuration concurrently, using either shared or private candidate -configurations. \* Reference commit: -`57dccdb1 <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. -\* Stop using the qobj infrastructure to keep track of configuration -objects. This is not necessary anymore, the northbound uses a similar -mechanism to keep track of YANG data nodes in the candidate -configuration. \* Reference commit: -`4e6d63ce <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. -\* Make the daemon SIGHUP handler re-read the configuration file (and -ensure it’s not doing anything other than that). \* Reference commit: -`5e57edb4 <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. +the following: + +* Remove the vty configuration lock, which is enabled by default in all daemons. + Now multiple users should be able to edit the configuration concurrently, + using either shared or private candidate configurations. + +* Reference commit: `57dccdb1 + <https://github.com/opensourcerouting/frr/commit/57dccdb18b799556214dcfb8943e248c0bf1f6a6>`__. + +* Stop using the qobj infrastructure to keep track of configuration objects. + This is not necessary anymore, the northbound uses a similar mechanism to keep + track of YANG data nodes in the candidate configuration. + +* Reference commit: `4e6d63ce + <https://github.com/opensourcerouting/frr/commit/4e6d63cebd988af650c1c29d0f2e5a251c8d2e7a>`__. + +* Make the daemon SIGHUP handler re-read the configuration file (and ensure it’s + not doing anything other than that). + +* Reference commit: `5e57edb4 + <https://github.com/opensourcerouting/frr/commit/5e57edb4b71ff03f9a22d9ec1412c3c5167f90cf>`__. Final Considerations -------------------- diff --git a/doc/developer/northbound/transactional-cli.rst b/doc/developer/northbound/transactional-cli.rst index 439bb6afce..5c495d3a89 100644 --- a/doc/developer/northbound/transactional-cli.rst +++ b/doc/developer/northbound/transactional-cli.rst @@ -1,25 +1,10 @@ -Table of Contents ------------------ - -- `Introduction <#introduction>`__ -- `Configuration modes <#config-modes>`__ -- `New commands <#retrofitting-process>`__ - - - `commit check <#cmd1>`__ - - `commit <#cmd2>`__ - - `discard <#cmd3>`__ - - `configuration database max-transactions <#cmd4>`__ - - `configuration load <#cmd5>`__ - - `rollback configuration <#cmd6>`__ - - `show configuration candidate <#cmd7>`__ - - `show configuration compare <#cmd8>`__ - - `show configuration running <#cmd9>`__ - - `show configuration transaction <#cmd10>`__ - - `show yang module <#cmd11>`__ - - `show yang module-translator <#cmd12>`__ - - `update <#cmd13>`__ - - `yang module-translator load <#cmd14>`__ - - `yang module-translator unload <#cmd15>`__ +Transactional CLI +================= + +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 Introduction ~~~~~~~~~~~~ @@ -70,18 +55,21 @@ Configuration modes ~~~~~~~~~~~~~~~~~~~ When using the transactional CLI (``--tcli``), FRR supports three -different forms of the ``configure`` command: \* ``configure terminal``: -in this mode, a single candidate configuration is shared by all users. -This means that one user might delete a configuration object that’s -being edited by another user, in which case the CLI will detect and -report the problem. If one user issues the ``commit`` command, all -changes done by all users are committed. \* ``configure private``: users -have a private candidate configuration that is edited separately from -the other users. The ``commit`` command commits only the changes done by -the user. \* ``configure exclusive``: similar to ``configure private``, -but also locks the running configuration to prevent other users from -changing it. The configuration lock is released when the user exits the -configuration mode. +different forms of the ``configure`` command: + +* ``configure terminal``: in this mode, a single candidate configuration is + shared by all users. This means that one user might delete a configuration + object that’s being edited by another user, in which case the CLI will detect + and report the problem. If one user issues the ``commit`` command, all changes + done by all users are committed. + +* ``configure private``: users have a private candidate configuration that is + edited separately from the other users. The ``commit`` command commits only + the changes done by the user. + +* ``configure exclusive``: similar to ``configure private``, but also locks the + running configuration to prevent other users from changing it. The + configuration lock is released when the user exits the configuration mode. When using ``configure terminal`` or ``configure private``, the candidate configuration being edited might become outdated if another @@ -112,12 +100,14 @@ Check if the candidate configuration is valid or not. Commit the changes done in the candidate configuration into the running configuration. -Options: \* ``force``: commit even if the candidate configuration is -outdated. It’s usually a better option to use the ``update`` command -instead. \* ``comment LINE...``: assign a comment to the configuration -transaction. This comment is displayed when viewing the recorded -transactions in the output of the ``show configuration transaction`` -command. +Options: + +* ``force``: commit even if the candidate configuration is outdated. It’s + usually a better option to use the ``update`` command instead. + +* ``comment LINE...``: assign a comment to the configuration transaction. This + comment is displayed when viewing the recorded transactions in the output of + the ``show configuration transaction`` command. ``discard`` ''''''''''' @@ -140,10 +130,13 @@ respectively. It’s also possible to load a configuration from a previous transaction by specifying the desired transaction ID (``(1-4294967296)``). -Options: \* ``translate WORD``: translate the JSON/XML configuration -file using the YANG module translator. \* ``replace``: replace the -candidate by the loaded configuration. The default is to merge the -loaded configuration into the candidate configuration. +Options: + +* ``translate WORD``: translate the JSON/XML configuration file using the YANG + module translator. + +* ``replace``: replace the candidate by the loaded configuration. The default is + to merge the loaded configuration into the candidate configuration. ``rollback configuration (1-4294967296)`` ''''''''''''''''''''''''''''''''''''''''' @@ -156,39 +149,42 @@ identified by its transaction ID (``(1-4294967296)``). Show the candidate configuration. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. \* ``changes``: show only the changes done in the candidate -configuration. +Options: + +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. +* ``changes``: show only the changes done in the candidate configuration. ``show configuration compare <candidate|running|transaction (1-4294967296)> <candidate|running|transaction (1-4294967296)> [<json|xml> [translate WORD]]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Show the difference between two different configurations. -Options: \* ``json``: show the configuration differences in the JSON -format. \* ``xml``: show the configuration differences in the XML -format. \* ``translate WORD``: translate the JSON/XML output using the -YANG module translator. +Options: + +* ``json``: show the configuration differences in the JSON format. +* ``xml``: show the configuration differences in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. ``show configuration running [<json|xml> [translate WORD]] [with-defaults]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' Show the running configuration. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. +Options: - NOTE: ``show configuration running`` shows only the running - configuration as known by the northbound layer. Configuration - commands not converted to the new northbound model will not be - displayed. To show the full running configuration, the legacy - ``show running-config`` command must be used. +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. + +NOTE: ``show configuration running`` shows only the running +configuration as known by the northbound layer. Configuration +commands not converted to the new northbound model will not be +displayed. To show the full running configuration, the legacy +``show running-config`` command must be used. ``show configuration transaction [(1-4294967296) [<json|xml> [translate WORD]] [changes]]`` ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -199,12 +195,13 @@ configuration associated to the previously committed transaction. When a transaction ID is not given, show all recorded transactions in the rollback log. -Options: \* ``json``: show the configuration in the JSON format. \* -``xml``: show the configuration in the XML format. \* -``translate WORD``: translate the JSON/XML output using the YANG module -translator. \* ``with-defaults``: show default values that are hidden by -default. \* ``changes``: show changes compared to the previous -transaction. +Options: + +* ``json``: show the configuration in the JSON format. +* ``xml``: show the configuration in the XML format. +* ``translate WORD``: translate the JSON/XML output using the YANG module translator. +* ``with-defaults``: show default values that are hidden by default. +* ``changes``: show changes compared to the previous transaction. ``show yang module [module-translator WORD] [WORD <summary|tree|yang|yin>]`` '''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' @@ -212,11 +209,14 @@ transaction. When a YANG module is not given, show all loaded YANG modules. Otherwise, show detailed information about the given module. -Options: \* ``module-translator WORD``: change the context to modules -loaded by the specified YANG module translator. \* ``summary``: display -summary information about the module. \* ``tree``: display module in the -tree (RFC 8340) format. \* ``yang``: display module in the YANG format. -\* ``yin``: display module in the YIN format. +Options: + +* ``module-translator WORD``: change the context to modules loaded by the + specified YANG module translator. +* ``summary``: display summary information about the module. +* ``tree``: display module in the tree (RFC 8340) format. +* ``yang``: display module in the YANG format. +* ``yin``: display module in the YIN format. ``show yang module-translator`` ''''''''''''''''''''''''''''''' diff --git a/doc/developer/northbound/yang-module-translator.rst b/doc/developer/northbound/yang-module-translator.rst index aa527ce6b2..17ae160f07 100644 --- a/doc/developer/northbound/yang-module-translator.rst +++ b/doc/developer/northbound/yang-module-translator.rst @@ -1,11 +1,10 @@ -Table of Contents ------------------ +YANG Module Translation +======================= -- `Introduction <#introduction>`__ -- `Deviation Modules <#deviation-modules>`__ -- `Translation Tables <#translation-tables>`__ -- `CLI Demonstration <#cli-demonstration>`__ -- `Implementation Details <#implementation-details>`__ +.. contents:: Table of contents + :local: + :backlinks: entry + :depth: 1 Introduction ------------ @@ -421,15 +420,20 @@ this shortcoming and make it possible to create more powerful YANG module translators. YANG module translators can be evaluated based on the following metrics: -\* Translation potential: is it possible to make complex translations, -taking several variables into account? \* Complexity: measure of how -easy or hard it is to write a module translator. \* Speed: measure of -how fast the translation can be achieved. Translation speed is of -fundamental importance, especially for operational data. \* Robustness: -can the translator be checked for inconsistencies at load time? A module -translator based on scripts wouldn’t fare well on this metric. \* -Round-trip conversions: can the translated data be translated back to -the original format without information loss? + +* Translation potential: is it possible to make complex translations, taking + several variables into account? + +* Complexity: measure of how easy or hard it is to write a module translator. + +* Speed: measure of how fast the translation can be achieved. Translation speed + is of fundamental importance, especially for operational data. + +* Robustness: can the translator be checked for inconsistencies at load time? A + module translator based on scripts wouldn’t fare well on this metric. + +* Round-trip conversions: can the translated data be translated back to the + original format without information loss? CLI Demonstration ----------------- diff --git a/doc/developer/northbound/yang-tools.rst b/doc/developer/northbound/yang-tools.rst index 346efcaaee..fb5a287245 100644 --- a/doc/developer/northbound/yang-tools.rst +++ b/doc/developer/northbound/yang-tools.rst @@ -1,5 +1,5 @@ Yang Tools -~~~~~~~~~~ +========== Here's some information about various tools for working with yang models. @@ -83,17 +83,19 @@ Indent a YANG file: --keep-comments -f yang --yang-canonical \ module.yang -o module.yang -Generate skeleton instance data: \* XML: +Generate skeleton instance data: -.. code:: sh +* XML: + + .. code:: sh $ pyang -p <yang-search-path> \ -f sample-xml-skeleton --sample-xml-skeleton-defaults \ module.yang [augmented-module1.yang ...] -o module.xml -- JSON: +* JSON: -.. code:: sh + .. code:: sh $ pyang -p <yang-search-path> \ -f jsonxsl module.yang -o module.xsl |