diff options
Diffstat (limited to 'doc/developer')
| -rw-r--r-- | doc/developer/building-frr-for-archlinux.rst | 4 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-centos7.rst | 3 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-centos8.rst | 3 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-debian8.rst | 2 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-debian9.rst | 2 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-fedora.rst | 2 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-opensuse.rst | 2 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-ubuntu1604.rst | 4 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-ubuntu1804.rst | 4 | ||||
| -rw-r--r-- | doc/developer/building-frr-for-ubuntu2004.rst | 4 | ||||
| -rw-r--r-- | doc/developer/frr-release-procedure.rst | 21 | ||||
| -rw-r--r-- | doc/developer/include-compile.rst | 5 | ||||
| -rw-r--r-- | doc/developer/packaging-debian.rst | 42 | ||||
| -rw-r--r-- | doc/developer/packaging-redhat.rst | 4 | ||||
| -rw-r--r-- | doc/developer/scripting.rst | 614 | ||||
| -rw-r--r-- | doc/developer/workflow.rst | 98 |
16 files changed, 534 insertions, 280 deletions
diff --git a/doc/developer/building-frr-for-archlinux.rst b/doc/developer/building-frr-for-archlinux.rst index e589a9f724..af1677e89e 100644 --- a/doc/developer/building-frr-for-archlinux.rst +++ b/doc/developer/building-frr-for-archlinux.rst @@ -10,8 +10,8 @@ Installing Dependencies sudo pacman -S \ git autoconf automake libtool make cmake pcre readline texinfo \ pkg-config pam json-c bison flex python-pytest \ - c-ares python systemd python2-ipaddress python-sphinx \ - systemd-libs net-snmp perl libcap libelf + c-ares python python2-ipaddress python-sphinx \ + net-snmp perl libcap libelf .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-centos7.rst b/doc/developer/building-frr-for-centos7.rst index 93b9993a38..ce11126f70 100644 --- a/doc/developer/building-frr-for-centos7.rst +++ b/doc/developer/building-frr-for-centos7.rst @@ -21,7 +21,7 @@ Add packages: sudo yum install git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig \ json-c-devel pam-devel bison flex pytest c-ares-devel \ - python-devel systemd-devel python-sphinx libcap-devel \ + python-devel python-sphinx libcap-devel \ elfutils-libelf-devel .. include:: building-libyang.rst @@ -66,7 +66,6 @@ an example.) --enable-user=frr \ --enable-group=frr \ --enable-vty-group=frrvty \ - --enable-systemd=yes \ --disable-ldpd \ --enable-fpm \ --with-pkg-git-version \ diff --git a/doc/developer/building-frr-for-centos8.rst b/doc/developer/building-frr-for-centos8.rst index 65c93286b7..109a7866d9 100644 --- a/doc/developer/building-frr-for-centos8.rst +++ b/doc/developer/building-frr-for-centos8.rst @@ -14,7 +14,7 @@ Add packages: sudo dnf install --enablerepo=PowerTools git autoconf pcre-devel \ automake libtool make readline-devel texinfo net-snmp-devel pkgconfig \ groff pkgconfig json-c-devel pam-devel bison flex python2-pytest \ - c-ares-devel python2-devel systemd-devel libcap-devel \ + c-ares-devel python2-devel libcap-devel \ elfutils-libelf-devel .. include:: building-libyang.rst @@ -59,7 +59,6 @@ an example.) --enable-user=frr \ --enable-group=frr \ --enable-vty-group=frrvty \ - --enable-systemd=yes \ --disable-ldpd \ --enable-fpm \ --with-pkg-git-version \ diff --git a/doc/developer/building-frr-for-debian8.rst b/doc/developer/building-frr-for-debian8.rst index 475e0303fc..5e58854ed7 100644 --- a/doc/developer/building-frr-for-debian8.rst +++ b/doc/developer/building-frr-for-debian8.rst @@ -17,7 +17,7 @@ Add packages: sudo apt-get install git autoconf automake libtool make \ libreadline-dev texinfo libjson-c-dev pkg-config bison flex python3-pip \ - libc-ares-dev python3-dev python3-sphinx build-essential libsystemd-dev \ + libc-ares-dev python3-dev python3-sphinx build-essential \ libsnmp-dev libcap-dev libelf-dev Install newer pytest (>3.0) from pip diff --git a/doc/developer/building-frr-for-debian9.rst b/doc/developer/building-frr-for-debian9.rst index 1981127b3d..f8d8025f62 100644 --- a/doc/developer/building-frr-for-debian9.rst +++ b/doc/developer/building-frr-for-debian9.rst @@ -11,7 +11,7 @@ Add packages: sudo apt-get install git autoconf automake libtool make \ libreadline-dev texinfo libjson-c-dev pkg-config bison flex \ libc-ares-dev python3-dev python3-pytest python3-sphinx build-essential \ - libsnmp-dev libsystemd-dev libcap-dev libelf-dev + libsnmp-dev libcap-dev libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-fedora.rst b/doc/developer/building-frr-for-fedora.rst index 5fecd8a826..6ce76ba158 100644 --- a/doc/developer/building-frr-for-fedora.rst +++ b/doc/developer/building-frr-for-fedora.rst @@ -14,7 +14,7 @@ Installing Dependencies sudo dnf install git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig json-c-devel \ pam-devel python3-pytest bison flex c-ares-devel python3-devel \ - python3-sphinx perl-core patch systemd-devel libcap-devel \ + python3-sphinx perl-core patch libcap-devel \ elfutils-libelf-devel .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-opensuse.rst b/doc/developer/building-frr-for-opensuse.rst index 4e886e9c25..ee6a36a14b 100644 --- a/doc/developer/building-frr-for-opensuse.rst +++ b/doc/developer/building-frr-for-opensuse.rst @@ -13,7 +13,7 @@ Installing Dependencies zypper in git autoconf automake libtool make \ readline-devel texinfo net-snmp-devel groff pkgconfig libjson-c-devel\ pam-devel python3-pytest bison flex c-ares-devel python3-devel\ - python3-Sphinx perl patch systemd-devel libcap-devel libyang-devel \ + python3-Sphinx perl patch libcap-devel libyang-devel \ libelf-devel Building & Installing FRR diff --git a/doc/developer/building-frr-for-ubuntu1604.rst b/doc/developer/building-frr-for-ubuntu1604.rst index 2cb9536f9b..d79545c859 100644 --- a/doc/developer/building-frr-for-ubuntu1604.rst +++ b/doc/developer/building-frr-for-ubuntu1604.rst @@ -13,8 +13,8 @@ Installing Dependencies apt-get install \ git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ - libc-ares-dev python3-dev libsystemd-dev python-ipaddress python3-sphinx \ - install-info build-essential libsystemd-dev libsnmp-dev perl libcap-dev \ + libc-ares-dev python3-dev python-ipaddress python3-sphinx \ + install-info build-essential libsnmp-dev perl libcap-dev \ libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-ubuntu1804.rst b/doc/developer/building-frr-for-ubuntu1804.rst index eb3991c139..39a17fc01c 100644 --- a/doc/developer/building-frr-for-ubuntu1804.rst +++ b/doc/developer/building-frr-for-ubuntu1804.rst @@ -13,8 +13,8 @@ Installing Dependencies sudo apt-get install \ git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ - libc-ares-dev python3-dev libsystemd-dev python-ipaddress python3-sphinx \ - install-info build-essential libsystemd-dev libsnmp-dev perl libcap-dev \ + libc-ares-dev python3-dev python-ipaddress python3-sphinx \ + install-info build-essential libsnmp-dev perl libcap-dev \ libelf-dev .. include:: building-libyang.rst diff --git a/doc/developer/building-frr-for-ubuntu2004.rst b/doc/developer/building-frr-for-ubuntu2004.rst index 58d72e2891..92ddead4a5 100644 --- a/doc/developer/building-frr-for-ubuntu2004.rst +++ b/doc/developer/building-frr-for-ubuntu2004.rst @@ -13,8 +13,8 @@ Installing Dependencies sudo apt-get install \ git autoconf automake libtool make libreadline-dev texinfo \ pkg-config libpam0g-dev libjson-c-dev bison flex python3-pytest \ - libc-ares-dev python3-dev libsystemd-dev python-ipaddress python3-sphinx \ - install-info build-essential libsystemd-dev libsnmp-dev perl \ + libc-ares-dev python3-dev python-ipaddress python3-sphinx \ + install-info build-essential libsnmp-dev perl \ libcap-dev python2 libelf-dev Note that Ubuntu 20 no longer installs python 2.x, so it must be diff --git a/doc/developer/frr-release-procedure.rst b/doc/developer/frr-release-procedure.rst index 5da73f61f6..08e3057ae6 100644 --- a/doc/developer/frr-release-procedure.rst +++ b/doc/developer/frr-release-procedure.rst @@ -49,23 +49,14 @@ FRR Release Procedure 5. Update Changelog for Debian Packages: - Edit :file:`changelog-auto.in`: + Update :file:`debian/changelog`: - - Change last (top of list) entry from ``@VERSION@`` to the **last** - released version number. For example, if ``<version>`` is ``7.3`` and the - last public release was ``7.2``, you would use ``7.2``, changing the file - like so:: + - Run following with **last** release version number and debian revision + (usually -1) as argument to ``dch --newversion VERSION``. For example, if + ``<version>`` is ``7.3`` then you will run ``dch --newversion 7.3-1``. - frr (@VERSION@) RELEASED; urgency=medium - - to:: - - frr (7.2) RELEASED; urgency=medium - - - Add a new entry to the top of the list with a ``@VERSION@`` tag. Make sure - to watch the format. - - - Add the changelog text below this entry. + - The ``dch`` will run an editor, and you should add the changelog text below + this entry, usually that would be: **New upstream version**. - Verify the changelog format using ``dpkg-parsechangelog``. In the repository root: diff --git a/doc/developer/include-compile.rst b/doc/developer/include-compile.rst index d9fa260221..513cac6179 100644 --- a/doc/developer/include-compile.rst +++ b/doc/developer/include-compile.rst @@ -2,11 +2,6 @@ Clone the FRR git repo and use the included ``configure`` script to configure FRR's build time options to your liking. The full option listing can be obtained by running ``./configure -h``. The options shown below are examples. -.. note:: - - If your platform uses ``systemd``, please make sure to add - ``--enable-systemd=yes`` to your configure options. - .. code-block:: console git clone https://github.com/frrouting/frr.git frr diff --git a/doc/developer/packaging-debian.rst b/doc/developer/packaging-debian.rst index b57286d5a1..a81e052490 100644 --- a/doc/developer/packaging-debian.rst +++ b/doc/developer/packaging-debian.rst @@ -30,33 +30,30 @@ buster.) .. code-block:: shell - sudo mk-build-deps --install debian/control + sudo mk-build-deps --install --remove debian/control Alternatively, you can manually install build dependencies for your platform as outlined in :ref:`building`. -4. Run ``tools/tarsource.sh -V``: +4. Install `git-buildpackage` package: .. code-block:: shell - ./tools/tarsource.sh -V - - This script sets up the ``debian/changelog-auto`` file with proper version - information. + sudo apt-get install git-buildpackage 5. (optional) Append a distribution identifier if needed (see below under :ref:`multi-dist`.) -6. Build Debian Package: +6. Build Debian Binary and/or Source Packages: .. code-block:: shell - dpkg-buildpackage $options + gbp buildpackage --git-builder=dpkg-buildpackage --git-debian-branch="$(git rev-parse --abbrev-ref HEAD)" $options Where `$options` may contain any or all of the following items: * build profiles specified with ``-P``, e.g. - ``-Ppkg.frr.nortrlib,pkg.frr.nosystemd``. + ``-Ppkg.frr.nortrlib,pkg.frr.rtrlib``. Multiple values are separated by commas and there must not be a space after the ``-P``. @@ -67,20 +64,24 @@ buster.) +================+===================+=========================================+ | pkg.frr.rtrlib | pkg.frr.nortrlib | builds frr-rpki-rtrlib package (or not) | +----------------+-------------------+-----------------------------------------+ - | n/a | pkg.frr.nosystemd | removes libsystemd dependency and | - | | | disables unit file installation | - +----------------+-------------------+-----------------------------------------+ - - .. note:: - - The ``pkg.frr.nosystemd`` option is only intended to support Ubuntu - 14.04 (and should be enabled when building for that.) * the ``-uc -us`` options to disable signing the packages with your GPG key (git builds of the `master` or `stable/X.X` branches won't be signed by default since their target release is set to ``UNRELEASED``.) + * the ``--build=type`` accepts following options (see ``dpkg-buildpackage`` manual page): + + * ``source`` builds the source package + * ``any`` builds the architecture specific binary packages + * ``all`` build the architecture independent binary packages + * ``binary`` build the architecture specific and independent binary packages (alias for ``any,all``) + * ``full`` builds everything (alias for ``source,any,all``) + + Alternatively, you might want to replace ``dpkg-buildpackage`` with + ``debuild`` wrapper that also runs ``lintian`` and ``debsign`` on the final + packages. + 7. Done! If all worked correctly, then you should end up with the Debian packages in @@ -97,12 +98,6 @@ buster.) a manually maintained changelog that contains proper Debian release versioning. - Furthermore, official Debian packages are built in ``3.0 (quilt)`` format - with an "orig" tarball and a "debian" tarball. These tarballs are created - by the ``tarsource.sh`` tool on any branch. The git repository however - contains a ``3.0 (git)`` source format specifier to easily allow direct - git builds. - .. _multi-dist: @@ -111,7 +106,6 @@ Multi-Distribution builds You can optionally append a distribution identifier in case you want to make multiple versions of the package available in the same repository. -Do the following after creating the changelog with `tarsource.sh`: .. code-block:: shell diff --git a/doc/developer/packaging-redhat.rst b/doc/developer/packaging-redhat.rst index 458cfa0ad4..9e64b912f3 100644 --- a/doc/developer/packaging-redhat.rst +++ b/doc/developer/packaging-redhat.rst @@ -18,10 +18,6 @@ Tested on CentOS 6, CentOS 7, CentOS 8 and Fedora 24. yum install rpm-build net-snmp-devel pam-devel libcap-devel - If your platform uses systemd:: - - yum install systemd-devel - For CentOS 7 and CentOS 8, the package will be built using python3 and requires additional python3 packages:: diff --git a/doc/developer/scripting.rst b/doc/developer/scripting.rst index 708f65ff7d..d543ed3560 100644 --- a/doc/developer/scripting.rst +++ b/doc/developer/scripting.rst @@ -14,8 +14,8 @@ is implemented using the standard Lua C bindings. The supported version of Lua is 5.3. C objects may be passed into Lua and Lua objects may be retrieved by C code via -a marshalling system. In this way, arbitrary data from FRR may be passed to -scripts. It is possible to pass C functions as well. +a encoding/decoding system. In this way, arbitrary data from FRR may be passed to +scripts. The Lua environment is isolated from the C environment; user scripts cannot access FRR's address space unless explicitly allowed by FRR. @@ -53,213 +53,297 @@ Reasons against supporting multiple scripting languages: with which a given script can be shared General -^^^^^^^ - -FRR's concept of a script is somewhat abstracted away from the fact that it is -Lua underneath. A script in has two things: - -- name -- state +------- -In code: +FRR's scripting functionality is provided in the form of Lua functions in Lua +scripts (``.lua`` files). One Lua script may contain many Lua functions. These +are respectively encapsulated in the following structures: .. code-block:: c struct frrscript { - /* Script name */ - char *name; + /* Lua file name */ + char *name; - /* Lua state */ - struct lua_State *L; + /* hash of lua_function_states */ + struct hash *lua_function_hash; }; + struct lua_function_state { + /* Lua function name */ + char *name; -``name`` is simply a string. Everything else is in ``state``, which is itself a -Lua library object (``lua_State``). This is an opaque struct that is -manipulated using ``lua_*`` functions. The basic ones are imported from -``lua.h`` and the rest are implemented within FRR to fill our use cases. The -thing to remember is that all operations beyond the initial loading the script -take place on this opaque state object. + lua_State *L; + }; -There are four basic actions that can be done on a script: -- load -- execute -- query state -- unload +`struct frrscript`: Since all Lua functions are contained within scripts, the +following APIs manipulates this structure. ``name`` contains the +Lua script name and a hash of Lua functions to their function names. -They are typically done in this order. +`struct lua_function_state` is an internal structure, but it essentially contains +the name of the Lua function and its state (a stack), which is run using Lua +library functions. +In general, to run a Lua function, these steps must take place: -Loading -^^^^^^^ +- Initialization +- Load +- Call +- Delete -A snippet of Lua code is referred to as a "chunk". These are simply text. FRR -presently assumes chunks are located in individual files specific to one task. -These files are stored in the scripts directory and must end in ``.lua``. +Initialization +^^^^^^^^^^^^^^ -A script object is created by loading a script. This is done with -``frrscript_load()``. This function takes the name of the script and an -optional callback function. The string ".lua" is appended to the script name, -and the resultant filename is looked for in the scripts directory. +The ``frrscript`` object encapsulates the Lua function state(s) from +one Lua script file. To create, use ``frrscript_new()`` which takes the +name of the Lua script. +The string ".lua" is appended to the script name, and the resultant filename +will be used to look for the script when we want to load a Lua function from it. -For example, to load ``/etc/frr/scripts/bingus.lua``: +For example, to create ``frrscript`` for ``/etc/frr/scripts/bingus.lua``: .. code-block:: c - struct frrscript *fs = frrscript_load("bingus", NULL); + struct frrscript *fs = frrscript_new("bingus"); -During loading the script is validated for syntax and its initial environment -is setup. By default this does not include the Lua standard library; there are -security issues to consider, though for practical purposes untrusted users -should not be able to write the scripts directory anyway. If desired the Lua -standard library may be added to the script environment using -``luaL_openlibs(fs->L)`` after loading the script. Further information on -setting up the script environment is in the Lua manual. +The script is *not* read at this stage. +This function cannot be used to test for a script's presence. -Executing -^^^^^^^^^ +Load +^^^^ -After loading, scripts may be executed. A script may take input in the form of -variable bindings set in its environment prior to being run, and may provide -results by setting the value of variables. Arbitrary C values may be -transferred into the script environment, including functions. +The function to be called must first be loaded. Use ``frrscript_load()`` +which takes a ``frrscript`` object, the name of the Lua function +and a callback function. -A typical execution call looks something like this: +For example, to load the Lua function ``on_foo`` +in ``/etc/frr/scripts/bingus.lua``: .. code-block:: c - struct frrscript *fs = frrscript_load(...); + int ret = frrscript_load(fs, "on_foo", NULL); - int status_ok = 0, status_fail = 1; - struct prefix p = ...; - struct frrscript_env env[] = { - {"integer", "STATUS_FAIL", &status_fail}, - {"integer", "STATUS_OK", &status_ok}, - {"prefix", "myprefix", &p}, - {}}; +This function returns 0 if and only if the Lua function was successfully loaded. +A non-zero return could indicate either a missing Lua script, a missing +Lua function, or an error when loading the function. - int result = frrscript_call(fs, env); +During loading the script is validated for syntax and its environment +is set up. By default this does not include the Lua standard library; there are +security issues to consider, though for practical purposes untrusted users +should not be able to write the scripts directory anyway. +Call +^^^^ -To execute a loaded script, we need to define the inputs. These inputs are -passed by binding values to variable names that will be accessible within the -Lua environment. Basically, all communication with the script takes place via -global variables within the script, and to provide inputs we predefine globals -before the script runs. This is done by passing ``frrscript_call()`` an array -of ``struct frrscript_env``. Each struct has three fields. The first identifies -the type of the value being passed; more on this later. The second defines the -name of the global variable within the script environment to bind the third -argument (the value) to. +After loading, Lua functions may be called. -The script is then executed and returns a general status code. In the success -case this will be 0, otherwise it will be nonzero. The script itself does not -determine this code, it is provided by the Lua interpreter. +Input +""""" +Inputs to the Lua script should be given by providing a list of parenthesized +pairs, +where the first and second field identify the name of the variable and the +value it is bound to, respectively. +The types of the values must have registered encoders (more below); the compiler +will warn you otherwise. -Querying State -^^^^^^^^^^^^^^ +These variables are first encoded in-order, then provided as arguments +to the Lua function. In the example, note that ``c`` is passed in as a value +while ``a`` and ``b`` are passed in as pointers. + +.. code-block:: c + + int a = 100, b = 200, c = 300; + frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); -When a chunk is executed, its state at exit is preserved and can be inspected. -After running a script, results may be retrieved by querying the script's -state. Again this is done by retrieving the values of global variables, which -are known to the script author to be "output" variables. +.. code-block:: lua + + function on_foo(a, b, c) + -- a is 100, b is 200, c is 300 + ... + -A result is retrieved like so: +Output +"""""" .. code-block:: c - struct frrscript_env myresult = {"string", "myresult"}; + int a = 100, b = 200, c = 300; + frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + // a is 500, b is 200, c is 300 + + int* d = frrscript_get_result(fs, "on_foo", "d", lua_tointegerp); + // d is 800 + + +.. code-block:: lua + + function on_foo(a, b, c) + b = 600 + return { ["a"] = 500, ["c"] = 700, ["d"] = 800 } + end + + +**Lua functions being called must return a single table of string names to +values.** +(Lua functions should return an empty table if there is no output.) +The keys of the table are mapped back to names of variables in C. Note that +the values in the table can also be tables. Since tables are Lua's primary +data structure, this design lets us return any Lua value. + +After the Lua function returns, the names of variables to ``frrscript_call()`` +are matched against keys of the returned table, and then decoded. The types +being decoded must have registered decoders (more below); the compiler will +warn you otherwise. - char *myresult = frrscript_get_result(fs, &myresult); +In the example, since ``a`` was in the returned table and ``b`` was not, +``a`` was decoded and its value modified, while ``b`` was not decoded. +``c`` was decoded as well, but its decoder is a noop. +What modifications happen given a variable depends whether its name was +in the returned table and the decoder's implementation. - ... do something ... +.. warning:: + Always keep in mind that non const-qualified pointers in + ``frrscript_call()`` may be modified - this may be a source of bugs. + On the other hand, const-qualified pointers and other values cannot + be modified. - XFREE(MTYPE_TMP, myresult); +.. tip:: + You can make a copy of a data structure and pass that in instead, + so that modifications only happen to that copy. -As with arguments, results are retrieved by providing a ``struct -frrscript_env`` specifying a type and a global name. No value is necessary, nor -is it modified by ``frrscript_get_result()``. That function simply extracts the -requested value from the script state and returns it. +``frrscript_call()`` returns 0 if and only if the Lua function was successfully +called. A non-zero return could indicate either a missing Lua script, a missing +Lua function, or an error from the Lua interpreter. -In most cases the returned value will be allocated with ``MTYPE_TMP`` and will -need to be freed after use. +In the above example, ``d`` was not an input to ``frrscript_call()``, so its +value must be explicitly retrieved with ``frrscript_get_result``. +``frrscript_get_result()`` takes a +decoder and string name which is used as a key to search the returned table. +Returns the pointer to the decoded value, or NULL if it was not found. +In the example, ``d`` is a "new" value in C space, +so memory allocation might take place. Hence the caller is +responsible for memory deallocation. -Unloading -^^^^^^^^^ -To destroy a script and its associated state: +Delete +^^^^^^ + +To delete a script and the all Lua states associated with it: + +.. code-block:: c + + frrscript_delete(fs); + + +A complete example +"""""""""""""""""" + +So, a typical execution call, with error checking, looks something like this: .. code-block:: c - frrscript_unload(fs); + struct frrscript *fs = frrscript_new("my_script"); // name *without* .lua + + int ret = frrscript_load(fs, "on_foo", NULL); + if (ret != 0) + goto DONE; // Lua script or function might have not been found + + int a = 100, b = 200, c = 300; + ret = frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + if (ret != 0) + goto DONE; // Lua function might have not successfully run -Values returned by ``frrscript_get_result`` are still valid after the script -they were retrieved from is unloaded. + // a and b might be modified + assert(a == 500); + assert(b == 200); -Note that you must unload and then load the script if you want to reset its -state, for example to run it again with different inputs. Otherwise the state -from the previous run carries over into subsequent runs. + // c could not have been modified + assert(c == 300); + // d is new + int* d = frrscript_get_result(fs, "on_foo", "d", lua_tointegerp); -.. _marshalling: + if (!d) + goto DONE; // "d" might not have been in returned table -Marshalling -^^^^^^^^^^^ + assert(*d == 800); + XFREE(MTYPE_SCRIPT_RES, d); // caller responsible for free -Earlier sections glossed over the meaning of the type name field in ``struct -frrscript_env`` and how data is passed between C and Lua. Lua, as a dynamically -typed, garbage collected language, cannot directly use C values without some -kind of marshalling / unmarshalling system to translate types between the two -runtimes. + DONE: + frrscript_delete(fs); + + +.. code-block:: lua + + function on_foo(a, b, c) + b = 600 + return { a = 500, c = 700, d = 800 } + end + + +Note that ``{ a = ...`` is same as ``{ ["a"] = ...``; it is Lua shorthand to +use the variable name as the key in a table. + +Encoding and Decoding +^^^^^^^^^^^^^^^^^^^^^ + +Earlier sections glossed over the types of values that can be passed into +``frrscript_call()`` and how data is passed between C and Lua. Lua, as a +dynamically typed, garbage collected language, cannot directly use C values +without some kind of encoding / decoding system to +translate types between the two runtimes. Lua communicates with C code using a stack. C code wishing to provide data to -Lua scripts must provide a function that marshalls the C data into a Lua +Lua scripts must provide a function that encodes the C data into a Lua representation and pushes it on the stack. C code wishing to retrieve data from -Lua must provide a corresponding unmarshalling function that retrieves a Lua -value from the stack and converts it to the corresponding C type. These two -functions, together with a chosen name of the type they operate on, are -referred to as ``codecs`` in FRR. +Lua must provide a corresponding decoder function that retrieves a Lua +value from the stack and converts it to the corresponding C type. -A codec is defined as: +Encoders and decoders are provided for common data types. +Developers wishing to pass their own data structures between C and Lua need to +create encoders and decoders for that data type. -.. code-block:: c +We try to keep them named consistently. +There are three kinds of encoders and decoders: - typedef void (*encoder_func)(lua_State *, const void *); - typedef void *(*decoder_func)(lua_State *, int); +1. lua_push*: encodes a value onto the Lua stack. + Required for ``frrscript_call``. - struct frrscript_codec { - const char *typename; - encoder_func encoder; - decoder_func decoder; - }; +2. lua_decode*: decodes a value from the Lua stack. + Required for ``frrscript_call``. + Only non const-qualified pointers may be actually decoded (more below). + +3. lua_to*: allocates memory and decodes a value from the Lua stack. + Required for ``frrscript_get_result``. -A typename string and two function pointers. +This design allows us to combine typesafe *modification* of C values as well as +*allocation* of new C values. -``typename`` can be anything you want. For example, for the combined types of -``struct prefix`` and its equivalent in Lua I have chosen the name ``prefix``. -There is no restriction on naming here, it is just a human name used as a key -and specified when passing and retrieving values. +In the following sections, we will use the encoders/decoders for ``struct prefix`` as an example. -``encoder`` is a function that takes a ``lua_State *`` and a C type and pushes -onto the Lua stack a value representing the C type. For C structs, the usual -case, this will typically be a Lua table (tables are the only datastructure Lua -has). For example, here is the encoder function for ``struct prefix``: +Encoding +"""""""" +An encoder function takes a ``lua_State *``, a C type and pushes that value onto +the Lua state (a stack). +For C structs, the usual case, +this will typically be encoded to a Lua table, then pushed onto the Lua stack. + +Here is the encoder function for ``struct prefix``: .. code-block:: c - void lua_pushprefix(lua_State *L, const struct prefix *prefix) + void lua_pushprefix(lua_State *L, struct prefix *prefix) { char buffer[PREFIX_STRLEN]; - zlog_debug("frrlua: pushing prefix table"); - lua_newtable(L); lua_pushstring(L, prefix2str(prefix, buffer, PREFIX_STRLEN)); lua_setfield(L, -2, "network"); @@ -269,65 +353,151 @@ has). For example, here is the encoder function for ``struct prefix``: lua_setfield(L, -2, "family"); } -This function pushes a single value onto the Lua stack. It is a table whose equivalent in Lua is: +This function pushes a single value, a table, onto the Lua stack, whose +equivalent in Lua is: .. code-block:: c { ["network"] = "1.2.3.4/24", ["prefixlen"] = 24, ["family"] = 2 } -``decoder`` does the reverse; it takes a ``lua_State *`` and an index into the -stack, and unmarshalls a Lua value there into the corresponding C type. Again -for ``struct prefix``: +Decoding +"""""""" +Decoders are a bit more involved. They do the reverse; a decoder function takes +a ``lua_State *``, pops a value off the Lua stack and converts it back into its +C type. + +There are two: ``lua_decode*`` and ``lua_to*``. The former does no mememory +allocation and is needed for ``frrscript_call``. +The latter performs allocation and is optional. + +A ``lua_decode_*`` function takes a ``lua_State*``, an index, and a pointer +to a C data structure, and directly modifies the structure with values from the +Lua stack. Note that only non const-qualified pointers may be modified; +``lua_decode_*`` for other types will be noops. + +Again, for ``struct prefix *``: .. code-block:: c - void *lua_toprefix(lua_State *L, int idx) + void lua_decode_prefix(lua_State *L, int idx, struct prefix *prefix) { - struct prefix *p = XCALLOC(MTYPE_TMP, sizeof(struct prefix)); + lua_getfield(L, idx, "network"); + (void)str2prefix(lua_tostring(L, -1), prefix); + /* pop the netork string */ + lua_pop(L, 1); + /* pop the prefix table */ + lua_pop(L, 1); + } + + +Note: + - Before ``lua_decode*`` is run, the "prefix" table is already on the top of + the stack. ``frrscript_call`` does this for us. + - However, at the end of ``lua_decode*``, the "prefix" table should be popped. + - The other two fields in the "network" table are disregarded, meaning that any + modification to them is discarded in C space. In this case, this is desired + behavior. + +.. warning:: + + ``lua_decode*`` functions should pop all values that ``lua_to*`` pushed onto + the Lua stack. + For encoders that pushed a table, its decoder should pop the table at the end. + The above is an example. + + + +``int`` is not a non const-qualified pointer, so for ``int``: - lua_getfield(L, idx, "network"); - str2prefix(lua_tostring(L, -1), p); - lua_pop(L, 1); +.. code-block:: c + + void lua_decode_int_noop(lua_State *L, int idx, int i) + { //noop + } + + +A ``lua_to*`` function provides identical functionality except that it first +allocates memory for the new C type before decoding the value from the Lua stack, +then returns a pointer to the newly allocated C type. You only need to implement +this function to use with ``frrscript_get_result`` to retrieve a result of +this type. + +This function can and should be implemented using ``lua_decode_*``: +.. code-block:: c + + void *lua_toprefix(lua_State *L, int idx) + { + struct prefix *p = XCALLOC(MTYPE_SCRIPT_RES, sizeof(struct prefix)); + + lua_decode_prefix(L, idx, p); return p; } -By convention these functions should be called ``lua_to*``, as this is the -naming convention used by the Lua C library for the basic types e.g. -``lua_tointeger`` and ``lua_tostring``. The returned data must always be copied off the stack and the copy must be -allocated with ``MTYPE_TMP``. This way it is possible to unload the script +allocated with ``MTYPE_SCRIPT_RES``. This way it is possible to unload the script (destroy the state) without invalidating any references to values stored in it. +Note that it is the caller's responsibility to free the data. -To register a new type with its corresponding encoding functions: -.. code-block:: c +Registering encoders and decoders for frrscript_call +"""""""""""""""""""""""""""""""""""""""""""""""""""" - struct frrscript_codec frrscript_codecs_lib[] = { - {.typename = "prefix", - .encoder = (encoder_func)lua_pushprefix, - .decoder = lua_toprefix}, - {.typename = "sockunion", - .encoder = (encoder_func)lua_pushsockunion, - .decoder = lua_tosockunion}, - ... - {}}; +To register a new type with its ``lua_push*`` and ``lua_decode*`` functions, +add the mapping in the following macros in ``frrscript.h``: - frrscript_register_type_codecs(frrscript_codecs_lib); +.. code-block:: diff + + #define ENCODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + - struct peer * : lua_pushpeer \ + + struct peer * : lua_pushpeer, \ + + struct prefix * : lua_pushprefix \ + )((L), (value)) + + #define DECODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + - struct peer * : lua_decode_peer \ + + struct peer * : lua_decode_peer, \ + + struct prefix * : lua_decode_prefix \ + )((L), -1, (value)) + + +At compile time, the compiler will search for encoders/decoders for the type of +each value passed in via ``frrscript_call``. If a encoder/decoder cannot be +found, it will appear as a compile warning. Note that the types must +match *exactly*. +In the above example, we defined encoders/decoders for a value of +``struct prefix *``, but not ``struct prefix`` or ``const struct prefix *``. + +``const`` values are a special case. We want to use them in our Lua scripts +but not modify them, so creating a decoder for them would be meaningless. +But we still need a decoder for the type of value so that the compiler will be +satisfied. +For that, use ``lua_decode_noop``: + +.. code-block:: diff + + #define DECODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + + const struct prefix * : lua_decode_noop \ + )(L, -1, value) -From this point on the type names are available to be used when calling any -script and getting its results. .. note:: - Marshalled types are not restricted to simple values like integers, strings - and tables. It is possible to marshall a type such that the resultant object - in Lua is an actual object-oriented object, complete with methods that call - back into defined C functions. See the Lua manual for how to do this; for a - code example, look at how zlog is exported into the script environment. + Encodable/decodable types are not restricted to simple values like integers, + strings and tables. + It is possible to encode a type such that the resultant object in Lua + is an actual object-oriented object, complete with methods that call + back into defined C functions. See the Lua manual for how to do this; + for a code example, look at how zlog is exported into the script environment. Script Environment @@ -356,10 +526,11 @@ Examples For a complete code example involving passing custom types, retrieving results, and doing complex calculations in Lua, look at the implementation of the ``match script SCRIPT`` command for BGP routemaps. This example calls into a -script with a route prefix and attributes received from a peer and expects the -script to return a match / no match / match and update result. +script with a function named ``route_match``, +provides route prefix and attributes received from a peer and expects the +function to return a match / no match / match and update result. -An example script to use with this follows. This script matches, does not match +An example script to use with this follows. This function matches, does not match or updates a route depending on how many BGP UPDATE messages the peer has received when the script is called, simply as a demonstration of what can be accomplished with scripting. @@ -370,64 +541,75 @@ accomplished with scripting. -- Example route map matching -- author: qlyoung -- - -- The following variables are available to us: + -- The following variables are available in the global environment: -- log -- logging library, with the usual functions - -- prefix + -- + -- route_match arguments: + -- table prefix -- the route under consideration - -- attributes + -- table attributes -- the route's attributes - -- peer + -- table peer -- the peer which received this route - -- RM_FAILURE + -- integer RM_FAILURE -- status code in case of failure - -- RM_NOMATCH + -- integer RM_NOMATCH -- status code for no match - -- RM_MATCH + -- integer RM_MATCH -- status code for match - -- RM_MATCH_AND_CHANGE + -- integer RM_MATCH_AND_CHANGE -- status code for match-and-set -- - -- We need to set the following out values: - -- action - -- Set to the appropriate status code to indicate what we did - -- attributes - -- Setting fields on here will propagate them back up to the caller if - -- 'action' is set to RM_MATCH_AND_CHANGE. - - - log.info("Evaluating route " .. prefix.network .. " from peer " .. peer.remote_id.string) - - function on_match (prefix, attrs) - log.info("Match") - action = RM_MATCH - end - - function on_nomatch (prefix, attrs) - log.info("No match") - action = RM_NOMATCH - end - - function on_match_and_change (prefix, attrs) - action = RM_MATCH_AND_CHANGE - log.info("Match and change") - attrs["metric"] = attrs["metric"] + 7 - end - - special_routes = { - ["172.16.10.4/24"] = on_match, - ["172.16.13.1/8"] = on_nomatch, - ["192.168.0.24/8"] = on_match_and_change, - } + -- route_match returns table with following keys: + -- integer action, required + -- resultant status code. Should be one of RM_* + -- table attributes, optional + -- updated route attributes + -- + + function route_match(prefix, attributes, peer, + RM_FAILURE, RM_NOMATCH, RM_MATCH, RM_MATCH_AND_CHANGE) + + log.info("Evaluating route " .. prefix.network .. " from peer " .. peer.remote_id.string) + function on_match (prefix, attributes) + log.info("Match") + return { + attributes = RM_MATCH + } + end - if special_routes[prefix.network] then - special_routes[prefix.network](prefix, attributes) - elseif peer.stats.update_in % 3 == 0 then - on_match(prefix, attributes) - elseif peer.stats.update_in % 2 == 0 then - on_nomatch(prefix, attributes) - else - on_match_and_change(prefix, attributes) - end - + function on_nomatch (prefix, attributes) + log.info("No match") + return { + action = RM_NOMATCH + } + end + + function on_match_and_change (prefix, attributes) + log.info("Match and change") + attributes["metric"] = attributes["metric"] + 7 + return { + action = RM_MATCH_AND_CHANGE, + attributes = attributes + } + end + + special_routes = { + ["172.16.10.4/24"] = on_match, + ["172.16.13.1/8"] = on_nomatch, + ["192.168.0.24/8"] = on_match_and_change, + } + + + if special_routes[prefix.network] then + return special_routes[prefix.network](prefix, attributes) + elseif peer.stats.update_in % 3 == 0 then + return on_match(prefix, attributes) + elseif peer.stats.update_in % 2 == 0 then + return on_nomatch(prefix, attributes) + else + return on_match_and_change(prefix, attributes) + end + end diff --git a/doc/developer/workflow.rst b/doc/developer/workflow.rst index 58b9c36e01..dae175a732 100644 --- a/doc/developer/workflow.rst +++ b/doc/developer/workflow.rst @@ -880,6 +880,104 @@ doesn't warrant an update to checkpatch, it is documented here. | should be wrapped in parentheses | all usages of the macro, which would be highly disruptive. | +------------------------------------------+---------------------------------------------------------------+ +Types of configurables +---------------------- + +.. note:: + + This entire section essentially just argues to not make configuration + unnecessarily involved for the user. Rather than rules, this is more of + a list of conclusions intended to help make FRR usable for operators. + + +Almost every feature FRR has comes with its own set of switches and options. +There are several stages at which configuration can be applied. In order of +preference, these are: + +- at configuration/runtime, through YANG. + + This is the preferred way for all FRR knobs. Not all daemons and features + are fully YANGified yet, so in some cases new features cannot rely on a + YANG interface. If a daemon already implements a YANG interface (even + partial), new CLI options must be implemented through a YANG model. + + .. warning:: + + Unlike everything else in this section being guidelines with some slack, + implementing and using a YANG interface for new CLI options in (even + partially!) YANGified daemons is a hard requirement. + + +- at configuration/runtime, through the CLI. + + The "good old" way for all regular configuration. More involved for users + to automate *correctly* than YANG. + +- at startup, by loading additional modules. + + If a feature introduces a dependency on additional libraries (e.g. libsnmp, + rtrlib, etc.), this is the best way to encapsulate the dependency. Having + a separate module allows the distribution to create a separate package + with the extra dependency, so FRR can still be installed without pulling + everything in. + + A module may also be appropriate if a feature is large and reasonably well + isolated. Reducing the amount of running the code is a security benefit, + so even if there are no new external dependencies, modules can be useful. + + While modules cannot currently be loaded at runtime, this is a tradeoff + decision that was made to allow modules to change/extend code that is very + hard to (re)adjust at runtime. If there is a case for runtime (un)loading + of modules, this tradeoff can absolutely be reevaluated. + +- at startup, with command line options. + + This interface is only appropriate for options that have an effect very + early in FRR startup, i.e. before configuration is loaded. Anything that + affects configuration load itself should be here, as well as options + changing the environment FRR runs in. + + If a tunable can be changed at runtime, a command line option is only + acceptable if the configured value has an effect before configuration is + loaded (e.g. zebra reads routes from the kernel before loading config, so + the netlink buffer size is an appropriate command line option.) + +- at compile time, with ``./configure`` options. + + This is the absolute last preference for tunables, since the distribution + needs to make the decision for the user and/or the user needs to rebuild + FRR in order to change the option. + + "Good" configure options do one of three things: + + - set distribution-specific parameters, most prominently all the path + options. File system layout is a distribution/packaging choice, so the + user would hopefully never need to adjust these. + + - changing toolchain behavior, e.g. instrumentation, warnings, + optimizations and sanitizers. + + - enabling/disabling parts of the build, especially if they need + additional dependencies. Being able to build only parts of FRR, or + without some library, is useful. **The only effect these options should + have is adding or removing files from the build result.** If a knob + in this category causes the same binary to exist in different variants, + it is likely implemented incorrectly! + + .. note:: + + This last guideline is currently ignored by several configure options. + ``vtysh`` in general depends on the entire list of enabled daemons, + and options like ``--enable-bgp-vnc`` and ``--enable-ospfapi`` change + daemons internally. Consider this more of an "ideal" than a "rule". + + +Whenever adding new knobs, please try reasonably hard to go up as far as +possible on the above list. Especially ``./configure`` flags are often enough +the "easy way out" but should be avoided when at all possible. To a lesser +degree, the same applies to command line options. + + Compile-time conditional code ----------------------------- |