diff options
Diffstat (limited to 'doc/developer/scripting.rst')
| -rw-r--r-- | doc/developer/scripting.rst | 614 |
1 files changed, 398 insertions, 216 deletions
diff --git a/doc/developer/scripting.rst b/doc/developer/scripting.rst index 708f65ff7d..d543ed3560 100644 --- a/doc/developer/scripting.rst +++ b/doc/developer/scripting.rst @@ -14,8 +14,8 @@ is implemented using the standard Lua C bindings. The supported version of Lua is 5.3. C objects may be passed into Lua and Lua objects may be retrieved by C code via -a marshalling system. In this way, arbitrary data from FRR may be passed to -scripts. It is possible to pass C functions as well. +a encoding/decoding system. In this way, arbitrary data from FRR may be passed to +scripts. The Lua environment is isolated from the C environment; user scripts cannot access FRR's address space unless explicitly allowed by FRR. @@ -53,213 +53,297 @@ Reasons against supporting multiple scripting languages: with which a given script can be shared General -^^^^^^^ - -FRR's concept of a script is somewhat abstracted away from the fact that it is -Lua underneath. A script in has two things: - -- name -- state +------- -In code: +FRR's scripting functionality is provided in the form of Lua functions in Lua +scripts (``.lua`` files). One Lua script may contain many Lua functions. These +are respectively encapsulated in the following structures: .. code-block:: c struct frrscript { - /* Script name */ - char *name; + /* Lua file name */ + char *name; - /* Lua state */ - struct lua_State *L; + /* hash of lua_function_states */ + struct hash *lua_function_hash; }; + struct lua_function_state { + /* Lua function name */ + char *name; -``name`` is simply a string. Everything else is in ``state``, which is itself a -Lua library object (``lua_State``). This is an opaque struct that is -manipulated using ``lua_*`` functions. The basic ones are imported from -``lua.h`` and the rest are implemented within FRR to fill our use cases. The -thing to remember is that all operations beyond the initial loading the script -take place on this opaque state object. + lua_State *L; + }; -There are four basic actions that can be done on a script: -- load -- execute -- query state -- unload +`struct frrscript`: Since all Lua functions are contained within scripts, the +following APIs manipulates this structure. ``name`` contains the +Lua script name and a hash of Lua functions to their function names. -They are typically done in this order. +`struct lua_function_state` is an internal structure, but it essentially contains +the name of the Lua function and its state (a stack), which is run using Lua +library functions. +In general, to run a Lua function, these steps must take place: -Loading -^^^^^^^ +- Initialization +- Load +- Call +- Delete -A snippet of Lua code is referred to as a "chunk". These are simply text. FRR -presently assumes chunks are located in individual files specific to one task. -These files are stored in the scripts directory and must end in ``.lua``. +Initialization +^^^^^^^^^^^^^^ -A script object is created by loading a script. This is done with -``frrscript_load()``. This function takes the name of the script and an -optional callback function. The string ".lua" is appended to the script name, -and the resultant filename is looked for in the scripts directory. +The ``frrscript`` object encapsulates the Lua function state(s) from +one Lua script file. To create, use ``frrscript_new()`` which takes the +name of the Lua script. +The string ".lua" is appended to the script name, and the resultant filename +will be used to look for the script when we want to load a Lua function from it. -For example, to load ``/etc/frr/scripts/bingus.lua``: +For example, to create ``frrscript`` for ``/etc/frr/scripts/bingus.lua``: .. code-block:: c - struct frrscript *fs = frrscript_load("bingus", NULL); + struct frrscript *fs = frrscript_new("bingus"); -During loading the script is validated for syntax and its initial environment -is setup. By default this does not include the Lua standard library; there are -security issues to consider, though for practical purposes untrusted users -should not be able to write the scripts directory anyway. If desired the Lua -standard library may be added to the script environment using -``luaL_openlibs(fs->L)`` after loading the script. Further information on -setting up the script environment is in the Lua manual. +The script is *not* read at this stage. +This function cannot be used to test for a script's presence. -Executing -^^^^^^^^^ +Load +^^^^ -After loading, scripts may be executed. A script may take input in the form of -variable bindings set in its environment prior to being run, and may provide -results by setting the value of variables. Arbitrary C values may be -transferred into the script environment, including functions. +The function to be called must first be loaded. Use ``frrscript_load()`` +which takes a ``frrscript`` object, the name of the Lua function +and a callback function. -A typical execution call looks something like this: +For example, to load the Lua function ``on_foo`` +in ``/etc/frr/scripts/bingus.lua``: .. code-block:: c - struct frrscript *fs = frrscript_load(...); + int ret = frrscript_load(fs, "on_foo", NULL); - int status_ok = 0, status_fail = 1; - struct prefix p = ...; - struct frrscript_env env[] = { - {"integer", "STATUS_FAIL", &status_fail}, - {"integer", "STATUS_OK", &status_ok}, - {"prefix", "myprefix", &p}, - {}}; +This function returns 0 if and only if the Lua function was successfully loaded. +A non-zero return could indicate either a missing Lua script, a missing +Lua function, or an error when loading the function. - int result = frrscript_call(fs, env); +During loading the script is validated for syntax and its environment +is set up. By default this does not include the Lua standard library; there are +security issues to consider, though for practical purposes untrusted users +should not be able to write the scripts directory anyway. +Call +^^^^ -To execute a loaded script, we need to define the inputs. These inputs are -passed by binding values to variable names that will be accessible within the -Lua environment. Basically, all communication with the script takes place via -global variables within the script, and to provide inputs we predefine globals -before the script runs. This is done by passing ``frrscript_call()`` an array -of ``struct frrscript_env``. Each struct has three fields. The first identifies -the type of the value being passed; more on this later. The second defines the -name of the global variable within the script environment to bind the third -argument (the value) to. +After loading, Lua functions may be called. -The script is then executed and returns a general status code. In the success -case this will be 0, otherwise it will be nonzero. The script itself does not -determine this code, it is provided by the Lua interpreter. +Input +""""" +Inputs to the Lua script should be given by providing a list of parenthesized +pairs, +where the first and second field identify the name of the variable and the +value it is bound to, respectively. +The types of the values must have registered encoders (more below); the compiler +will warn you otherwise. -Querying State -^^^^^^^^^^^^^^ +These variables are first encoded in-order, then provided as arguments +to the Lua function. In the example, note that ``c`` is passed in as a value +while ``a`` and ``b`` are passed in as pointers. + +.. code-block:: c + + int a = 100, b = 200, c = 300; + frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); -When a chunk is executed, its state at exit is preserved and can be inspected. -After running a script, results may be retrieved by querying the script's -state. Again this is done by retrieving the values of global variables, which -are known to the script author to be "output" variables. +.. code-block:: lua + + function on_foo(a, b, c) + -- a is 100, b is 200, c is 300 + ... + -A result is retrieved like so: +Output +"""""" .. code-block:: c - struct frrscript_env myresult = {"string", "myresult"}; + int a = 100, b = 200, c = 300; + frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + // a is 500, b is 200, c is 300 + + int* d = frrscript_get_result(fs, "on_foo", "d", lua_tointegerp); + // d is 800 + + +.. code-block:: lua + + function on_foo(a, b, c) + b = 600 + return { ["a"] = 500, ["c"] = 700, ["d"] = 800 } + end + + +**Lua functions being called must return a single table of string names to +values.** +(Lua functions should return an empty table if there is no output.) +The keys of the table are mapped back to names of variables in C. Note that +the values in the table can also be tables. Since tables are Lua's primary +data structure, this design lets us return any Lua value. + +After the Lua function returns, the names of variables to ``frrscript_call()`` +are matched against keys of the returned table, and then decoded. The types +being decoded must have registered decoders (more below); the compiler will +warn you otherwise. - char *myresult = frrscript_get_result(fs, &myresult); +In the example, since ``a`` was in the returned table and ``b`` was not, +``a`` was decoded and its value modified, while ``b`` was not decoded. +``c`` was decoded as well, but its decoder is a noop. +What modifications happen given a variable depends whether its name was +in the returned table and the decoder's implementation. - ... do something ... +.. warning:: + Always keep in mind that non const-qualified pointers in + ``frrscript_call()`` may be modified - this may be a source of bugs. + On the other hand, const-qualified pointers and other values cannot + be modified. - XFREE(MTYPE_TMP, myresult); +.. tip:: + You can make a copy of a data structure and pass that in instead, + so that modifications only happen to that copy. -As with arguments, results are retrieved by providing a ``struct -frrscript_env`` specifying a type and a global name. No value is necessary, nor -is it modified by ``frrscript_get_result()``. That function simply extracts the -requested value from the script state and returns it. +``frrscript_call()`` returns 0 if and only if the Lua function was successfully +called. A non-zero return could indicate either a missing Lua script, a missing +Lua function, or an error from the Lua interpreter. -In most cases the returned value will be allocated with ``MTYPE_TMP`` and will -need to be freed after use. +In the above example, ``d`` was not an input to ``frrscript_call()``, so its +value must be explicitly retrieved with ``frrscript_get_result``. +``frrscript_get_result()`` takes a +decoder and string name which is used as a key to search the returned table. +Returns the pointer to the decoded value, or NULL if it was not found. +In the example, ``d`` is a "new" value in C space, +so memory allocation might take place. Hence the caller is +responsible for memory deallocation. -Unloading -^^^^^^^^^ -To destroy a script and its associated state: +Delete +^^^^^^ + +To delete a script and the all Lua states associated with it: + +.. code-block:: c + + frrscript_delete(fs); + + +A complete example +"""""""""""""""""" + +So, a typical execution call, with error checking, looks something like this: .. code-block:: c - frrscript_unload(fs); + struct frrscript *fs = frrscript_new("my_script"); // name *without* .lua + + int ret = frrscript_load(fs, "on_foo", NULL); + if (ret != 0) + goto DONE; // Lua script or function might have not been found + + int a = 100, b = 200, c = 300; + ret = frrscript_call(fs, "on_foo", ("a", &a), ("b", &b), ("c", c)); + if (ret != 0) + goto DONE; // Lua function might have not successfully run -Values returned by ``frrscript_get_result`` are still valid after the script -they were retrieved from is unloaded. + // a and b might be modified + assert(a == 500); + assert(b == 200); -Note that you must unload and then load the script if you want to reset its -state, for example to run it again with different inputs. Otherwise the state -from the previous run carries over into subsequent runs. + // c could not have been modified + assert(c == 300); + // d is new + int* d = frrscript_get_result(fs, "on_foo", "d", lua_tointegerp); -.. _marshalling: + if (!d) + goto DONE; // "d" might not have been in returned table -Marshalling -^^^^^^^^^^^ + assert(*d == 800); + XFREE(MTYPE_SCRIPT_RES, d); // caller responsible for free -Earlier sections glossed over the meaning of the type name field in ``struct -frrscript_env`` and how data is passed between C and Lua. Lua, as a dynamically -typed, garbage collected language, cannot directly use C values without some -kind of marshalling / unmarshalling system to translate types between the two -runtimes. + DONE: + frrscript_delete(fs); + + +.. code-block:: lua + + function on_foo(a, b, c) + b = 600 + return { a = 500, c = 700, d = 800 } + end + + +Note that ``{ a = ...`` is same as ``{ ["a"] = ...``; it is Lua shorthand to +use the variable name as the key in a table. + +Encoding and Decoding +^^^^^^^^^^^^^^^^^^^^^ + +Earlier sections glossed over the types of values that can be passed into +``frrscript_call()`` and how data is passed between C and Lua. Lua, as a +dynamically typed, garbage collected language, cannot directly use C values +without some kind of encoding / decoding system to +translate types between the two runtimes. Lua communicates with C code using a stack. C code wishing to provide data to -Lua scripts must provide a function that marshalls the C data into a Lua +Lua scripts must provide a function that encodes the C data into a Lua representation and pushes it on the stack. C code wishing to retrieve data from -Lua must provide a corresponding unmarshalling function that retrieves a Lua -value from the stack and converts it to the corresponding C type. These two -functions, together with a chosen name of the type they operate on, are -referred to as ``codecs`` in FRR. +Lua must provide a corresponding decoder function that retrieves a Lua +value from the stack and converts it to the corresponding C type. -A codec is defined as: +Encoders and decoders are provided for common data types. +Developers wishing to pass their own data structures between C and Lua need to +create encoders and decoders for that data type. -.. code-block:: c +We try to keep them named consistently. +There are three kinds of encoders and decoders: - typedef void (*encoder_func)(lua_State *, const void *); - typedef void *(*decoder_func)(lua_State *, int); +1. lua_push*: encodes a value onto the Lua stack. + Required for ``frrscript_call``. - struct frrscript_codec { - const char *typename; - encoder_func encoder; - decoder_func decoder; - }; +2. lua_decode*: decodes a value from the Lua stack. + Required for ``frrscript_call``. + Only non const-qualified pointers may be actually decoded (more below). + +3. lua_to*: allocates memory and decodes a value from the Lua stack. + Required for ``frrscript_get_result``. -A typename string and two function pointers. +This design allows us to combine typesafe *modification* of C values as well as +*allocation* of new C values. -``typename`` can be anything you want. For example, for the combined types of -``struct prefix`` and its equivalent in Lua I have chosen the name ``prefix``. -There is no restriction on naming here, it is just a human name used as a key -and specified when passing and retrieving values. +In the following sections, we will use the encoders/decoders for ``struct prefix`` as an example. -``encoder`` is a function that takes a ``lua_State *`` and a C type and pushes -onto the Lua stack a value representing the C type. For C structs, the usual -case, this will typically be a Lua table (tables are the only datastructure Lua -has). For example, here is the encoder function for ``struct prefix``: +Encoding +"""""""" +An encoder function takes a ``lua_State *``, a C type and pushes that value onto +the Lua state (a stack). +For C structs, the usual case, +this will typically be encoded to a Lua table, then pushed onto the Lua stack. + +Here is the encoder function for ``struct prefix``: .. code-block:: c - void lua_pushprefix(lua_State *L, const struct prefix *prefix) + void lua_pushprefix(lua_State *L, struct prefix *prefix) { char buffer[PREFIX_STRLEN]; - zlog_debug("frrlua: pushing prefix table"); - lua_newtable(L); lua_pushstring(L, prefix2str(prefix, buffer, PREFIX_STRLEN)); lua_setfield(L, -2, "network"); @@ -269,65 +353,151 @@ has). For example, here is the encoder function for ``struct prefix``: lua_setfield(L, -2, "family"); } -This function pushes a single value onto the Lua stack. It is a table whose equivalent in Lua is: +This function pushes a single value, a table, onto the Lua stack, whose +equivalent in Lua is: .. code-block:: c { ["network"] = "1.2.3.4/24", ["prefixlen"] = 24, ["family"] = 2 } -``decoder`` does the reverse; it takes a ``lua_State *`` and an index into the -stack, and unmarshalls a Lua value there into the corresponding C type. Again -for ``struct prefix``: +Decoding +"""""""" +Decoders are a bit more involved. They do the reverse; a decoder function takes +a ``lua_State *``, pops a value off the Lua stack and converts it back into its +C type. + +There are two: ``lua_decode*`` and ``lua_to*``. The former does no mememory +allocation and is needed for ``frrscript_call``. +The latter performs allocation and is optional. + +A ``lua_decode_*`` function takes a ``lua_State*``, an index, and a pointer +to a C data structure, and directly modifies the structure with values from the +Lua stack. Note that only non const-qualified pointers may be modified; +``lua_decode_*`` for other types will be noops. + +Again, for ``struct prefix *``: .. code-block:: c - void *lua_toprefix(lua_State *L, int idx) + void lua_decode_prefix(lua_State *L, int idx, struct prefix *prefix) { - struct prefix *p = XCALLOC(MTYPE_TMP, sizeof(struct prefix)); + lua_getfield(L, idx, "network"); + (void)str2prefix(lua_tostring(L, -1), prefix); + /* pop the netork string */ + lua_pop(L, 1); + /* pop the prefix table */ + lua_pop(L, 1); + } + + +Note: + - Before ``lua_decode*`` is run, the "prefix" table is already on the top of + the stack. ``frrscript_call`` does this for us. + - However, at the end of ``lua_decode*``, the "prefix" table should be popped. + - The other two fields in the "network" table are disregarded, meaning that any + modification to them is discarded in C space. In this case, this is desired + behavior. + +.. warning:: + + ``lua_decode*`` functions should pop all values that ``lua_to*`` pushed onto + the Lua stack. + For encoders that pushed a table, its decoder should pop the table at the end. + The above is an example. + + + +``int`` is not a non const-qualified pointer, so for ``int``: - lua_getfield(L, idx, "network"); - str2prefix(lua_tostring(L, -1), p); - lua_pop(L, 1); +.. code-block:: c + + void lua_decode_int_noop(lua_State *L, int idx, int i) + { //noop + } + + +A ``lua_to*`` function provides identical functionality except that it first +allocates memory for the new C type before decoding the value from the Lua stack, +then returns a pointer to the newly allocated C type. You only need to implement +this function to use with ``frrscript_get_result`` to retrieve a result of +this type. + +This function can and should be implemented using ``lua_decode_*``: +.. code-block:: c + + void *lua_toprefix(lua_State *L, int idx) + { + struct prefix *p = XCALLOC(MTYPE_SCRIPT_RES, sizeof(struct prefix)); + + lua_decode_prefix(L, idx, p); return p; } -By convention these functions should be called ``lua_to*``, as this is the -naming convention used by the Lua C library for the basic types e.g. -``lua_tointeger`` and ``lua_tostring``. The returned data must always be copied off the stack and the copy must be -allocated with ``MTYPE_TMP``. This way it is possible to unload the script +allocated with ``MTYPE_SCRIPT_RES``. This way it is possible to unload the script (destroy the state) without invalidating any references to values stored in it. +Note that it is the caller's responsibility to free the data. -To register a new type with its corresponding encoding functions: -.. code-block:: c +Registering encoders and decoders for frrscript_call +"""""""""""""""""""""""""""""""""""""""""""""""""""" - struct frrscript_codec frrscript_codecs_lib[] = { - {.typename = "prefix", - .encoder = (encoder_func)lua_pushprefix, - .decoder = lua_toprefix}, - {.typename = "sockunion", - .encoder = (encoder_func)lua_pushsockunion, - .decoder = lua_tosockunion}, - ... - {}}; +To register a new type with its ``lua_push*`` and ``lua_decode*`` functions, +add the mapping in the following macros in ``frrscript.h``: - frrscript_register_type_codecs(frrscript_codecs_lib); +.. code-block:: diff + + #define ENCODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + - struct peer * : lua_pushpeer \ + + struct peer * : lua_pushpeer, \ + + struct prefix * : lua_pushprefix \ + )((L), (value)) + + #define DECODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + - struct peer * : lua_decode_peer \ + + struct peer * : lua_decode_peer, \ + + struct prefix * : lua_decode_prefix \ + )((L), -1, (value)) + + +At compile time, the compiler will search for encoders/decoders for the type of +each value passed in via ``frrscript_call``. If a encoder/decoder cannot be +found, it will appear as a compile warning. Note that the types must +match *exactly*. +In the above example, we defined encoders/decoders for a value of +``struct prefix *``, but not ``struct prefix`` or ``const struct prefix *``. + +``const`` values are a special case. We want to use them in our Lua scripts +but not modify them, so creating a decoder for them would be meaningless. +But we still need a decoder for the type of value so that the compiler will be +satisfied. +For that, use ``lua_decode_noop``: + +.. code-block:: diff + + #define DECODE_ARGS_WITH_STATE(L, value) \ + _Generic((value), \ + ... + + const struct prefix * : lua_decode_noop \ + )(L, -1, value) -From this point on the type names are available to be used when calling any -script and getting its results. .. note:: - Marshalled types are not restricted to simple values like integers, strings - and tables. It is possible to marshall a type such that the resultant object - in Lua is an actual object-oriented object, complete with methods that call - back into defined C functions. See the Lua manual for how to do this; for a - code example, look at how zlog is exported into the script environment. + Encodable/decodable types are not restricted to simple values like integers, + strings and tables. + It is possible to encode a type such that the resultant object in Lua + is an actual object-oriented object, complete with methods that call + back into defined C functions. See the Lua manual for how to do this; + for a code example, look at how zlog is exported into the script environment. Script Environment @@ -356,10 +526,11 @@ Examples For a complete code example involving passing custom types, retrieving results, and doing complex calculations in Lua, look at the implementation of the ``match script SCRIPT`` command for BGP routemaps. This example calls into a -script with a route prefix and attributes received from a peer and expects the -script to return a match / no match / match and update result. +script with a function named ``route_match``, +provides route prefix and attributes received from a peer and expects the +function to return a match / no match / match and update result. -An example script to use with this follows. This script matches, does not match +An example script to use with this follows. This function matches, does not match or updates a route depending on how many BGP UPDATE messages the peer has received when the script is called, simply as a demonstration of what can be accomplished with scripting. @@ -370,64 +541,75 @@ accomplished with scripting. -- Example route map matching -- author: qlyoung -- - -- The following variables are available to us: + -- The following variables are available in the global environment: -- log -- logging library, with the usual functions - -- prefix + -- + -- route_match arguments: + -- table prefix -- the route under consideration - -- attributes + -- table attributes -- the route's attributes - -- peer + -- table peer -- the peer which received this route - -- RM_FAILURE + -- integer RM_FAILURE -- status code in case of failure - -- RM_NOMATCH + -- integer RM_NOMATCH -- status code for no match - -- RM_MATCH + -- integer RM_MATCH -- status code for match - -- RM_MATCH_AND_CHANGE + -- integer RM_MATCH_AND_CHANGE -- status code for match-and-set -- - -- We need to set the following out values: - -- action - -- Set to the appropriate status code to indicate what we did - -- attributes - -- Setting fields on here will propagate them back up to the caller if - -- 'action' is set to RM_MATCH_AND_CHANGE. - - - log.info("Evaluating route " .. prefix.network .. " from peer " .. peer.remote_id.string) - - function on_match (prefix, attrs) - log.info("Match") - action = RM_MATCH - end - - function on_nomatch (prefix, attrs) - log.info("No match") - action = RM_NOMATCH - end - - function on_match_and_change (prefix, attrs) - action = RM_MATCH_AND_CHANGE - log.info("Match and change") - attrs["metric"] = attrs["metric"] + 7 - end - - special_routes = { - ["172.16.10.4/24"] = on_match, - ["172.16.13.1/8"] = on_nomatch, - ["192.168.0.24/8"] = on_match_and_change, - } + -- route_match returns table with following keys: + -- integer action, required + -- resultant status code. Should be one of RM_* + -- table attributes, optional + -- updated route attributes + -- + + function route_match(prefix, attributes, peer, + RM_FAILURE, RM_NOMATCH, RM_MATCH, RM_MATCH_AND_CHANGE) + + log.info("Evaluating route " .. prefix.network .. " from peer " .. peer.remote_id.string) + function on_match (prefix, attributes) + log.info("Match") + return { + attributes = RM_MATCH + } + end - if special_routes[prefix.network] then - special_routes[prefix.network](prefix, attributes) - elseif peer.stats.update_in % 3 == 0 then - on_match(prefix, attributes) - elseif peer.stats.update_in % 2 == 0 then - on_nomatch(prefix, attributes) - else - on_match_and_change(prefix, attributes) - end - + function on_nomatch (prefix, attributes) + log.info("No match") + return { + action = RM_NOMATCH + } + end + + function on_match_and_change (prefix, attributes) + log.info("Match and change") + attributes["metric"] = attributes["metric"] + 7 + return { + action = RM_MATCH_AND_CHANGE, + attributes = attributes + } + end + + special_routes = { + ["172.16.10.4/24"] = on_match, + ["172.16.13.1/8"] = on_nomatch, + ["192.168.0.24/8"] = on_match_and_change, + } + + + if special_routes[prefix.network] then + return special_routes[prefix.network](prefix, attributes) + elseif peer.stats.update_in % 3 == 0 then + return on_match(prefix, attributes) + elseif peer.stats.update_in % 2 == 0 then + return on_nomatch(prefix, attributes) + else + return on_match_and_change(prefix, attributes) + end + end |