From 0f84d13839ae6b05639a65ce01e05200a3309131 Mon Sep 17 00:00:00 2001 From: David Schweizer Date: Thu, 21 Jan 2021 16:22:39 +0100 Subject: [PATCH] doc: developer documentation for pytest markers Signed-off-by: David Schweizer --- doc/developer/topotests-markers.rst | 115 ++++++++++++++++++++++++++++ doc/developer/topotests.rst | 4 + 2 files changed, 119 insertions(+) create mode 100644 doc/developer/topotests-markers.rst diff --git a/doc/developer/topotests-markers.rst b/doc/developer/topotests-markers.rst new file mode 100644 index 0000000000..02ffe3f777 --- /dev/null +++ b/doc/developer/topotests-markers.rst @@ -0,0 +1,115 @@ +.. _topotests-markers: + +Markers +-------- + +To allow for automated selective testing on large scale continuous integration +systems, all tests must be marked with at least one of the following markers: + +* babeld +* bfdd +* bgpd +* eigrpd +* isisd +* ldpd +* nhrpd +* ospf6d +* ospfd +* pathd +* pbrd +* pimd +* ripd +* ripngd +* sharpd +* staticd +* vrrpd + +The markers corespond to the daemon subdirectories in FRR's source code and have +to be added to tests on a module level depending on which daemons are used +during the test. + +The goal is to have continuous integration systems scan code submissions, detect +changes to files in a daemons subdirectory and select only tests using that +daemon to run to shorten developers waiting times for test results and save test +infrastructure resources. + +Newly written modules and code changes on tests, which do not contain any or +incorrect markers will be rejected by reviewers. + + +Registering markers +^^^^^^^^^^^^^^^^^^^ +The Registration of new markers takes place in the file +``tests/topotests/pytest.ini`` and should be discussed with members of the TSC +beforehand. + +.. code:: python3 + + # tests/topotests/pytest.ini + [pytest] + ... + markers = + babeld: Tests that run against BABELD + bfdd: Tests that run against BFDD + ... + vrrpd: Tests that run against VRRPD + + +Adding markers to tests +^^^^^^^^^^^^^^^^^^^^^^^ +Markers are added to a test by placing a global variable in the test module. + +Adding a single marker: + +.. code:: python3 + + import pytest + + ... + + pytestmark = pytest.mark.bfdd + + ... + + def test_using_bfdd(): + + +Adding multiple markers: + +.. code:: python3 + + import pytest + + ... + + pytestmark = [ + pytest.mark.bgpd, + pytest.mark.ospfd, + pytest.mark.ospf6d + ] + + ... + + def test_using_bgpd_ospfd_ospf6d(): + + +Selecting marked modules fort testing +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Selecting by a single marker: + +.. code:: bash + + pytest -v -m isisd + +Selecting by multiple markers: + +.. code:: bash + + pytest -v -m "isisd or ldpd or nhrpd" + + +Further Information +^^^^^^^^^^^^^^^^^^^ +The `online pytest documentation `_ +provides further information and usage examples for pytest markers. + diff --git a/doc/developer/topotests.rst b/doc/developer/topotests.rst index 688f73c991..3e8987f126 100644 --- a/doc/developer/topotests.rst +++ b/doc/developer/topotests.rst @@ -792,6 +792,8 @@ Requirements: conforms with this, run it without the :option:`-s` parameter. - Use `black `_ code formatter before creating a pull request. This ensures we have a unified code style. +- Mark test modules with pytest markers depending on the daemons used during the + tests (s. Markers) Tips: @@ -950,6 +952,8 @@ does what you need. If nothing is similar, then you may create a new topology, preferably, using the newest template (:file:`tests/topotests/example-test/test_template.py`). +.. include:: topotests-markers.rst + .. include:: topotests-snippets.rst License -- 2.39.5