summaryrefslogtreecommitdiff
path: root/doc/developer
diff options
context:
space:
mode:
Diffstat (limited to 'doc/developer')
-rw-r--r--doc/developer/.gitignore3
-rw-r--r--doc/developer/BGP-TypeCode24
-rw-r--r--doc/developer/Building_FRR_on_CentOS6.rst255
-rw-r--r--doc/developer/Building_FRR_on_CentOS7.rst169
-rw-r--r--doc/developer/Building_FRR_on_Debian8.rst156
-rw-r--r--doc/developer/Building_FRR_on_Debian9.rst131
-rw-r--r--doc/developer/Building_FRR_on_Fedora24.rst177
-rw-r--r--doc/developer/Building_FRR_on_FreeBSD10.rst109
-rw-r--r--doc/developer/Building_FRR_on_FreeBSD11.rst109
-rw-r--r--doc/developer/Building_FRR_on_FreeBSD9.rst122
-rw-r--r--doc/developer/Building_FRR_on_LEDE-OpenWRT.rst108
-rw-r--r--doc/developer/Building_FRR_on_NetBSD6.rst147
-rw-r--r--doc/developer/Building_FRR_on_NetBSD7.rst137
-rw-r--r--doc/developer/Building_FRR_on_OmniOS.rst146
-rw-r--r--doc/developer/Building_FRR_on_OpenBSD6.rst169
-rw-r--r--doc/developer/Building_FRR_on_Ubuntu1204.rst187
-rw-r--r--doc/developer/Building_FRR_on_Ubuntu1404.rst139
-rw-r--r--doc/developer/Building_FRR_on_Ubuntu1604.rst176
-rw-r--r--doc/developer/Makefile1
-rw-r--r--doc/developer/OSPF-API.md263
-rw-r--r--doc/developer/bgpd.rst8
-rw-r--r--doc/developer/building.rst22
-rw-r--r--doc/developer/cli.rst723
-rw-r--r--doc/developer/conf.py341
-rw-r--r--doc/developer/dev-modules.md119
-rw-r--r--doc/developer/draft-zebra-00.ms209
-rw-r--r--doc/developer/hooks.rst171
-rw-r--r--doc/developer/index.rst13
-rw-r--r--doc/developer/ldpd-basic-test-setup.md681
-rw-r--r--doc/developer/library.rst12
-rw-r--r--doc/developer/memtypes.rst117
-rw-r--r--doc/developer/modules.rst123
-rw-r--r--doc/developer/next-hop-tracking.rst352
-rw-r--r--doc/developer/ospf-api.rst388
-rw-r--r--doc/developer/ospf-sr.rst285
-rw-r--r--doc/developer/workflow.rst657
36 files changed, 6949 insertions, 0 deletions
diff --git a/doc/developer/.gitignore b/doc/developer/.gitignore
new file mode 100644
index 0000000000..0505537159
--- /dev/null
+++ b/doc/developer/.gitignore
@@ -0,0 +1,3 @@
+/_templates
+/_build
+!/Makefile
diff --git a/doc/developer/BGP-TypeCode b/doc/developer/BGP-TypeCode
new file mode 100644
index 0000000000..b3218079cc
--- /dev/null
+++ b/doc/developer/BGP-TypeCode
@@ -0,0 +1,24 @@
+
+ BGP-4[+] UPDATE Attribute TypeCode list
+
+ Value Attribute References
+=========================================================================
+ 1 ORIGIN [RFC 4271]
+ 2 AS_PATH [RFC 4271]
+ 3 NEXT_HOP [RFC 4271]
+ 4 MULTI_EXIT_DISC [RFC 4271]
+ 5 LOCAL_PREF [RFC 4271]
+ 6 ATOMIC_AGGREGATE [RFC 4271]
+ 7 AGGREGATOR [RFC 4271]
+ 8 COMMUNITIES [RFC 1997]
+ 9 ORIGINATOR_ID [RFC 4456]
+ 10 CLUSTER_LIST [RFC 4456]
+ 11 DPA [draft-ietf-idr-bgp-dpa-05.txt(expired)]
+ 12 ADVERTISER [RFC 1863]
+ 13 RCID_PATH [RFC 1863]
+ 14 MP_REACH_NLRI [RFC 4760]
+ 15 MP_UNREACH_NLRI [RFC 4760]
+ 16 EXT_COMMUNITIES [RFC 4360]
+ 17 AS4_PATH [RFC 4893]
+ 18 AS4_AGGREGATOR [RFC 4893]
+=========================================================================
diff --git a/doc/developer/Building_FRR_on_CentOS6.rst b/doc/developer/Building_FRR_on_CentOS6.rst
new file mode 100644
index 0000000000..1c53ea6ba3
--- /dev/null
+++ b/doc/developer/Building_FRR_on_CentOS6.rst
@@ -0,0 +1,255 @@
+CentOS 6
+========================================
+
+(As an alternative to this installation, you may prefer to create a FRR
+rpm package yourself and install that package instead. See instructions
+in redhat/README.rpm\_build.md on how to build a rpm package)
+
+Instructions are tested with ``CentOS 6.8`` on ``x86_64`` platform
+
+Warning:
+--------
+``CentOS 6`` is very old and not fully supported by the FRR community
+anymore. Building FRR takes multiple manual steps to update the build
+system with newer packages than what's available from the archives.
+However, the built packages can still be installed afterwards on
+a standard ``CentOS 6`` without any special packages.
+
+Support for CentOS 6 is now on a best-effort base by the community.
+
+CentOS 6 restrictions:
+----------------------
+
+- PIMd is not supported on ``CentOS 6``. Upgrade to ``CentOS 7`` if
+ PIMd is needed
+- MPLS is not supported on ``CentOS 6``. MPLS requires Linux Kernel 4.5
+ or higher (LDP can be built, but may have limited use without MPLS)
+- Zebra is unable to detect what bridge/vrf an interface is associcated
+ with (IFLA\_INFO\_SLAVE\_KIND does not exist in the kernel headers,
+ you can use a newer kernel + headers to get this functionality)
+- frr\_reload.py will not work, as this requires Python 2.7, and CentOS
+ 6 only has 2.6. You can install Python 2.7 via IUS, but it won't work
+ properly unless you compile and install the ipaddr package for it.
+- Building the package requires Sphinx >= 1.1. Only a non-standard
+ package provides a newer sphinx and requires manual installation
+ (see below)
+
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ sudo yum install git autoconf automake libtool make gawk \
+ readline-devel texinfo net-snmp-devel groff pkgconfig \
+ json-c-devel pam-devel flex epel-release perl-XML-LibXML \
+ c-ares-devel
+
+Install newer version of bison (CentOS 6 package source is too old) from
+CentOS 7
+
+::
+
+ sudo yum install rpm-build
+ curl -O http://vault.centos.org/7.0.1406/os/Source/SPackages/bison-2.7-4.el7.src.rpm
+ rpmbuild --rebuild ./bison-2.7-4.el7.src.rpm
+ sudo yum install ./rpmbuild/RPMS/x86_64/bison-2.7-4.el6.x86_64.rpm
+ rm -rf rpmbuild
+
+Install newer version of autoconf and automake (Package versions are too
+old)
+
+::
+
+ curl -O http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
+ tar xvf autoconf-2.69.tar.gz
+ cd autoconf-2.69
+ ./configure --prefix=/usr
+ make
+ sudo make install
+ cd ..
+
+ curl -O http://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz
+ tar xvf automake-1.15.tar.gz
+ cd automake-1.15
+ ./configure --prefix=/usr
+ make
+ sudo make install
+ cd ..
+
+Install ``Python 2.7`` in parallel to default 2.6. Make sure you've
+install EPEL (``epel-release`` as above). Then install current
+``python27``, ``python27-devel`` and ``pytest``
+
+::
+
+ sudo rpm -ivh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
+ sudo rpm -ivh https://centos6.iuscommunity.org/ius-release.rpm
+ sudo yum install python27 python27-pip python27-devel
+ sudo pip2.7 install pytest
+
+Please note that ``CentOS 6`` needs to keep python pointing to version
+2.6 for ``yum`` to keep working, so don't create a symlink for python2.7
+to python
+
+Install newer ``Sphinx-Build`` based on ``Python 2.7``
+
+Create a new repo ``/etc/yum.repos.d/puias6.repo`` with the following contents:
+
+::
+
+ ### Name: RPM Repository for RHEL 6 - PUIAS (used for Sphinx-Build)
+ ### URL: http://springdale.math.ias.edu/data/puias/computational
+ [puias-computational]
+ name = RPM Repository for RHEL 6 - Sphinx-Build
+ baseurl = http://springdale.math.ias.edu/data/puias/computational/$releasever/$basearch
+ #mirrorlist =
+ enabled = 1
+ protect = 0
+ gpgkey =
+ gpgcheck = 0
+
+Update rpm database & Install newer sphinx
+
+::
+
+ sudo yum update
+ sudo yum install python27-sphinx
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -r -g 85 frrvt
+ sudo useradd -u 92 -g 92 -M -r -G frrvt -s /sbin/nologin \
+ -c "FRR FRRouting suite" -d /var/run/frr frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --bindir=/usr/bin \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --libdir=/usr/lib/frr \
+ --libexecdir=/usr/lib/frr \
+ --localstatedir=/var/run/frr \
+ --with-moduledir=/usr/lib/frr/modules \
+ --disable-pimd \
+ --enable-snmp=agentx \
+ --enable-multipath=64 \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvt \
+ --enable-rtadv \
+ --disable-exampledir \
+ --enable-watchfrr \
+ --disable-ldpd \
+ --enable-fpm \
+ --enable-nhrpd \
+ --enable-eigrpd \
+ --enable-babeld \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make SPHINXBUILD=sphinx-build2.7
+ make check PYTHON=/usr/bin/python2.7 SPHINXBUILD=sphinx-build2.7
+ sudo make SPHINXBUILD=sphinx-build2.7 install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /var/log/frr
+ sudo mkdir /etc/frr
+ sudo touch /etc/frr/zebra.conf
+ sudo touch /etc/frr/bgpd.conf
+ sudo touch /etc/frr/ospfd.conf
+ sudo touch /etc/frr/ospf6d.conf
+ sudo touch /etc/frr/isisd.conf
+ sudo touch /etc/frr/ripd.conf
+ sudo touch /etc/frr/ripngd.conf
+ sudo touch /etc/frr/nhrpd.conf
+ sudo touch /etc/frr/eigrpd.conf
+ sudo touch /etc/frr/babeld.conf
+ sudo chown -R frr:frr /etc/frr/
+ sudo touch /etc/frr/vtysh.conf
+ sudo chown frr:frrvt /etc/frr/vtysh.conf
+ sudo chmod 640 /etc/frr/*.conf
+
+Install daemon config file
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -p -m 644 redhat/daemons /etc/frr/
+ sudo chown frr:frr /etc/frr/daemons
+
+Edit /etc/frr/daemons as needed to select the required daemons
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Look for the section with ``watchfrr_enable=...`` and ``zebra=...`` etc.
+Enable the daemons as required by changing the value to ``yes``
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and set the following values (ignore the other
+settings)
+
+::
+
+ # Controls IP packet forwarding
+ net.ipv4.ip_forward = 1
+ net.ipv6.conf.all.forwarding=1
+
+ # Controls source route verification
+ net.ipv4.conf.default.rp_filter = 0
+
+Load the modifed sysctl's on the system:
+
+::
+
+ sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf
+
+Add init.d startup files
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -p -m 755 redhat/frr.init /etc/init.d/frr
+ sudo chkconfig --add frr
+
+Enable frr daemon at startup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo chkconfig frr on
+
+Start FRR manually (or reboot)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo /etc/init.d/frr start
diff --git a/doc/developer/Building_FRR_on_CentOS7.rst b/doc/developer/Building_FRR_on_CentOS7.rst
new file mode 100644
index 0000000000..9807543e17
--- /dev/null
+++ b/doc/developer/Building_FRR_on_CentOS7.rst
@@ -0,0 +1,169 @@
+CentOS 7
+========================================
+
+(As an alternative to this installation, you may prefer to create a FRR
+rpm package yourself and install that package instead. See instructions
+in redhat/README.rpm\_build.md on how to build a rpm package)
+
+CentOS 7 restrictions:
+----------------------
+
+- MPLS is not supported on ``CentOS 7`` with default kernel. MPLS
+ requires Linux Kernel 4.5 or higher (LDP can be built, but may have
+ limited use without MPLS)
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ sudo yum install git autoconf automake libtool make gawk \
+ readline-devel texinfo net-snmp-devel groff pkgconfig \
+ json-c-devel pam-devel bison flex pytest c-ares-devel \
+ perl-XML-LibXML python-devel systemd-devel python-sphinx
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -r -g 85 frrvt
+ sudo useradd -u 92 -g 92 -M -r -G frrvt -s /sbin/nologin \
+ -c "FRR FRRouting suite" -d /var/run/frr frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --bindir=/usr/bin \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --libdir=/usr/lib/frr \
+ --libexecdir=/usr/lib/frr \
+ --localstatedir=/var/run/frr \
+ --with-moduledir=/usr/lib/frr/modules \
+ --enable-pimd \
+ --enable-snmp=agentx \
+ --enable-multipath=64 \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvt \
+ --enable-rtadv \
+ --enable-systemd=yes \
+ --disable-exampledir \
+ --enable-watchfrr \
+ --disable-ldpd \
+ --enable-fpm \
+ --enable-nhrpd \
+ --enable-eigrpd \
+ --enable-babeld \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /var/log/frr
+ sudo mkdir /etc/frr
+ sudo touch /etc/frr/zebra.conf
+ sudo touch /etc/frr/bgpd.conf
+ sudo touch /etc/frr/ospfd.conf
+ sudo touch /etc/frr/ospf6d.conf
+ sudo touch /etc/frr/isisd.conf
+ sudo touch /etc/frr/ripd.conf
+ sudo touch /etc/frr/ripngd.conf
+ sudo touch /etc/frr/pimd.conf
+ sudo touch /etc/frr/nhrpd.conf
+ sudo touch /etc/frr/eigrpd.conf
+ sudo touch /etc/frr/babeld.conf
+ sudo chown -R frr:frr /etc/frr/
+ sudo touch /etc/frr/vtysh.conf
+ sudo chown frr:frrvt /etc/frr/vtysh.conf
+ sudo chmod 640 /etc/frr/*.conf
+
+Install daemon config file
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -p -m 644 redhat/daemons /etc/frr/
+ sudo chown frr:frr /etc/frr/daemons
+
+Edit /etc/frr/daemons as needed to select the required daemons
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Look for the section with ``watchfrr_enable=...`` and ``zebra=...`` etc.
+Enable the daemons as required by changing the value to ``yes``
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Create a new file ``/etc/sysctl.d/90-routing-sysctl.conf`` with the
+following content:
+
+::
+
+ # Sysctl for routing
+ #
+ # Routing: We need to forward packets
+ net.ipv4.conf.all.forwarding=1
+ net.ipv6.conf.all.forwarding=1
+
+Load the modifed sysctl's on the system:
+
+::
+
+ sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf
+
+Install frr Service and redhat init files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -p -m 644 redhat/frr.service /usr/lib/systemd/system/frr.service
+ sudo install -p -m 755 redhat/frr.init /usr/lib/frr/frr
+
+Register the systemd files
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo systemctl preset frr.service
+
+Enable required frr at startup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo systemctl enable frr
+
+Reboot or start FRR manually
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo systemctl start frr
diff --git a/doc/developer/Building_FRR_on_Debian8.rst b/doc/developer/Building_FRR_on_Debian8.rst
new file mode 100644
index 0000000000..36ed0c4e66
--- /dev/null
+++ b/doc/developer/Building_FRR_on_Debian8.rst
@@ -0,0 +1,156 @@
+Debian 8
+========================================
+
+Debian 8 restrictions:
+----------------------
+
+- MPLS is not supported on ``Debian 8`` with default kernel. MPLS
+ requires Linux Kernel 4.5 or higher (LDP can be built, but may have
+ limited use without MPLS)
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ sudo apt-get install git autoconf automake libtool make gawk \
+ libreadline-dev texinfo libjson-c-dev pkg-config bison flex \
+ python-pip libc-ares-dev python3-dev python3-sphinx
+
+Install newer pytest (>3.0) from pip
+
+::
+
+ sudo pip install pytest
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo addgroup --system --gid 92 frr
+ sudo addgroup --system --gid 85 frrvty
+ sudo adduser --system --ingroup frr --home /var/run/frr/ \
+ --gecos "FRR suite" --shell /bin/false frr
+ sudo usermod -a -G frrvty frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --enable-exampledir=/usr/share/doc/frr/examples/ \
+ --localstatedir=/var/run/frr \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --enable-vtysh \
+ --enable-isisd \
+ --enable-pimd \
+ --enable-watchfrr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --enable-ldpd \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 755 -o frr -g frr -d /var/log/frr
+ sudo install -m 775 -o frr -g frrvty -d /etc/frr
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf
+ sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the
+other settings)
+
+::
+
+ # Uncomment the next line to enable packet forwarding for IPv4
+ net.ipv4.ip_forward=1
+
+ # Uncomment the next line to enable packet forwarding for IPv6
+ # Enabling this option disables Stateless Address Autoconfiguration
+ # based on Router Advertisements for this host
+ net.ipv6.conf.all.forwarding=1
+
+**Reboot** or use ``sysctl -p`` to apply the same config to the running
+system
+
+Troubleshooting
+~~~~~~~~~~~~~~~
+
+**Local state directory**
+
+The local state directory must exist and have the correct permissions
+applied for the frrouting daemons to start. In the above ./configure
+example the local state directory is set to /var/run/frr
+(--localstatedir=/var/run/frr) Debian considers /var/run/frr to be
+temporary and this is removed after a reboot.
+
+When using a different local state directory you need to create the new
+directory and change the ownership to the frr user, for example:
+
+::
+
+ mkdir /var/opt/frr
+ chown frr /var/opt/frr
+
+**Shared library error**
+
+If you try and start any of the frrouting daemons you may see the below
+error due to the frrouting shared library directory not being found:
+
+::
+
+ ./zebra: error while loading shared libraries: libfrr.so.0: cannot open shared object file: No such file or directory
+
+The fix is to add the following line to /etc/ld.so.conf which will
+continue to reference the library directory after the system reboots. To
+load the library directory path immediately run the ldconfig command
+after adding the line to the file eg:
+
+::
+
+ echo include /usr/local/lib >> /etc/ld.so.conf
+ ldconfig
diff --git a/doc/developer/Building_FRR_on_Debian9.rst b/doc/developer/Building_FRR_on_Debian9.rst
new file mode 100644
index 0000000000..a1aa63236c
--- /dev/null
+++ b/doc/developer/Building_FRR_on_Debian9.rst
@@ -0,0 +1,131 @@
+Debian 9
+========================================
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ sudo apt-get install git autoconf automake libtool make \
+ libreadline-dev texinfo libjson-c-dev pkg-config bison flex \
+ python-pip libc-ares-dev python3-dev python-pytest python3-sphinx
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo addgroup --system --gid 92 frr
+ sudo addgroup --system --gid 85 frrvty
+ sudo adduser --system --ingroup frr --home /var/opt/frr/ \
+ --gecos "FRR suite" --shell /bin/false frr
+ sudo usermod -a -G frrvty frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ git checkout stable/3.0
+ ./bootstrap.sh
+ ./configure \
+ --enable-exampledir=/usr/share/doc/frr/examples/ \
+ --localstatedir=/var/opt/frr \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --enable-vtysh \
+ --enable-isisd \
+ --enable-pimd \
+ --enable-watchfrr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --enable-ldpd \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 755 -o frr -g frr -d /var/log/frr
+ sudo install -m 755 -o frr -g frr -d /var/opt/frr
+ sudo install -m 775 -o frr -g frrvty -d /etc/frr
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf
+ sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the
+other settings)
+
+::
+
+ # Uncomment the next line to enable packet forwarding for IPv4
+ net.ipv4.ip_forward=1
+
+ # Uncomment the next line to enable packet forwarding for IPv6
+ # Enabling this option disables Stateless Address Autoconfiguration
+ # based on Router Advertisements for this host
+ net.ipv6.conf.all.forwarding=1
+
+**Reboot** or use ``sysctl -p`` to apply the same config to the running
+system
+
+Troubleshooting
+---------------
+
+Shared library error
+~~~~~~~~~~~~~~~~~~~~
+
+If you try and start any of the frrouting daemons you may see the below
+error due to the frrouting shared library directory not being found:
+
+::
+
+ ./zebra: error while loading shared libraries: libfrr.so.0: cannot open
+ shared object file: No such file or directory
+
+The fix is to add the following line to /etc/ld.so.conf which will
+continue to reference the library directory after the system reboots. To
+load the library directory path immediately run the ldconfig command
+after adding the line to the file eg:
+
+::
+
+ echo include /usr/local/lib >> /etc/ld.so.conf
+ ldconfig
diff --git a/doc/developer/Building_FRR_on_Fedora24.rst b/doc/developer/Building_FRR_on_Fedora24.rst
new file mode 100644
index 0000000000..ed81d3f59c
--- /dev/null
+++ b/doc/developer/Building_FRR_on_Fedora24.rst
@@ -0,0 +1,177 @@
+Fedora 24
+=========================================
+
+(As an alternative to this installation, you may prefer to create a FRR
+rpm package yourself and install that package instead. See instructions
+in redhat/README.rpm\_build.md on how to build a rpm package)
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ sudo dnf install git autoconf automake libtool make gawk \
+ readline-devel texinfo net-snmp-devel groff pkgconfig \
+ json-c-devel pam-devel perl-XML-LibXML pytest bison flex \
+ c-ares-devel python3-devel python3-sphinx
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -r -g 85 frrvt
+ sudo useradd -u 92 -g 92 -M -r -G frrvt -s /sbin/nologin \
+ -c "FRR FRRouting suite" -d /var/run/frr frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --bindir=/usr/bin \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --libdir=/usr/lib/frr \
+ --libexecdir=/usr/lib/frr \
+ --localstatedir=/var/run/frr \
+ --with-moduledir=/usr/lib/frr/modules \
+ --enable-pimd \
+ --enable-snmp=agentx \
+ --enable-multipath=64 \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvt \
+ --enable-rtadv \
+ --disable-exampledir \
+ --enable-watchfrr \
+ --enable-ldpd \
+ --enable-fpm \
+ --enable-nhrpd \
+ --enable-eigrpd \
+ --enable-babeld \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /var/log/frr
+ sudo mkdir /etc/frr
+ sudo touch /etc/frr/zebra.conf
+ sudo touch /etc/frr/bgpd.conf
+ sudo touch /etc/frr/ospfd.conf
+ sudo touch /etc/frr/ospf6d.conf
+ sudo touch /etc/frr/isisd.conf
+ sudo touch /etc/frr/ripd.conf
+ sudo touch /etc/frr/ripngd.conf
+ sudo touch /etc/frr/pimd.conf
+ sudo touch /etc/frr/ldpd.conf
+ sudo touch /etc/frr/nhrpd.conf
+ sudo touch /etc/frr/eigrpd.conf
+ sudo touch /etc/frr/babeld.conf
+ sudo chown -R frr:frr /etc/frr/
+ sudo touch /etc/frr/vtysh.conf
+ sudo chown frr:frrvt /etc/frr/vtysh.conf
+ sudo chmod 640 /etc/frr/*.conf
+
+Install daemon config file
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -p -m 644 redhat/daemons /etc/frr/
+ sudo chown frr:frr /etc/frr/daemons
+
+Edit /etc/frr/daemons as needed to select the required daemons
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Look for the section with ``watchfrr_enable=...`` and ``zebra=...`` etc.
+Enable the daemons as required by changing the value to ``yes``
+
+Enable IP & IPv6 forwarding (and MPLS)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Create a new file ``/etc/sysctl.d/90-routing-sysctl.conf`` with the
+following content: (Please make sure to list all interfaces with
+required MPLS similar to ``net.mpls.conf.eth0.input=1``)
+
+::
+
+ # Sysctl for routing
+ #
+ # Routing: We need to forward packets
+ net.ipv4.conf.all.forwarding=1
+ net.ipv6.conf.all.forwarding=1
+ #
+ # Enable MPLS Label processing on all interfaces
+ net.mpls.conf.eth0.input=1
+ net.mpls.conf.eth1.input=1
+ net.mpls.conf.eth2.input=1
+ net.mpls.platform_labels=100000
+
+Load the modifed sysctl's on the system:
+
+::
+
+ sudo sysctl -p /etc/sysctl.d/90-routing-sysctl.conf
+
+Create a new file ``/etc/modules-load.d/mpls.conf`` with the following
+content:
+
+::
+
+ # Load MPLS Kernel Modules
+ mpls-router
+ mpls-iptunnel
+
+And load the kernel modules on the running system:
+
+::
+
+ sudo modprobe mpls-router mpls-iptunnel
+
+Install frr Service and redhat init files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -p -m 644 redhat/frr.service /usr/lib/systemd/system/frr.service
+ sudo install -p -m 755 redhat/frr.init /usr/lib/frr/frr
+
+Enable required frr at startup
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo systemctl enable frr
+
+Reboot or start FRR manually
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo systemctl start frr
diff --git a/doc/developer/Building_FRR_on_FreeBSD10.rst b/doc/developer/Building_FRR_on_FreeBSD10.rst
new file mode 100644
index 0000000000..ccbe8c55c3
--- /dev/null
+++ b/doc/developer/Building_FRR_on_FreeBSD10.rst
@@ -0,0 +1,109 @@
+FreeBSD 10
+==========================================
+
+FreeBSD 10 restrictions:
+------------------------
+
+- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel
+ (4.5 or higher). LDP can be built, but may have limited use without
+ MPLS
+
+Install required packages
+-------------------------
+
+Add packages: (Allow the install of the package managment tool if this
+is first package install and asked)
+
+::
+
+ pkg install git autoconf automake libtool gmake gawk json-c pkgconf \
+ bison flex py27-pytest c-ares python3 py-sphinx
+
+Make sure there is no /usr/bin/flex preinstalled (and use the newly
+installed in /usr/local/bin): (FreeBSD frequently provides a older flex
+as part of the base OS which takes preference in path)
+
+::
+
+ rm -f /usr/bin/flex
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr group and user
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ pw groupadd frr -g 101
+ pw groupadd frrvty -g 102
+ pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \
+ -d /usr/local/etc/frr -s /usr/sbin/nologin
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ export MAKE=gmake
+ export LDFLAGS="-L/usr/local/lib"
+ export CPPFLAGS="-I/usr/local/include"
+ ./configure \
+ --sysconfdir=/usr/local/etc/frr \
+ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \
+ --localstatedir=/var/run/frr \
+ --prefix=/usr/local \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ sudo gmake install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /usr/local/etc/frr
+ sudo touch /usr/local/etc/frr/zebra.conf
+ sudo touch /usr/local/etc/frr/bgpd.conf
+ sudo touch /usr/local/etc/frr/ospfd.conf
+ sudo touch /usr/local/etc/frr/ospf6d.conf
+ sudo touch /usr/local/etc/frr/isisd.conf
+ sudo touch /usr/local/etc/frr/ripd.conf
+ sudo touch /usr/local/etc/frr/ripngd.conf
+ sudo touch /usr/local/etc/frr/pimd.conf
+ sudo chown -R frr:frr /usr/local/etc/frr
+ sudo touch /usr/local/etc/frr/vtysh.conf
+ sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf
+ sudo chmod 640 /usr/local/etc/frr/*.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to the end of ``/etc/sysctl.conf``:
+
+::
+
+ # Routing: We need to forward packets
+ net.inet.ip.forwarding=1
+ net.inet6.ip6.forwarding=1
+
+**Reboot** or use ``sysctl`` to apply the same config to the running
+system
diff --git a/doc/developer/Building_FRR_on_FreeBSD11.rst b/doc/developer/Building_FRR_on_FreeBSD11.rst
new file mode 100644
index 0000000000..214fdbf9c8
--- /dev/null
+++ b/doc/developer/Building_FRR_on_FreeBSD11.rst
@@ -0,0 +1,109 @@
+FreeBSD 11
+==========================================
+
+FreeBSD 11 restrictions:
+------------------------
+
+- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel
+ (4.5 or higher). LDP can be built, but may have limited use without
+ MPLS
+
+Install required packages
+-------------------------
+
+Add packages: (Allow the install of the package managment tool if this
+is first package install and asked)
+
+::
+
+ pkg install git autoconf automake libtool gmake gawk json-c pkgconf \
+ bison flex py27-pytest c-ares python3 py-sphinx
+
+Make sure there is no /usr/bin/flex preinstalled (and use the newly
+installed in /usr/local/bin): (FreeBSD frequently provides a older flex
+as part of the base OS which takes preference in path)
+
+::
+
+ rm -f /usr/bin/flex
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr group and user
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ pw groupadd frr -g 101
+ pw groupadd frrvty -g 102
+ pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \
+ -d /usr/local/etc/frr -s /usr/sbin/nologin
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ export MAKE=gmake
+ export LDFLAGS="-L/usr/local/lib"
+ export CPPFLAGS="-I/usr/local/include"
+ ./configure \
+ --sysconfdir=/usr/local/etc/frr \
+ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \
+ --localstatedir=/var/run/frr \
+ --prefix=/usr/local \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ sudo gmake install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /usr/local/etc/frr
+ sudo touch /usr/local/etc/frr/zebra.conf
+ sudo touch /usr/local/etc/frr/bgpd.conf
+ sudo touch /usr/local/etc/frr/ospfd.conf
+ sudo touch /usr/local/etc/frr/ospf6d.conf
+ sudo touch /usr/local/etc/frr/isisd.conf
+ sudo touch /usr/local/etc/frr/ripd.conf
+ sudo touch /usr/local/etc/frr/ripngd.conf
+ sudo touch /usr/local/etc/frr/pimd.conf
+ sudo chown -R frr:frr /usr/local/etc/frr
+ sudo touch /usr/local/etc/frr/vtysh.conf
+ sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf
+ sudo chmod 640 /usr/local/etc/frr/*.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to the end of ``/etc/sysctl.conf``:
+
+::
+
+ # Routing: We need to forward packets
+ net.inet.ip.forwarding=1
+ net.inet6.ip6.forwarding=1
+
+**Reboot** or use ``sysctl`` to apply the same config to the running
+system
diff --git a/doc/developer/Building_FRR_on_FreeBSD9.rst b/doc/developer/Building_FRR_on_FreeBSD9.rst
new file mode 100644
index 0000000000..909b3a8d64
--- /dev/null
+++ b/doc/developer/Building_FRR_on_FreeBSD9.rst
@@ -0,0 +1,122 @@
+FreeBSD 9
+=========================================
+
+FreeBSD 9 restrictions:
+-----------------------
+
+- MPLS is not supported on ``FreeBSD``. MPLS requires a Linux Kernel
+ (4.5 or higher). LDP can be built, but may have limited use without
+ MPLS
+
+Install required packages
+-------------------------
+
+Add packages: (Allow the install of the package managment tool if this
+is first package install and asked)
+
+::
+
+ pkg install -y git autoconf automake libtool gmake gawk \
+ pkgconf texinfo json-c bison flex py27-pytest c-ares \
+ python3 py-sphinx
+
+Make sure there is no /usr/bin/flex preinstalled (and use the newly
+installed in /usr/local/bin): (FreeBSD frequently provides a older flex
+as part of the base OS which takes preference in path)
+
+::
+
+ rm -f /usr/bin/flex
+
+For building with clang (instead of gcc), upgrade clang from 3.4 default
+to 3.6 *This is needed to build FreeBSD packages as well - for packages
+clang is default* (Clang 3.4 as shipped with FreeBSD 9 crashes during
+compile)
+
+::
+
+ pkg install clang36
+ pkg delete clang34
+ mv /usr/bin/clang /usr/bin/clang34
+ ln -s /usr/local/bin/clang36 /usr/bin/clang
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr group and user
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ pw groupadd frr -g 101
+ pw groupadd frrvty -g 102
+ pw adduser frr -g 101 -u 101 -G 102 -c "FRR suite" \
+ -d /usr/local/etc/frr -s /usr/sbin/nologin
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ export MAKE=gmake
+ export LDFLAGS="-L/usr/local/lib"
+ export CPPFLAGS="-I/usr/local/include"
+ ./configure \
+ --sysconfdir=/usr/local/etc/frr \
+ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \
+ --localstatedir=/var/run/frr \
+ --prefix=/usr/local \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ sudo gmake install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /usr/local/etc/frr
+ sudo touch /usr/local/etc/frr/zebra.conf
+ sudo touch /usr/local/etc/frr/bgpd.conf
+ sudo touch /usr/local/etc/frr/ospfd.conf
+ sudo touch /usr/local/etc/frr/ospf6d.conf
+ sudo touch /usr/local/etc/frr/isisd.conf
+ sudo touch /usr/local/etc/frr/ripd.conf
+ sudo touch /usr/local/etc/frr/ripngd.conf
+ sudo touch /usr/local/etc/frr/pimd.conf
+ sudo chown -R frr:frr /usr/local/etc/frr
+ sudo touch /usr/local/etc/frr/vtysh.conf
+ sudo chown frr:frrvty /usr/local/etc/frr/vtysh.conf
+ sudo chmod 640 /usr/local/etc/frr/*.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to the end of ``/etc/sysctl.conf``:
+
+::
+
+ # Routing: We need to forward packets
+ net.inet.ip.forwarding=1
+ net.inet6.ip6.forwarding=1
+
+**Reboot** or use ``sysctl`` to apply the same config to the running
+system
diff --git a/doc/developer/Building_FRR_on_LEDE-OpenWRT.rst b/doc/developer/Building_FRR_on_LEDE-OpenWRT.rst
new file mode 100644
index 0000000000..b9ee9c51e1
--- /dev/null
+++ b/doc/developer/Building_FRR_on_LEDE-OpenWRT.rst
@@ -0,0 +1,108 @@
+OpenWRT/LEDE
+=============================================
+
+- for the moment because of cross compile problems, master is not
+ supported, only up to 3.0
+- LDP can't be built because of missing Perl-XML-LibXML in OpenWRT/LEDE
+ tree
+
+Prepare build environment
+-------------------------
+
+https://lede-project.org/docs/guide-developer/install-buildsystem
+
+for
+
+Ubuntu 12.04LTS:
+
+::
+
+ sudo apt-get install build-essential subversion git-core \
+ libncurses5-dev zlib1g-dev gawk flex quilt libssl-dev xsltproc \
+ libxml-parser-perl mercurial bzr ecj cvs unzip python3-sphinx
+
+Ubuntu 64bit:
+
+::
+
+ sudo apt-get install build-essential subversion libncurses5-dev zlib1g-dev \
+ gawk gcc-multilib flex git-core gettext libssl-dev python3-sphinx
+
+Debian 8 Jessie:
+
+::
+
+ sudo apt-get install build-essential libncurses5-dev gawk git subversion \
+ libssl-dev gettext unzip zlib1g-dev file python python3-sphinx
+
+Debian 9 Stretch:
+
+::
+
+ sudo apt-get install build-essential libncurses5-dev gawk git subversion \
+ libssl-dev gettext zlib1g-dev python3-sphinx
+
+Centos x86-64 (some packages require EPEL):
+
+::
+
+ yum install subversion binutils bzip2 gcc gcc-c++ gawk gettext flex \
+ ncurses-devel zlib-devel zlib-static make patch unzip glibc glibc-devel \
+ perl-ExtUtils-MakeMaker glibc-static quilt ncurses-libs sed sdcc bison \
+ intltool sharutils wget git-core openssl-devel xz python-sphinx
+
+Fedora 24 - 64Bit:
+
+::
+
+ dnf install -y subversion binutils bzip2 gcc gcc-c++ gawk gettext git-core \
+ unzip ncurses-devel ncurses-compat-libs zlib-devel zlib-static make \
+ flex patch perl-ExtUtils-MakeMaker perl-Thread-Queue glibc glibc-devel \
+ glibc-static quilt sed sdcc intltool sharutils bison wget openssl-devel \
+ python3-sphinx
+
+Get LEDE Sources (from Git)
+---------------------------
+
+LEDE and OpenWRT is planned to remerge and won't cover the similar
+OpenWRT build As normal user: git clone
+https://git.lede-project.org/source.git lede cd lede ./scripts/feeds
+update -a ./scripts/feeds install -a cd feeds/routing git pull origin
+pull/319/head ln -s ../../../feeds/routing/frr/
+../../package/feeds/routing/ cd ../.. make menuconfig
+
+Select the needed target then select needed packages in Network ->
+Routing and Redirection -> frr, exit and save
+
+::
+
+ make or make package/frr/compile
+
+It may be possible that on first build ``make package/frr/compile`` not
+to work and it may be needed to run a ``make`` for the entire build
+envronment, add V=s for debugging
+
+Work with sources
+-----------------
+
+To update the rc1 version or add other options, the Makefile is found in
+feeds/routing/frr
+
+edit: PKG\_VERSION:= PKG\_SOURCE\_VERSION:=
+
+Usage
+-----
+
+Edit ``/usr/sbin/frr.init`` and add/remove the daemons name in section
+DAEMONS= or don't install unneded packages For example: zebra bgpd ldpd
+isisd nhrpd ospfd ospf6d pimd ripd ripngd
+
+Enable the serivce
+~~~~~~~~~~~~~~~~~~
+
+- service frr enable
+
+Start the service
+~~~~~~~~~~~~~~~~~
+
+- service frr start
diff --git a/doc/developer/Building_FRR_on_NetBSD6.rst b/doc/developer/Building_FRR_on_NetBSD6.rst
new file mode 100644
index 0000000000..ce5c58045f
--- /dev/null
+++ b/doc/developer/Building_FRR_on_NetBSD6.rst
@@ -0,0 +1,147 @@
+NetBSD 6
+========================================
+
+NetBSD 6 restrictions:
+----------------------
+
+- MPLS is not supported on ``NetBSD``. MPLS requires a Linux Kernel
+ (4.5 or higher). LDP can be built, but may have limited use without
+ MPLS
+
+Install required packages
+-------------------------
+
+Configure Package location:
+
+::
+
+ PKG_PATH="ftp://ftp.NetBSD.org/pub/pkgsrc/packages/NetBSD/`uname -m`/`uname -r`/All"
+ export PKG_PATH
+
+Add packages:
+
+::
+
+ sudo pkg_add git autoconf automake libtool gmake gawk openssl \
+ pkg-config json-c python27 py27-test python35 py-sphinx
+
+Install SSL Root Certificates (for git https access):
+
+::
+
+ sudo pkg_add mozilla-rootcerts
+ sudo touch /etc/openssl/openssl.cnf
+ sudo mozilla-rootcerts install
+
+Select default Python and py.test
+
+::
+
+ sudo ln -s /usr/pkg/bin/python2.7 /usr/bin/python
+ sudo ln -s /usr/pkg/bin/py.test-2.7 /usr/bin/py.test
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -g 93 frrvty
+ sudo useradd -g 92 -u 92 -G frrvty -c "FRR suite" \
+ -d /nonexistent -s /sbin/nologin frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ MAKE=gmake
+ export LDFLAGS="-L/usr/pkg/lib -R/usr/pkg/lib"
+ export CPPFLAGS="-I/usr/pkg/include"
+ ./configure \
+ --sysconfdir=/usr/pkg/etc/frr \
+ --enable-exampledir=/usr/pkg/share/examples/frr \
+ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \
+ --localstatedir=/var/run/frr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ sudo gmake install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /var/log/frr
+ sudo mkdir /usr/pkg/etc/frr
+ sudo touch /usr/pkg/etc/frr/zebra.conf
+ sudo touch /usr/pkg/etc/frr/bgpd.conf
+ sudo touch /usr/pkg/etc/frr/ospfd.conf
+ sudo touch /usr/pkg/etc/frr/ospf6d.conf
+ sudo touch /usr/pkg/etc/frr/isisd.conf
+ sudo touch /usr/pkg/etc/frr/ripd.conf
+ sudo touch /usr/pkg/etc/frr/ripngd.conf
+ sudo touch /usr/pkg/etc/frr/pimd.conf
+ sudo chown -R frr:frr /usr/pkg/etc/frr
+ sudo touch /usr/local/etc/frr/vtysh.conf
+ sudo chown frr:frrvty /usr/pkg/etc/frr/*.conf
+ sudo chmod 640 /usr/pkg/etc/frr/*.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to the end of ``/etc/sysctl.conf``:
+
+::
+
+ # Routing: We need to forward packets
+ net.inet.ip.forwarding=1
+ net.inet6.ip6.forwarding=1
+
+**Reboot** or use ``sysctl`` to apply the same config to the running
+system
+
+Install rc.d init files
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ cp pkgsrc/*.sh /etc/rc.d/
+ chmod 555 /etc/rc.d/*.sh
+
+Enable FRR processes
+~~~~~~~~~~~~~~~~~~~~
+
+(Enable the required processes only)
+
+::
+
+ echo "zebra=YES" >> /etc/rc.conf
+ echo "bgpd=YES" >> /etc/rc.conf
+ echo "ospfd=YES" >> /etc/rc.conf
+ echo "ospf6d=YES" >> /etc/rc.conf
+ echo "isisd=YES" >> /etc/rc.conf
+ echo "ripngd=YES" >> /etc/rc.conf
+ echo "ripd=YES" >> /etc/rc.conf
+ echo "pimd=YES" >> /etc/rc.conf
diff --git a/doc/developer/Building_FRR_on_NetBSD7.rst b/doc/developer/Building_FRR_on_NetBSD7.rst
new file mode 100644
index 0000000000..eaaf87fd19
--- /dev/null
+++ b/doc/developer/Building_FRR_on_NetBSD7.rst
@@ -0,0 +1,137 @@
+NetBSD 7
+========================================
+
+NetBSD 7 restrictions:
+----------------------
+
+- MPLS is not supported on ``NetBSD``. MPLS requires a Linux Kernel
+ (4.5 or higher). LDP can be built, but may have limited use without
+ MPLS
+
+Install required packages
+-------------------------
+
+::
+
+ sudo pkgin install git autoconf automake libtool gmake gawk openssl \
+ pkg-config json-c python27 py27-test python35 py-sphinx
+
+Install SSL Root Certificates (for git https access):
+
+::
+
+ sudo pkgin install mozilla-rootcerts
+ sudo touch /etc/openssl/openssl.cnf
+ sudo mozilla-rootcerts install
+
+Select default Python and py.test
+
+::
+
+ sudo ln -s /usr/pkg/bin/python2.7 /usr/bin/python
+ sudo ln -s /usr/pkg/bin/py.test-2.7 /usr/bin/py.test
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -g 93 frrvty
+ sudo useradd -g 92 -u 92 -G frrvty -c "FRR suite" \
+ -d /nonexistent -s /sbin/nologin frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ MAKE=gmake
+ export LDFLAGS="-L/usr/pkg/lib -R/usr/pkg/lib"
+ export CPPFLAGS="-I/usr/pkg/include"
+ ./configure \
+ --sysconfdir=/usr/pkg/etc/frr \
+ --enable-exampledir=/usr/pkg/share/examples/frr \
+ --enable-pkgsrcrcdir=/usr/pkg/share/examples/rc.d \
+ --localstatedir=/var/run/frr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ sudo gmake install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo mkdir /usr/pkg/etc/frr
+ sudo touch /usr/pkg/etc/frr/zebra.conf
+ sudo touch /usr/pkg/etc/frr/bgpd.conf
+ sudo touch /usr/pkg/etc/frr/ospfd.conf
+ sudo touch /usr/pkg/etc/frr/ospf6d.conf
+ sudo touch /usr/pkg/etc/frr/isisd.conf
+ sudo touch /usr/pkg/etc/frr/ripd.conf
+ sudo touch /usr/pkg/etc/frr/ripngd.conf
+ sudo touch /usr/pkg/etc/frr/pimd.conf
+ sudo chown -R frr:frr /usr/pkg/etc/frr
+ sudo touch /usr/local/etc/frr/vtysh.conf
+ sudo chown frr:frrvty /usr/pkg/etc/frr/*.conf
+ sudo chmod 640 /usr/pkg/etc/frr/*.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to the end of ``/etc/sysctl.conf``:
+
+::
+
+ # Routing: We need to forward packets
+ net.inet.ip.forwarding=1
+ net.inet6.ip6.forwarding=1
+
+**Reboot** or use ``sysctl`` to apply the same config to the running
+system
+
+Install rc.d init files
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ cp pkgsrc/*.sh /etc/rc.d/
+ chmod 555 /etc/rc.d/*.sh
+
+Enable FRR processes
+~~~~~~~~~~~~~~~~~~~~
+
+(Enable the required processes only)
+
+::
+
+ echo "zebra=YES" >> /etc/rc.conf
+ echo "bgpd=YES" >> /etc/rc.conf
+ echo "ospfd=YES" >> /etc/rc.conf
+ echo "ospf6d=YES" >> /etc/rc.conf
+ echo "isisd=YES" >> /etc/rc.conf
+ echo "ripngd=YES" >> /etc/rc.conf
+ echo "ripd=YES" >> /etc/rc.conf
+ echo "pimd=YES" >> /etc/rc.conf
diff --git a/doc/developer/Building_FRR_on_OmniOS.rst b/doc/developer/Building_FRR_on_OmniOS.rst
new file mode 100644
index 0000000000..0eb2b9fec8
--- /dev/null
+++ b/doc/developer/Building_FRR_on_OmniOS.rst
@@ -0,0 +1,146 @@
+OmniOS (OpenSolaris)
+====================================================
+
+OmniOS restrictions:
+--------------------
+
+- MPLS is not supported on ``OmniOS`` or ``Solaris``. MPLS requires a
+ Linux Kernel (4.5 or higher). LDP can be built, but may have limited
+ use without MPLS
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ routeadm -e ipv4-forwarding
+ routeadm -e ipv6-forwarding
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ pkg install \
+ developer/build/autoconf \
+ developer/build/automake \
+ developer/lexer/flex \
+ developer/parser/bison \
+ developer/object-file \
+ developer/linker \
+ developer/library/lint \
+ developer/build/gnu-make \
+ developer/gcc51 \
+ library/idnkit \
+ library/idnkit/header-idnkit \
+ system/header \
+ system/library/math/header-math \
+ git libtool gawk pkg-config
+
+Add additional Solaris packages:
+
+::
+
+ pkgadd -d http://get.opencsw.org/now
+ /opt/csw/bin/pkgutil -U
+ /opt/csw/bin/pkgutil -y -i texinfo
+ /opt/csw/bin/pkgutil -y -i perl
+ /opt/csw/bin/pkgutil -y -i libjson_c_dev
+ /opt/csw/bin/pkgutil -y -i python27 py_pip python27_dev
+
+Add libjson to Solaris equivalent of ld.so.conf
+
+::
+
+ crle -l /opt/csw/lib -u
+
+Add pytest:
+
+::
+
+ pip install pytest
+
+Install Sphinx:::
+
+ pip install sphinx
+
+Select Python 2.7 as default (required for pytest)
+
+::
+
+ rm -f /usr/bin/python
+ ln -s /opt/csw/bin/python2.7 /usr/bin/python
+
+Fix PATH for all users and non-interactive sessions. Edit
+``/etc/default/login`` and add the following default PATH:
+
+::
+
+ PATH=/usr/gnu/bin:/usr/bin:/usr/sbin:/sbin:/opt/csw/bin
+
+Edit ``~/.profile`` and add the following default PATH:
+
+::
+
+ PATH=/usr/gnu/bin:/usr/bin:/usr/sbin:/sbin:/opt/csw/bin
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr group and user
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 93 frr
+ sudo groupadd -g 94 frrvty
+ sudo useradd -g 93 -u 93 -G frrvty -c "FRR suite" \
+ -d /nonexistent -s /bin/false frr
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ export MAKE=gmake
+ export LDFLAGS="-L/opt/csw/lib"
+ export CPPFLAGS="-I/opt/csw/include"
+ export PKG_CONFIG_PATH=/opt/csw/lib/pkgconfig
+ ./configure \
+ --sysconfdir=/etc/frr \
+ --enable-exampledir=/usr/share/doc/frr/examples/ \
+ --localstatedir=/var/run/frr \
+ --sbindir=/usr/lib/frr \
+ --enable-vtysh \
+ --enable-watchfrr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ sudo gmake install
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ routeadm -e ipv4-forwarding
+ routeadm -e ipv6-forwarding
diff --git a/doc/developer/Building_FRR_on_OpenBSD6.rst b/doc/developer/Building_FRR_on_OpenBSD6.rst
new file mode 100644
index 0000000000..895048886e
--- /dev/null
+++ b/doc/developer/Building_FRR_on_OpenBSD6.rst
@@ -0,0 +1,169 @@
+OpenBSD 6
+=========================================
+
+Install required packages
+-------------------------
+
+Configure PKG\_PATH
+
+::
+
+ export PKG_PATH=http://ftp5.usa.openbsd.org/pub/OpenBSD/$(uname -r)/packages/$(machine -a)/
+
+Add packages:
+
+::
+
+ pkg_add git autoconf-2.69p2 automake-1.15p0 libtool bison
+ pkg_add gmake gawk dejagnu openssl json-c py-test py-sphinx
+
+Select Python2.7 as default (required for pytest)
+
+::
+
+ ln -s /usr/local/bin/python2.7 /usr/local/bin/python
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr group and user
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ groupadd -g 525 _frr
+ groupadd -g 526 _frrvty
+ useradd -g 525 -u 525 -c "FRR suite" -G _frrvty \
+ -d /nonexistent -s /sbin/nologin _frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ export AUTOCONF_VERSION="2.69"
+ export AUTOMAKE_VERSION="1.15"
+ ./bootstrap.sh
+ export LDFLAGS="-L/usr/local/lib"
+ export CPPFLAGS="-I/usr/local/include"
+ ./configure \
+ --sysconfdir=/etc/frr \
+ --localstatedir=/var/frr \
+ --enable-pimd \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=_frr \
+ --enable-group=_frr \
+ --enable-vty-group=_frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ gmake
+ gmake check
+ doas gmake install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ doas mkdir /var/frr
+ doas chown _frr:_frr /var/frr
+ doas chmod 755 /var/frr
+ doas mkdir /etc/frr
+ doas touch /etc/frr/zebra.conf
+ doas touch /etc/frr/bgpd.conf
+ doas touch /etc/frr/ospfd.conf
+ doas touch /etc/frr/ospf6d.conf
+ doas touch /etc/frr/isisd.conf
+ doas touch /etc/frr/ripd.conf
+ doas touch /etc/frr/ripngd.conf
+ doas touch /etc/frr/pimd.conf
+ doas touch /etc/frr/ldpd.conf
+ doas touch /etc/frr/nhrpd.conf
+ doas chown -R _frr:_frr /etc/frr
+ doas touch /etc/frr/vtysh.conf
+ doas chown -R _frr:_frrvty /etc/frr/vtysh.conf
+ doas chmod 750 /etc/frr
+ doas chmod 640 /etc/frr/*.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to the end of ``/etc/rc.conf``:
+
+::
+
+ net.inet6.ip6.forwarding=1 # 1=Permit forwarding of IPv6 packets
+ net.inet6.ip6.mforwarding=1 # 1=Permit forwarding of IPv6 multicast packets
+ net.inet6.ip6.multipath=1 # 1=Enable IPv6 multipath routing
+
+**Reboot** to apply the config to the system
+
+Enable MPLS Forwarding
+~~~~~~~~~~~~~~~~~~~~~~
+
+To enable MPLS forwarding on a given interface, use the following
+command:
+
+::
+
+ doas ifconfig em0 mpls
+
+Alternatively, to make MPLS forwarding persistent across reboots, add
+the "mpls" keyword in the hostname.\* files of the desired interfaces.
+Example:
+
+::
+
+ cat /etc/hostname.em0
+ inet 10.0.1.1 255.255.255.0 mpls
+
+Install rc.d init files
+~~~~~~~~~~~~~~~~~~~~~~~
+
+(create them in /etc/rc.d - no example are included at this time with
+FRR source)
+
+Example (for zebra - store as ``/etc/rc.d/frr_zebra.sh``)
+
+::
+
+ #!/bin/sh
+ #
+ # $OpenBSD: frr_zebra.rc,v 1.1 2013/04/18 20:29:08 sthen Exp $
+
+ daemon="/usr/local/sbin/zebra -d"
+
+ . /etc/rc.d/rc.subr
+
+ rc_cmd $1
+
+Enable FRR processes
+~~~~~~~~~~~~~~~~~~~~
+
+(Enable the required processes only)
+
+::
+
+ echo "frr_zebra=YES" >> /etc/rc.conf
+ echo "frr_bgpd=YES" >> /etc/rc.conf
+ echo "frr_ospfd=YES" >> /etc/rc.conf
+ echo "frr_ospf6d=YES" >> /etc/rc.conf
+ echo "frr_isisd=YES" >> /etc/rc.conf
+ echo "frr_ripngd=YES" >> /etc/rc.conf
+ echo "frr_ripd=YES" >> /etc/rc.conf
+ echo "frr_pimd=YES" >> /etc/rc.conf
+ echo "frr_ldpd=YES" >> /etc/rc.conf
diff --git a/doc/developer/Building_FRR_on_Ubuntu1204.rst b/doc/developer/Building_FRR_on_Ubuntu1204.rst
new file mode 100644
index 0000000000..68e476fec2
--- /dev/null
+++ b/doc/developer/Building_FRR_on_Ubuntu1204.rst
@@ -0,0 +1,187 @@
+Ubuntu 12.04LTS
+===============================================
+
+- MPLS is not supported on ``Ubuntu 12.04`` with default kernel. MPLS
+ requires Linux Kernel 4.5 or higher (LDP can be built, but may have
+ limited use without MPLS) For an updated Ubuntu Kernel, see
+ http://kernel.ubuntu.com/~kernel-ppa/mainline/
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ apt-get install git autoconf automake libtool make gawk libreadline-dev \
+ texinfo libpam0g-dev dejagnu libjson0-dev pkg-config libpam0g-dev \
+ libjson0-dev flex python-pip libc-ares-dev python3-dev python3-sphinx
+
+Install newer bison from 14.04 package source (Ubuntu 12.04 package
+source is too old)
+
+::
+
+ mkdir builddir
+ cd builddir
+ wget http://archive.ubuntu.com/ubuntu/pool/main/b/bison/bison_3.0.2.dfsg-2.dsc
+ wget http://archive.ubuntu.com/ubuntu/pool/main/b/bison/bison_3.0.2.dfsg.orig.tar.bz2
+ wget http://archive.ubuntu.com/ubuntu/pool/main/b/bison/bison_3.0.2.dfsg-2.debian.tar.gz
+ tar -jxvf bison_3.0.2.dfsg.orig.tar.bz2
+ cd bison-3.0.2.dfsg/
+ tar xzf ../bison_3.0.2.dfsg-2.debian.tar.gz
+ sudo apt-get build-dep bison
+ debuild -b -uc -us
+ cd ..
+ sudo dpkg -i ./libbison-dev_3.0.2.dfsg-2_amd64.deb ./bison_3.0.2.dfsg-2_amd64.deb
+ cd ..
+ rm -rf builddir
+
+Install newer version of autoconf and automake:
+
+::
+
+ wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
+ tar xvf autoconf-2.69.tar.gz
+ cd autoconf-2.69
+ ./configure --prefix=/usr
+ make
+ sudo make install
+ cd ..
+
+ wget http://ftp.gnu.org/gnu/automake/automake-1.15.tar.gz
+ tar xvf automake-1.15.tar.gz
+ cd automake-1.15
+ ./configure --prefix=/usr
+ make
+ sudo make install
+ cd ..
+
+Install pytest:
+
+::
+
+ pip install pytest
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -r -g 85 frrvty
+ sudo adduser --system --ingroup frr --home /var/run/frr/ \
+ --gecos "FRR suite" --shell /sbin/nologin frr
+ sudo usermod -a -G frrvty frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --prefix=/usr \
+ --enable-exampledir=/usr/share/doc/frr/examples/ \
+ --localstatedir=/var/run/frr \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --enable-pimd \
+ --enable-watchfrr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 755 -o frr -g frr -d /var/log/frr
+ sudo install -m 775 -o frr -g frrvty -d /etc/frr
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf
+ sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the
+other settings)
+
+::
+
+ # Uncomment the next line to enable packet forwarding for IPv4
+ net.ipv4.ip_forward=1
+
+ # Uncomment the next line to enable packet forwarding for IPv6
+ # Enabling this option disables Stateless Address Autoconfiguration
+ # based on Router Advertisements for this host
+ net.ipv6.conf.all.forwarding=1
+
+**Reboot** or use ``sysctl -p`` to apply the same config to the running
+system
+
+Install the init.d service
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 755 tools/frr /etc/init.d/frr
+ sudo install -m 644 tools/etc/frr/daemons /etc/frr/daemons
+ sudo install -m 644 tools/etc/frr/daemons.conf /etc/frr/daemons.conf
+ sudo install -m 644 -o frr -g frr tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf
+
+Enable daemons
+~~~~~~~~~~~~~~
+
+| Edit ``/etc/frr/daemons`` and change the value from "no" to "yes" for
+ those daemons you want to start by systemd.
+| For example.
+
+::
+
+ zebra=yes
+ bgpd=yes
+ ospfd=yes
+ ospf6d=yes
+ ripd=yes
+ ripngd=yes
+ isisd=yes
+
+Start the init.d service
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+- /etc/init.d/frr start
+- use ``/etc/init.d/frr status`` to check its status.
diff --git a/doc/developer/Building_FRR_on_Ubuntu1404.rst b/doc/developer/Building_FRR_on_Ubuntu1404.rst
new file mode 100644
index 0000000000..10944cb8e1
--- /dev/null
+++ b/doc/developer/Building_FRR_on_Ubuntu1404.rst
@@ -0,0 +1,139 @@
+Ubuntu 14.04LTS
+===============================================
+
+- MPLS is not supported on ``Ubuntu 14.04`` with default kernel. MPLS
+ requires Linux Kernel 4.5 or higher (LDP can be built, but may have
+ limited use without MPLS) For an updated Ubuntu Kernel, see
+ http://kernel.ubuntu.com/~kernel-ppa/mainline/
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ apt-get install git autoconf automake libtool make gawk libreadline-dev \
+ texinfo dejagnu pkg-config libpam0g-dev libjson-c-dev bison flex \
+ python-pytest libc-ares-dev python3-dev python3-sphinx
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -r -g 85 frrvty
+ sudo adduser --system --ingroup frr --home /var/run/frr/ \
+ --gecos "FRR suite" --shell /sbin/nologin frr
+ sudo usermod -a -G frrvty frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --prefix=/usr \
+ --enable-exampledir=/usr/share/doc/frr/examples/ \
+ --localstatedir=/var/run/frr \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --enable-pimd \
+ --enable-watchfrr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --enable-ldpd \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 755 -o frr -g frr -d /var/log/frr
+ sudo install -m 775 -o frr -g frrvty -d /etc/frr
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf
+ sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf
+
+Enable IP & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the
+other settings)
+
+::
+
+ # Uncomment the next line to enable packet forwarding for IPv4
+ net.ipv4.ip_forward=1
+
+ # Uncomment the next line to enable packet forwarding for IPv6
+ # Enabling this option disables Stateless Address Autoconfiguration
+ # based on Router Advertisements for this host
+ net.ipv6.conf.all.forwarding=1
+
+**Reboot** or use ``sysctl -p`` to apply the same config to the running
+system ### Install the init.d service
+
+::
+
+ sudo install -m 755 tools/frr /etc/init.d/frr
+ sudo install -m 644 tools/etc/frr/daemons /etc/frr/daemons
+ sudo install -m 644 tools/etc/frr/daemons.conf /etc/frr/daemons.conf
+ sudo install -m 644 -o frr -g frr tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf
+
+Enable daemons
+~~~~~~~~~~~~~~
+
+| Edit ``/etc/frr/daemons`` and change the value from "no" to "yes" for
+ those daemons you want to start by systemd.
+| For example.
+
+::
+
+ zebra=yes
+ bgpd=yes
+ ospfd=yes
+ ospf6d=yes
+ ripd=yes
+ ripngd=yes
+ isisd=yes
+
+Start the init.d service
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+- /etc/init.d/frr start
+- use ``/etc/init.d/frr status`` to check its status.
diff --git a/doc/developer/Building_FRR_on_Ubuntu1604.rst b/doc/developer/Building_FRR_on_Ubuntu1604.rst
new file mode 100644
index 0000000000..9c296f8edf
--- /dev/null
+++ b/doc/developer/Building_FRR_on_Ubuntu1604.rst
@@ -0,0 +1,176 @@
+Ubuntu 16.04LTS
+===============================================
+
+- MPLS is not supported on ``Ubuntu 16.04`` with default kernel. MPLS
+ requires Linux Kernel 4.5 or higher (LDP can be built, but may have
+ limited use without MPLS) For an updated Ubuntu Kernel, see
+ http://kernel.ubuntu.com/~kernel-ppa/mainline/
+
+Install required packages
+-------------------------
+
+Add packages:
+
+::
+
+ apt-get install git autoconf automake libtool make gawk libreadline-dev \
+ texinfo dejagnu pkg-config libpam0g-dev libjson-c-dev bison flex \
+ python-pytest libc-ares-dev python3-dev libsystemd-dev python-ipaddr \
+ python3-sphinx
+
+Get FRR, compile it and install it (from Git)
+---------------------------------------------
+
+**This assumes you want to build and install FRR from source and not
+using any packages**
+
+Add frr groups and user
+~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo groupadd -g 92 frr
+ sudo groupadd -r -g 85 frrvty
+ sudo adduser --system --ingroup frr --home /var/run/frr/ \
+ --gecos "FRR suite" --shell /sbin/nologin frr
+ sudo usermod -a -G frrvty frr
+
+Download Source, configure and compile it
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(You may prefer different options on configure statement. These are just
+an example.)
+
+::
+
+ git clone https://github.com/frrouting/frr.git frr
+ cd frr
+ ./bootstrap.sh
+ ./configure \
+ --prefix=/usr \
+ --enable-exampledir=/usr/share/doc/frr/examples/ \
+ --localstatedir=/var/run/frr \
+ --sbindir=/usr/lib/frr \
+ --sysconfdir=/etc/frr \
+ --enable-pimd \
+ --enable-watchfrr \
+ --enable-ospfclient=yes \
+ --enable-ospfapi=yes \
+ --enable-multipath=64 \
+ --enable-user=frr \
+ --enable-group=frr \
+ --enable-vty-group=frrvty \
+ --enable-configfile-mask=0640 \
+ --enable-logfile-mask=0640 \
+ --enable-rtadv \
+ --enable-fpm \
+ --enable-systemd=yes \
+ --with-pkg-git-version \
+ --with-pkg-extra-version=-MyOwnFRRVersion
+ make
+ make check
+ sudo make install
+
+Create empty FRR configuration files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 755 -o frr -g frr -d /var/log/frr
+ sudo install -m 775 -o frr -g frrvty -d /etc/frr
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/zebra.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/bgpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospfd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ospf6d.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/isisd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ripngd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/pimd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/ldpd.conf
+ sudo install -m 640 -o frr -g frr /dev/null /etc/frr/nhrpd.conf
+ sudo install -m 640 -o frr -g frrvty /dev/null /etc/frr/vtysh.conf
+
+Enable IPv4 & IPv6 forwarding
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and uncomment the following values (ignore the
+other settings)
+
+::
+
+ # Uncomment the next line to enable packet forwarding for IPv4
+ net.ipv4.ip_forward=1
+
+ # Uncomment the next line to enable packet forwarding for IPv6
+ # Enabling this option disables Stateless Address Autoconfiguration
+ # based on Router Advertisements for this host
+ net.ipv6.conf.all.forwarding=1
+
+Enable MPLS Forwarding (with Linux Kernel >= 4.5)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Edit ``/etc/sysctl.conf`` and the following lines. Make sure to add a
+line equal to ``net.mpls.conf.eth0.input`` or each interface used with
+MPLS
+
+::
+
+ # Enable MPLS Label processing on all interfaces
+ net.mpls.conf.eth0.input=1
+ net.mpls.conf.eth1.input=1
+ net.mpls.conf.eth2.input=1
+ net.mpls.platform_labels=100000
+
+Add MPLS kernel modules
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Add the following lines to ``/etc/modules-load.d/modules.conf``:
+
+::
+
+ # Load MPLS Kernel Modules
+ mpls-router
+ mpls-iptunnel
+
+**Reboot** or use ``sysctl -p`` to apply the same config to the running
+system
+
+Install the systemd service (if rebooted from last step, change directory back to frr directory)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ sudo install -m 644 tools/frr.service /etc/systemd/system/frr.service
+ sudo install -m 644 tools/etc/default/frr /etc/default/frr
+ sudo install -m 644 tools/etc/frr/daemons /etc/frr/daemons
+ sudo install -m 644 tools/etc/frr/daemons.conf /etc/frr/daemons.conf
+ sudo install -m 644 tools/etc/frr/frr.conf /etc/frr/frr.conf
+ sudo install -m 644 -o frr -g frr tools/etc/frr/vtysh.conf /etc/frr/vtysh.conf
+
+Enable daemons
+~~~~~~~~~~~~~~
+
+| Edit ``/etc/frr/daemons`` and change the value from "no" to "yes" for
+ those daemons you want to start by systemd.
+| For example.
+
+::
+
+ zebra=yes
+ bgpd=yes
+ ospfd=yes
+ ospf6d=yes
+ ripd=yes
+ ripngd=yes
+ isisd=yes
+
+Enable the systemd service
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- systemctl enable frr
+
+Start the systemd service
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- systemctl start frr
+- use ``systemctl status frr`` to check its status.
diff --git a/doc/developer/Makefile b/doc/developer/Makefile
new file mode 100644
index 0000000000..9807a750bf
--- /dev/null
+++ b/doc/developer/Makefile
@@ -0,0 +1 @@
+include ../frr-sphinx.mk
diff --git a/doc/developer/OSPF-API.md b/doc/developer/OSPF-API.md
new file mode 100644
index 0000000000..5774300c0b
--- /dev/null
+++ b/doc/developer/OSPF-API.md
@@ -0,0 +1,263 @@
+# OSPF API Documentation
+
+[TOC]
+
+## Disclaimer
+
+The OSPF daemon contains an API for application access to the LSA database. This API was created by Ralph Keller, originally as patch for Zebra. Unfortunately, the page containing documentation of the API is no longer online. This page is an attempt to recreate documentation for the API (with lots of help of the WayBackMachine)
+
+## 1.  Introduction
+
+This page describes an API that allows external applications to access the link-state database (LSDB) of the OSPF daemon. The implementation is based on the OSPF code from FRRouting (forked from Quagga and formerly Zebra) routing protocol suite and is subject to the GNU General Public License. The OSPF API provides you with the following functionality:
+
+* Retrieval of the full or partial link-state database of the OSPF daemon. This allows applications to obtain an exact copy of the LSDB including router LSAs, network LSAs and so on. Whenever a new LSA arrives at the OSPF daemon, the API module immediately informs the application by sending a message. This way, the application is always synchronized with the LSDB of the OSPF daemon.
+* Origination of own opaque LSAs (of type 9, 10, or 11) which are then distributed transparently to other routers within the flooding scope and received by other applications through the OSPF API.
+
+Opaque LSAs, which are described in RFC 2370 , allow you to distribute application-specific information within a network using the OSPF protocol. The information contained in opaque LSAs is transparent for the routing process but it can be processed by other modules such as traffic engineering (e.g., MPLS-TE).
+
+## 2.  Architecture
+
+The following picture depicts the architecture of the Quagga/Zebra protocol suite. The OSPF daemon is extended with opaque LSA capabilities and an API for external applications. The OSPF core module executes the OSPF protocol by discovering neighbors and exchanging neighbor state. The opaque module, implemented by Masahiko Endo, provides functions to exchange opaque LSAs between routers. Opaque LSAs can be generated by several modules such as the MPLS-TE module or the API server module. These modules then invoke the opaque module to flood their data to neighbors within the flooding scope.
+
+The client, which is an application potentially running on a different node than the OSPF daemon, links against the OSPF API client library. This client library establishes a socket connection with the API server module of the OSPF daemon and uses this connection to retrieve LSAs and originate opaque LSAs.
+
+![image](ospf_api_architecture.png)
+
+The OSPF API server module works like any other internal opaque module (such as the MPLS-TE module), but listens to connections from external applications that want to communicate with the OSPF daemon. The API server module can handle multiple clients concurrently.
+
+One of the main objectives of the implementation is to make as little changes to the existing Zebra code as possible.
+
+## 3.  Installation & Configuration
+
+Download FRRouting and unpack
+
+Configure your frr version (note that --enable-opaque-lsa also enables the ospfapi server and ospfclient).
+
+```
+% update-autotools
+% sh ./configure --enable-opaque-lsa
+% make
+
+```
+
+This should also compile the client library and sample application in ospfclient.
+
+Make sure that you have enabled opaque LSAs in your configuration. Add the ospf opaque-lsa statement to your ospfd.conf:
+
+```
+! -*- ospf -*-
+!
+! OSPFd sample configuration file
+!
+!
+hostname xxxxx
+password xxxxx
+
+router ospf
+ router-id 10.0.0.1
+ network 10.0.0.1/24 area 1
+ neighbor 10.0.0.2
+ network 10.0.1.2/24 area 1
+ neighbor 10.0.1.1
+ ospf opaque-lsa <============ add this statement!
+
+```
+
+## 4. Usage
+
+In the following we describe how you can use the sample application to originate opaque LSAs. The sample application first registers with the OSPF daemon the opaque type it wants to inject and then waits until the OSPF daemon is ready to accept opaque LSAs of that type. Then the client application originates an opaque LSA, waits 10 seconds and then updates the opaque LSA with new opaque data. After another 20 seconds, the client application deletes the opaque LSA from the LSDB. If the clients terminates unexpectedly, the OSPF API module will remove all the opaque LSAs that the application registered. Since the opaque LSAs are flooded to other routers, we will see the opaque LSAs in all routers according to the flooding scope of the opaque LSA.
+
+We have a very simple demo setup, just two routers connected with an ATM point-to-point link. Start the modified OSPF daemons on two adjacent routers. First run on msr2:
+
+```
+ > msr2:/home/keller/ospfapi/zebra/ospfd# ./ospfd -f /usr/local/etc/ospfd.conf
+```
+
+And on the neighboring router msr3:
+
+
+```
+ > msr3:/home/keller/ospfapi/zebra/ospfd# ./ospfd -f /usr/local/etc/ospfd.conf
+```
+
+Now the two routers form adjacency and start exchanging their databases. Looking at the OSPF daemon of msr2 (or msr3), you see this:
+
+```
+ ospfd> show ip ospf database
+
+ OSPF Router with ID (10.0.0.1)
+
+ Router Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum Link count
+ 10.0.0.1 10.0.0.1 55 0x80000003 0xc62f 2
+ 10.0.0.2 10.0.0.2 55 0x80000003 0xe3e4 3
+
+ Net Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum
+ 10.0.0.2 10.0.0.2 60 0x80000001 0x5fcb
+
+```
+
+Now we start the sample main application that originates an opaque LSA.
+
+
+```
+ > cd ospfapi/apiclient
+ > ./main msr2 10 250 20 0.0.0.0 0.0.0.1
+
+```
+
+This originates an opaque LSA of type 10 (area local), with opaque type 250 (experimental), opaque id of 20 (chosen arbitrarily), interface address 0.0.0.0 (which is used only for opaque LSAs type 9), and area 0.0.0.1
+
+Again looking at the OSPF database you see:
+
+```
+ ospfd> show ip ospf database
+
+ OSPF Router with ID (10.0.0.1)
+
+ Router Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum Link count
+ 10.0.0.1 10.0.0.1 437 0x80000003 0xc62f 2
+ 10.0.0.2 10.0.0.2 437 0x80000003 0xe3e4 3
+
+ Net Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum
+ 10.0.0.2 10.0.0.2 442 0x80000001 0x5fcb
+
+ Area-Local Opaque-LSA (Area 0.0.0.1)
+
+ Opaque-Type/Id ADV Router Age Seq# CkSum
+ 250.0.0.20 10.0.0.1 0 0x80000001 0x58a6 <=== opaque LSA
+
+```
+
+You can take a closer look at this opaque LSA:
+
+```
+ ospfd> show ip ospf database opaque-area
+
+ OSPF Router with ID (10.0.0.1)
+
+
+ Area-Local Opaque-LSA (Area 0.0.0.1)
+
+ LS age: 4
+ Options: 66
+ LS Type: Area-Local Opaque-LSA
+ Link State ID: 250.0.0.20 (Area-Local Opaque-Type/ID)
+ Advertising Router: 10.0.0.1
+ LS Seq Number: 80000001
+ Checksum: 0x58a6
+ Length: 24
+ Opaque-Type 250 (Private/Experimental)
+ Opaque-ID 0x14
+ Opaque-Info: 4 octets of data
+ Added using OSPF API: 4 octets of opaque data
+ Opaque data: 1 0 0 0 <==== counter is 1
+
+```
+
+Note that the main application updates the opaque LSA after 10 seconds, then it looks as follows:
+
+```
+ ospfd> show ip ospf database opaque-area
+
+ OSPF Router with ID (10.0.0.1)
+
+
+ Area-Local Opaque-LSA (Area 0.0.0.1)
+
+ LS age: 1
+ Options: 66
+ LS Type: Area-Local Opaque-LSA
+ Link State ID: 250.0.0.20 (Area-Local Opaque-Type/ID)
+ Advertising Router: 10.0.0.1
+ LS Seq Number: 80000002
+ Checksum: 0x59a3
+ Length: 24
+ Opaque-Type 250 (Private/Experimental)
+ Opaque-ID 0x14
+ Opaque-Info: 4 octets of data
+ Added using OSPF API: 4 octets of opaque data
+ Opaque data: 2 0 0 0 <==== counter is now 2
+
+```
+
+Note that the payload of the opaque LSA has changed as you can see above.
+
+Then, again after another 20 seconds, the opaque LSA is flushed from the LSDB.
+
+#### Important note:
+
+In order to originate an opaque LSA, there must be at least one active opaque-capable neighbor. Thus, you cannot originate opaque LSAs of no neighbors are present. If you try to originate even so no neighbor is ready, you will receive a not ready error message. The reason for this restriction is that it might be possible that some routers have an identical opaque LSA from a previous origination in their LSDB that unfortunately could not be flushed due to a crash, and now if the router comes up again and starts originating a new opaque LSA, the new opaque LSA is considered older since it has a lower sequence number and is ignored by other routers (that consider the stalled opaque LSA as more recent). However, if the originating router first synchronizes the database before originating opaque LSAs, it will detect the older opaque LSA and can flush it first.
+
+
+## 5.  Protocol and Message Formats
+
+If you are developing your own client application and you don't want to make use of the client library (due to the GNU license restriction or whatever reason), you can implement your own client-side message handling. The OSPF API uses two connections between the client and the OSPF API server: One connection is used for a synchronous request /reply protocol and another connection is used for asynchronous notifications (e.g., LSA update, neighbor status change).
+
+Each message begins with the following header:
+
+![image](ospf_api_msghdr.png)
+
+The message type field can take one of the following values:
+
+Messages to OSPF deamon | Value
+----------------------- | -----
+MSG_REGISTER_OPAQUETYPE | 1
+MSG_UNREGISTER_OPAQUETYPE | 2
+MSG_REGISTER_EVENT | 3
+MSG_SYNC_LSDB | 4
+MSG_ORIGINATE_REQUEST | 5
+MSG_DELETE_REQUEST | 6
+
+Messages from OSPF deamon | Value
+------------------------- | -----
+MSG_REPLY | 10
+MSG_READY_NOTIFY | 11
+MSG_LSA_UPDATE_NOTIFY | 12
+MSG_LSA_DELETE_NOTIFY | 13
+MSG_NEW_IF | 14
+MSG_DEL_IF | 15
+MSG_ISM_CHANGE | 16
+MSG_NSM_CHANGE | 17
+
+The synchronous requests and replies have the following message formats:
+
+![image](ospf_api_msgs1.png)
+
+The origin field allows to select according to the following types of origins:
+
+Origin | Value
+------ | -----
+NON_SELF_ORIGINATED | 0
+SELF_ORIGINATED | 1
+ANY_ORIGIN | 2
+
+The reply message has on of the following error codes:
+
+Error code | Value
+---------- | -----
+API_OK | 0
+API_NOSUCHINTERFACE | -1
+API_NOSUCHAREA | -2
+API_NOSUCHLSA | -3
+API_ILLEGALSATYPE | -4
+API_ILLEGALOPAQUETYPE | -5
+API_OPAQUETYPEINUSE | -6
+API_NOMEMORY | -7
+API_ERROR | -99
+API_UNDEF | -100
+
+The asynchronous notifications have the following message formats:
+
+![image](ospf_api_msgs2.png)
+
+## 6.  Original Acknowledgments from Ralph Keller
+
+I would like to thank Masahiko Endo, the author of the opaque LSA extension module, for his great support. His wonderful ASCII graphs explaining the internal workings of this code, and his invaluable input proved to be crucial in designing a useful API for accessing the link state database of the OSPF daemon. Once, he even decided to take the plane from Tokyo to Zurich so that we could actually meet and have face-to-face discussions, which was a lot of fun. Clearly, without Masahiko no API would ever be completed. I also would like to thank Daniel Bauer who wrote an opaque LSA implementation too and was willing to test the OSPF API code in one of his projects.
diff --git a/doc/developer/bgpd.rst b/doc/developer/bgpd.rst
new file mode 100644
index 0000000000..053c0779de
--- /dev/null
+++ b/doc/developer/bgpd.rst
@@ -0,0 +1,8 @@
+BGPD
+=========================
+
+.. toctree::
+ :maxdepth: 2
+
+ next-hop-tracking
+
diff --git a/doc/developer/building.rst b/doc/developer/building.rst
new file mode 100644
index 0000000000..4715bca532
--- /dev/null
+++ b/doc/developer/building.rst
@@ -0,0 +1,22 @@
+Building FRR
+=========================
+
+.. toctree::
+ :maxdepth: 2
+
+ Building_FRR_on_LEDE-OpenWRT
+ Building_FRR_on_CentOS6
+ Building_FRR_on_CentOS7
+ Building_FRR_on_Debian8
+ Building_FRR_on_Debian9
+ Building_FRR_on_Fedora24
+ Building_FRR_on_FreeBSD10
+ Building_FRR_on_FreeBSD11
+ Building_FRR_on_FreeBSD9
+ Building_FRR_on_NetBSD6
+ Building_FRR_on_NetBSD7
+ Building_FRR_on_OmniOS
+ Building_FRR_on_OpenBSD6
+ Building_FRR_on_Ubuntu1204
+ Building_FRR_on_Ubuntu1404
+ Building_FRR_on_Ubuntu1604
diff --git a/doc/developer/cli.rst b/doc/developer/cli.rst
new file mode 100644
index 0000000000..cff3d21ef7
--- /dev/null
+++ b/doc/developer/cli.rst
@@ -0,0 +1,723 @@
+Command Line Interface
+======================
+
+FRR features a flexible modal command line interface. Often when adding new
+features or modifying existing code it is necessary to create or modify CLI
+commands. FRR has a powerful internal CLI system that does most of the heavy
+lifting for you.
+
+All definitions for the CLI system are exposed in ``lib/command.h``. In this
+header there are a set of macros used to define commands. These macros are
+collectively referred to as "DEFUNs", because of their syntax:
+
+::
+
+ DEFUN(command_name,
+ command_name_cmd,
+ "example command FOO...",
+ "Examples\n"
+ "CLI command\n"
+ "Argument\n")
+ {
+ // ...command handler...
+ }
+
+DEFUNs generally take four arguments which are expanded into the appropriate
+constructs for hooking into the CLI. In order these are:
+
+- **Function name** - the name of the handler function for the command
+- **Command name** - the identifier of the ``struct cmd_element`` for the
+ command. By convention this should be the function name with ``_cmd``
+ appended.
+- **Command definition** - an expression in FRR's CLI grammar that defines the
+ form of the command and its arguments, if any
+- **Doc string** - a newline-delimited string that documents each element in
+ the command definition
+
+In the above example, ``command_name`` is the function name,
+``command_name_cmd`` is the command name, ``"example..."`` is the definition
+and the last argument is the doc string. The block following the macro is the
+body of the handler function, details on which are presented later in this
+section.
+
+In order to make the command show up to the user it must be installed into the
+CLI graph. To do this, call:
+
+``install_element(NODE, &command_name_cmd);``
+
+This will install the command into the specified CLI node. Usually these calls
+are grouped together in a CLI initialization function for a set of commands,
+and the DEFUNs themselves are grouped into the same source file to avoid
+cluttering the codebase. The names of these files follow the form
+``*_vty.[ch]`` by convention. Please do not scatter individual CLI commands in
+the middle of source files; instead expose the necessary functions in a header
+and place the command definition in a ``*_vty.[ch]`` file.
+
+Definition Grammar
+------------------
+
+FRR uses its own grammar for defining CLI commands. The grammar draws from
+syntax commonly seen in \*nix manpages and should be fairly intuitive. The
+parser is implemented in Bison and the lexer in Flex. These may be found in
+``lib/command_lex.l`` and ``lib/command_parse.y``, respectively.
+
+ **ProTip**: if you define a new command and find that the parser is
+ throwing syntax or other errors, the parser is the last place you want
+ to look. Bison is very stable and if it detects a syntax error, 99% of
+ the time it will be a syntax error in your definition.
+
+The formal grammar in BNF is given below. This is the grammar implemented in
+the Bison parser. At runtime, the Bison parser reads all of the CLI strings and
+builds a combined directed graph that is used to match and interpret user
+input.
+
+Human-friendly explanations of how to use this grammar are given a bit later in
+this section alongside information on the :ref:`cli-data-structures` constructed
+by the parser.
+
+.. productionlist::
+ command: `cmd_token_seq`
+ : `cmd_token_seq` `placeholder_token` "..."
+ cmd_token_seq: *empty*
+ : `cmd_token_seq` `cmd_token`
+ cmd_token: `simple_token`
+ : `selector`
+ simple_token: `literal_token`
+ : `placeholder_token`
+ literal_token: WORD `varname_token`
+ varname_token: "$" WORD
+ placeholder_token: `placeholder_token_real` `varname_token`
+ placeholder_token_real: IPV4
+ : IPV4_PREFIX
+ : IPV6
+ : IPV6_PREFIX
+ : VARIABLE
+ : RANGE
+ : MAC
+ : MAC_PREFIX
+ selector: "<" `selector_seq_seq` ">" `varname_token`
+ : "{" `selector_seq_seq` "}" `varname_token`
+ : "[" `selector_seq_seq` "]" `varname_token`
+ selector_seq_seq: `selector_seq_seq` "|" `selector_token_seq`
+ : `selector_token_seq`
+ selector_token_seq: `selector_token_seq` `selector_token`
+ : `selector_token`
+ selector_token: `selector`
+ : `simple_token`
+
+Tokens
+~~~~~~
+The various capitalized tokens in the BNF above are in fact themselves
+placeholders, but not defined as such in the formal grammar; the grammar
+provides the structure, and the tokens are actually more like a type system for
+the strings you write in your CLI definitions. A CLI definition string is
+broken apart and each piece is assigned a type by the lexer based on a set of
+regular expressions. The parser uses the type information to verify the string
+and determine the structure of the CLI graph; additional metadata (such as the
+raw text of each token) is encoded into the graph as it is constructed by the
+parser, but this is merely a dumb copy job.
+
+Here is a brief summary of the various token types along with examples.
+
++-----------------+-----------------+-------------------------------------------------------------+
+| Token type | Syntax | Description |
++=================+=================+=============================================================+
+| ``WORD`` | ``show ip bgp`` | Matches itself. In the given example every token is a WORD. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``IPV4`` | ``A.B.C.D`` | Matches an IPv4 address. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``IPV6`` | ``X:X::X:X`` | Matches an IPv6 address. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``IPV4_PREFIX`` | ``A.B.C.D/M`` | Matches an IPv4 prefix in CIDR notation. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``IPV6_PREFIX`` | ``X:X::X:X/M`` | Matches an IPv6 prefix in CIDR notation. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``MAC`` | ``M:A:C`` | Matches a 48-bit mac address. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``MAC_PREFIX`` | ``M:A:C/M`` | Matches a 48-bit mac address with a mask. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``VARIABLE`` | ``FOOBAR`` | Matches anything. |
++-----------------+-----------------+-------------------------------------------------------------+
+| ``RANGE`` | ``(X-Y)`` | Matches numbers in the range X..Y inclusive. |
++-----------------+-----------------+-------------------------------------------------------------+
+
+When presented with user input, the parser will search over all defined
+commands in the current context to find a match. It is aware of the various
+types of user input and has a ranking system to help disambiguate commands. For
+instance, suppose the following commands are defined in the user's current
+context:
+
+::
+
+ example command FOO
+ example command (22-49)
+ example command A.B.C.D/X
+
+The following table demonstrates the matcher's choice for a selection of
+possible user input.
+
++-----------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+
+| Input | Matched command | Reason |
++=============================+===========================+==============================================================================================================+
+| example command eLi7eH4xx0r | example command FOO | ``eLi7eH4xx0r`` is not an integer or IPv4 prefix, |
+| | | but FOO is a variable and matches all input. |
++-----------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+
+| example command 42 | example command (22-49) | ``42`` is not an IPv4 prefix. It does match both |
+| | | ``(22-49)`` and ``FOO``, but RANGE tokens are more specific and have a higher priority than VARIABLE tokens. |
++-----------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+
+| example command 10.3.3.0/24 | example command A.B.C.D/X | The user entered an IPv4 prefix, which is best matched by the last command. |
++-----------------------------+---------------------------+--------------------------------------------------------------------------------------------------------------+
+
+Rules
+~~~~~
+
+There are also constructs which allow optional tokens, mutual exclusion, one-or-more selection and repetition.
+
+- ``<angle|brackets>`` -- Contain sequences of tokens separated by pipes and
+ provide mutual exclusion. User input matches at most one option.
+- ``[square brackets]`` -- Contains sequences of tokens that can be omitted.
+ ``[<a|b>]`` can be shortened to ``[a|b]``.
+- ``{curly|braces}`` -- similar to angle brackets, but instead of mutual
+ exclusion, curly braces indicate that one or more of the pipe-separated
+ sequences may be provided in any order.
+- ``VARIADICS...`` -- Any token which accepts input (anything except WORD)
+ which occurs as the last token of a line may be followed by an ellipsis,
+ which indicates that input matching the token may be repeated an unlimited
+ number of times.
+- ``$name`` -- Specify a variable name for the preceding token. See
+ "Variable Names" below.
+
+Some general notes:
+
+- Options are allowed at the beginning of the command. The developer is
+ entreated to use these extremely sparingly. They are most useful for
+ implementing the 'no' form of configuration commands. Please think carefully
+ before using them for anything else. There is usually a better solution, even
+ if it is just separating out the command definition into separate ones.
+- The developer should judiciously apply separation of concerns when defining
+ commands. CLI definitions for two unrelated or vaguely related commands or
+ configuration items should be defined in separate commands. Clarity is
+ preferred over LOC (within reason).
+- The maximum number of space-separated tokens that can be entered is
+ presently limited to 256. Please keep this limit in mind when
+ implementing new CLI.
+
+Variable Names
+--------------
+
+The parser tries to fill the "varname" field on each token. This can
+happen either manually or automatically. Manual specifications work by
+appending ``"$name"`` after the input specifier:
+
+::
+
+ foo bar$cmd WORD$name A.B.C.D$ip
+
+Note that you can also assign variable names to fixed input tokens, this
+can be useful if multiple commands share code. You can also use "$name"
+after a multiple-choice option:
+
+::
+
+ foo bar <A.B.C.D|X:X::X:X>$addr [optionA|optionB]$mode
+
+The variable name is in this case assigned to the last token in each of
+the branches.
+
+Automatic assignment of variable names works by applying the following
+rules:
+
+- manual names always have priority
+- a "[no]" at the beginning receives "no" as varname on the "no" token
+- VARIABLE tokens whose text is not "WORD" or "NAME" receive a cleaned
+ lowercase version of the token text as varname, e.g. "ROUTE-MAP"
+ becomes "route\_map".
+- other variable tokens (i.e. everything except "fixed") receive the
+ text of the preceding fixed token as varname, if one can be found.
+ E.g.: "ip route A.B.C.D/M INTERFACE" assigns "route" to the
+ "A.B.C.D/M" token.
+
+These rules should make it possible to avoid manual varname assignment
+in 90% of the cases.
+
+DEFPY
+-----
+
+``DEFPY(...)`` is an enhanced version of ``DEFUN()`` which is
+preprocessed by ``python/clidef.py``. The python script parses the
+command definition string, extracts variable names and types, and
+generates a C wrapper function that parses the variables and passes them
+on. This means that in the CLI function body, you will receive
+additional parameters with appropriate types.
+
+This is best explained by an example:
+
+::
+
+ DEFPY(func, func_cmd, "[no] foo bar A.B.C.D (0-99)$num", "...help...")
+
+ =>
+
+ func(self, vty, argc, argv, /* standard CLI arguments */
+
+ const char *no, /* unparsed "no" */
+ struct in_addr bar, /* parsed IP address */
+ const char *bar_str, /* unparsed IP address */
+ long num, /* parsed num */
+ const char *num_str) /* unparsed num */
+
+Note that as documented in the previous section, "bar" is automatically
+applied as variable name for "A.B.C.D". The python code then detects
+this is an IP address argument and generates code to parse it into a
+``struct in_addr``, passing it in ``bar``. The raw value is passed in
+``bar_str``. The range/number argument works in the same way with the
+explicitly given variable name.
+
+Type rules
+~~~~~~~~~~
+
++-----------------------------+--------------------------------+--------------------------+
+| Token(s) | Type | Value if omitted by user |
++=============================+================================+==========================+
+| ``A.B.C.D`` | ``struct in_addr`` | 0.0.0.0 |
++-----------------------------+--------------------------------+--------------------------+
+| ``X:X::X:X`` | ``struct in6_addr`` | \:: |
++-----------------------------+--------------------------------+--------------------------+
+| ``A.B.C.D + X:X::X:X`` | ``const union sockunion *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+| ``A.B.C.D/M`` | ``const struct prefix_ipv4 *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+| ``X:X::X:X/M`` | ``const struct prefix_ipv6 *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+| ``A.B.C.D/M + X:X::X:X/M`` | ``const struct prefix *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+| ``(0-9)`` | ``long`` | 0 |
++-----------------------------+--------------------------------+--------------------------+
+| ``VARIABLE`` | ``const char *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+| ``word`` | ``const char *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+| *all other* | ``const char *`` | NULL |
++-----------------------------+--------------------------------+--------------------------+
+
+Note the following details:
+
+- Not all parameters are pointers, some are passed as values.
+- When the type is not ``const char *``, there will be an extra
+ ``_str`` argument with type ``const char *``.
+- You can give a variable name not only to ``VARIABLE`` tokens but also
+ to ``word`` tokens (e.g. constant words). This is useful if some
+ parts of a command are optional. The type will be ``const char *``.
+- ``[no]`` will be passed as ``const char *no``.
+- Pointers will be NULL when the argument is optional and the user did
+ not use it.
+- If a parameter is not a pointer, but is optional and the user didn't
+ use it, the default value will be passed. Check the ``_str`` argument
+ if you need to determine whether the parameter was omitted.
+- If the definition contains multiple parameters with the same variable
+ name, they will be collapsed into a single function parameter. The
+ python code will detect if the types are compatible (i.e. IPv4 + IPv6
+ variantes) and choose a corresponding C type.
+- The standard DEFUN parameters (self, vty, argc, argv) are still
+ present and can be used. A DEFUN can simply be **edited into a DEFPY
+ without further changes and it will still work**; this allows easy
+ forward migration.
+- A file may contain both DEFUN and DEFPY statements.
+
+Getting a parameter dump
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The clidef.py script can be called to get a list of DEFUNs/DEFPYs with
+the parameter name/type list:
+
+::
+
+ lib/clippy python/clidef.py --all-defun --show lib/plist.c > /dev/null
+
+The generated code is printed to stdout, the info dump to stderr. The
+``--all-defun`` argument will make it process DEFUN blocks as well as
+DEFPYs, which is useful prior to converting some DEFUNs. **The dump does
+not list the ``_str`` arguments** to keep the output shorter.
+
+Note that the clidef.py script cannot be run with python directly, it
+needs to be run with *clippy* since the latter makes the CLI parser
+available.
+
+Include & Makefile requirements
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A source file that uses DEFPY needs to include the ``_clippy.c`` file
+**before all DEFPY statements**:
+
+::
+
+ /* GPL header */
+ #include ...
+
+ ...
+
+ #include "daemon/filename_clippy.c"
+
+ DEFPY(...)
+ DEFPY(...)
+
+ install_element(...)
+
+This dependency needs to be marked in Makefile.am: (there is no ordering
+requirement)
+
+::
+
+ include ../common.am
+
+ # ...
+
+ # if linked into a LTLIBRARY (.la/.so):
+ filename.lo: filename_clippy.c
+
+ # if linked into an executable or static library (.a):
+ filename.o: filename_clippy.c
+
+Doc Strings
+-----------
+
+Each token in a command definition should be documented with a brief doc string
+that informs a user of the meaning and/or purpose of the subsequent command
+tree. These strings are provided as the last parameter to DEFUN macros,
+concatenated together and separated by an escaped newline (\n). These are best
+explained by example.
+
+::
+
+ DEFUN (config_terminal,
+ config_terminal_cmd,
+ "configure terminal",
+ "Configuration from vty interface\n"
+ "Configuration terminal\n")
+
+The last parameter is split into two lines for readability. Two newline
+delimited doc strings are present, one for each token in the command.
+The second string documents the functionality of the 'terminal' command
+in the 'configure' tree.
+
+Note that the first string, for 'configure' does not contain
+documentation for 'terminal'. This is because the CLI is best envisioned
+as a tree, with tokens defining branches. An imaginary 'start' token is
+the root of every command in a CLI node. Each subsequent written token
+descends into a subtree, so the documentation for that token ideally
+summarizes all the functionality contained in the subtree.
+
+A consequence of this structure is that the developer must be careful to
+use the same doc strings when defining multiple commands that are part
+of the same tree. Commands which share prefixes must share the same doc
+strings for those prefixes. On startup the parser will generate warnings
+if it notices inconsistent doc strings. Behavior is undefined; the same
+token may show up twice in completions, with different doc strings, or
+it may show up once with a random doc string. Parser warnings should be
+heeded and fixed to avoid confusing users.
+
+The number of doc strings provided must be equal to the amount of tokens
+present in the command definition, read left to right, ignoring any
+special constructs.
+
+In the examples below, each arrowed token needs a doc string.
+
+::
+
+ "show ip bgp"
+ ^ ^ ^
+
+ "command <foo|bar> [example]"
+ ^ ^ ^ ^
+
+.. _cli-data-structures:
+
+Data Structures
+---------------
+
+On startup, the CLI parser sequentially parses each command string
+definition and constructs a directed graph with each token forming a
+node. This graph is the basis of the entire CLI system. It is used to
+match user input in order to generate command completions and match
+commands to functions.
+
+There is one graph per CLI node (not the same as a graph node in the CLI
+graph). The CLI node struct keeps a reference to its graph (see
+lib/command.h).
+
+While most of the graph maintains the form of a tree, special constructs
+outlined in the Rules section introduce some quirks. <>, [] and {} form
+self-contained 'subgraphs'. Each subgraph is a tree except that all of
+the 'leaves' actually share a child node. This helps with minimizing
+graph size and debugging.
+
+As a working example, here is the graph of the following command: ::
+
+ show [ip] bgp neighbors [<A.B.C.D|X:X::X:X|WORD>] [json]
+
+.. figure:: ../figures/cligraph.svg
+ :align: center
+
+ Graph of example CLI command
+
+
+``FORK`` and ``JOIN`` nodes are plumbing nodes that don't correspond to user
+input. They're necessary in order to deduplicate these constructs where
+applicable.
+
+Options follow the same form, except that there is an edge from the ``FORK``
+node to the ``JOIN`` node. Since all of the subgraphs in the example command
+are optional, all of them have this edge.
+
+Keywords follow the same form, except that there is an edge from ``JOIN`` to
+``FORK``. Because of this the CLI graph cannot be called acyclic. There is
+special logic in the input matching code that keeps a stack of paths already
+taken through the node in order to disallow following the same path more than
+once.
+
+Variadics are a bit special; they have an edge back to themselves, which allows
+repeating the same input indefinitely.
+
+The leaves of the graph are nodes that have no out edges. These nodes are
+special; their data section does not contain a token, as most nodes do, or
+NULL, as in ``FORK``/``JOIN`` nodes, but instead has a pointer to a
+cmd\_element. All paths through the graph that terminate on a leaf are
+guaranteed to be defined by that command. When a user enters a complete
+command, the command matcher tokenizes the input and executes a DFS on the CLI
+graph. If it is simultaneously able to exhaust all input (one input token per
+graph node), and then find exactly one leaf connected to the last node it
+reaches, then the input has matched the corresponding command and the command
+is executed. If it finds more than one node, then the command is ambiguous
+(more on this in deduplication). If it cannot exhaust all input, the command is
+unknown. If it exhausts all input but does not find an edge node, the command
+is incomplete.
+
+The parser uses an incremental strategy to build the CLI graph for a node. Each
+command is parsed into its own graph, and then this graph is merged into the
+overall graph. During this merge step, the parser makes a best-effort attempt
+to remove duplicate nodes. If it finds a node in the overall graph that is
+equal to a node in the corresponding position in the command graph, it will
+intelligently merge the properties from the node in the command graph into the
+already-existing node. Subgraphs are also checked for isomorphism and merged
+where possible. The definition of whether two nodes are 'equal' is based on the
+equality of some set of token properties; read the parser source for the most
+up-to-date definition of equality.
+
+When the parser is unable to deduplicate some complicated constructs, this can
+result in two identical paths through separate parts of the graph. If this
+occurs and the user enters input that matches these paths, they will receive an
+'ambiguous command' error and will be unable to execute the command. Most of
+the time the parser can detect and warn about duplicate commands, but it will
+not always be able to do this. Hence care should be taken before defining a
+new command to ensure it is not defined elsewhere.
+
+Command handlers
+----------------
+
+The block that follows a CLI definition is executed when a user enters
+input that matches the definition. Its function signature looks like
+this:
+
+::
+
+ int (*func) (const struct cmd_element *, struct vty *, int, struct cmd_token *[]);
+
+The first argument is the command definition struct. The last argument
+is an ordered array of tokens that correspond to the path taken through
+the graph, and the argument just prior to that is the length of the
+array.
+
+The arrangement of the token array has changed from the prior
+incarnation of the CLI system. In the old system, missing arguments were
+padded with NULLs so that the same parts of a command would show up at
+the same indices regardless of what was entered. The new system does not
+perform such padding and therefore it is generally *incorrect* to assume
+consistent indices in this array. As a simple example:
+
+Command definition:
+
+::
+
+ command [foo] <bar|baz>
+
+User enters:
+
+::
+
+ command foo bar
+
+Array:
+
+::
+
+ [0] -> command
+ [1] -> foo
+ [2] -> bar
+
+User enters:
+
+::
+
+ command baz
+
+Array:
+
+::
+
+ [0] -> command
+ [1] -> baz
+
+Command abbreviation & matching priority
+----------------------------------------
+
+As in the prior implementation, it is possible for users to elide parts
+of tokens when the CLI matcher does not need them to make an unambiguous
+match. This is best explained by example.
+
+Command definitions:
+
+::
+
+ command dog cow
+ command dog crow
+
+User input:
+
+::
+
+ c d c -> ambiguous command
+ c d co -> match "command dog cow"
+
+In the new implementation, this functionality has improved. Where
+previously the parser would stop at the first ambiguous token, it will
+now look ahead and attempt to disambiguate based on tokens later on in
+the input string.
+
+Command definitions:
+
+::
+
+ show ip bgp A.B.C.D
+ show ipv6 bgp X:X::X:X
+
+User enters:
+
+::
+
+ s i b 4.3.2.1 -> match "show ip bgp A.B.C.D"
+ s i b ::e0 -> match "show ipv6 bgp X:X::X:X"
+
+Previously both of these commands would be ambiguous since 'i' does not
+explicitly select either 'ip' or 'ipv6'. However, since the user later
+provides a token that matches only one of the commands (an IPv4 or IPv6
+address) the parser is able to look ahead and select the appropriate
+command. This has some implications for parsing the argv\*[] that is
+passed to the command handler.
+
+Now consider a command definition such as:
+
+::
+
+ command <foo|VAR>
+
+'foo' only matches the string 'foo', but 'VAR' matches any input,
+including 'foo'. Who wins? In situations like this the matcher will
+always choose the 'better' match, so 'foo' will win.
+
+Consider also:
+
+::
+
+ show <ip|ipv6> foo
+
+User input:
+
+::
+
+ show ip foo
+
+'ip' partially matches 'ipv6' but exactly matches 'ip', so 'ip' will
+win.
+
+struct cmd\_token
+-----------------
+
+::
+
+ /* Command token struct. */
+ struct cmd_token
+ {
+ enum cmd_token_type type; // token type
+ u_char attr; // token attributes
+ bool allowrepeat; // matcher allowed to match token repetitively?
+
+ char *text; // token text
+ char *desc; // token description
+ long long min, max; // for ranges
+ char *arg; // user input that matches this token
+ char *varname; // variable name
+ };
+
+This struct is used in the CLI graph to match input against. It is also
+used to pass user input to command handler functions, as it is
+frequently useful for handlers to have access to that information. When
+a command is matched, the sequence of cmd\_tokens that form the matching
+path are duplicated and placed in order into argv\*[]. Before this
+happens the ->arg field is set to point at the snippet of user input
+that matched it.
+
+For most nontrivial commands the handler function will need to determine
+which of the possible matching inputs was entered. Previously this was
+done by looking at the first few characters of input. This is now
+considered an anti-pattern and should be avoided. Instead, the ->type or
+->text fields for this logic. The ->type field can be used when the
+possible inputs differ in type. When the possible types are the same,
+use the ->text field. This field has the full text of the corresponding
+token in the definition string and using it makes for much more readable
+code. An example is helpful.
+
+Command definition:
+
+::
+
+ command <(1-10)|foo|BAR>
+
+In this example, the user may enter any one of: \* an integer between 1
+and 10 \* "foo" \* anything at all
+
+If the user enters "command f", then:
+
+::
+
+ argv[1]->type == WORD_TKN
+ argv[1]->arg == "f"
+ argv[1]->text == "foo"
+
+Range tokens have some special treatment; a token with ->type ==
+RANGE\_TKN will have the ->min and ->max fields set to the bounding
+values of the range.
+
+Permutations
+------------
+
+Finally, it is sometimes useful to check all the possible combinations
+of input that would match an arbitrary definition string. There is a
+tool in tools/ called 'permutations' that reads CLI definition strings
+on stdin and prints out all matching input permutations. It also dumps a
+text representation of the graph, which is more useful for debugging
+than anything else. It looks like this:
+
+::
+
+ $ ./permutations "show [ip] bgp [<view|vrf> WORD]"
+
+ show ip bgp view WORD
+ show ip bgp vrf WORD
+ show ip bgp
+ show bgp view WORD
+ show bgp vrf WORD
+ show bgp
+
+This functionality is also built into VTY/VTYSH; the 'list permutations'
+command will list all possible matching input permutations in the
+current CLI node.
diff --git a/doc/developer/conf.py b/doc/developer/conf.py
new file mode 100644
index 0000000000..e2293b2a6b
--- /dev/null
+++ b/doc/developer/conf.py
@@ -0,0 +1,341 @@
+# -*- coding: utf-8 -*-
+#
+# FRR documentation build configuration file, created by
+# sphinx-quickstart on Tue Jan 31 16:00:52 2017.
+#
+# This file is execfile()d with the current directory set to its
+# containing dir.
+#
+# Note that not all possible configuration values are present in this
+# autogenerated file.
+#
+# All configuration values have a default; values that are commented out
+# serve to show the default.
+
+import sys
+import os
+import re
+
+# If extensions (or modules to document with autodoc) are in another directory,
+# add these directories to sys.path here. If the directory is relative to the
+# documentation root, use os.path.abspath to make it absolute, like shown here.
+#sys.path.insert(0, os.path.abspath('.'))
+
+# -- General configuration ------------------------------------------------
+
+# If your documentation needs a minimal Sphinx version, state it here.
+needs_sphinx = '1.0'
+
+# prolog for various variable substitutions
+rst_prolog = ''
+
+# Add any Sphinx extension module names here, as strings. They can be
+# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
+# ones.
+extensions = ['sphinx.ext.todo']
+
+# Add any paths that contain templates here, relative to this directory.
+templates_path = ['_templates']
+
+# The suffix(es) of source filenames.
+# You can specify multiple suffix as a list of string:
+# source_suffix = ['.rst']
+source_suffix = '.rst'
+
+# The encoding of source files.
+#source_encoding = 'utf-8-sig'
+
+# The master toctree document.
+master_doc = 'index'
+
+# General information about the project.
+project = u'FRR'
+copyright = u'2017, FRR'
+author = u'FRR'
+
+# The version info for the project you're documenting, acts as replacement for
+# |version| and |release|, also used in various other places throughout the
+# built documents.
+
+# The short X.Y version.
+version = u'?.?'
+# The full version, including alpha/beta/rc tags.
+release = u'?.?-?'
+
+
+# -----------------------------------------------------------------------------
+# Extract values from codebase for substitution into docs.
+# -----------------------------------------------------------------------------
+
+# Various installation prefixes. Reasonable defaults are set where possible.
+# Values are overridden by logic below.
+replace_vars = {
+ 'AUTHORS': 'Kunihiro Ishiguro, et al.',
+ 'COPYRIGHT_YEAR': '1999-2005',
+ 'COPYRIGHT_STR': None,
+ 'PACKAGE_NAME': project.lower(),
+ 'PACKAGE_TARNAME': project.lower(),
+ 'PACKAGE_STRING': None,
+ 'PACKAGE_URL': 'https://frrouting.org/',
+ 'PACKAGE_VERSION': None,
+ 'INSTALL_PREFIX_ETC': None,
+ 'INSTALL_PREFIX_SBIN': None,
+ 'INSTALL_PREFIX_STATE': None,
+ 'INSTALL_PREFIX_MODULES': None,
+ 'INSTALL_USER': None,
+ 'INSTALL_GROUP': None,
+ 'INSTALL_VTY_GROUP': None,
+}
+
+# extract version information, installation location, other stuff we need to
+# use when building final documents
+val = re.compile('^S\["([^"]+)"\]="(.*)"$')
+with open('../../config.status', 'r') as cfgstatus:
+ for ln in cfgstatus.readlines():
+ m = val.match(ln)
+ if not m or m.group(1) not in replace_vars.keys(): continue
+ replace_vars[m.group(1)] = m.group(2)
+
+# manually fill out some of these we can't get from config.status
+replace_vars['COPYRIGHT_STR'] = "Copyright (c)"
+replace_vars['COPYRIGHT_STR'] += ' {0}'.format(replace_vars['COPYRIGHT_YEAR'])
+replace_vars['COPYRIGHT_STR'] += ' {0}'.format(replace_vars['AUTHORS'])
+release = replace_vars['PACKAGE_VERSION']
+version = release.split('-')[0]
+
+# add substitutions to prolog
+for key, value in replace_vars.items():
+ rst_prolog += '.. |{0}| replace:: {1}\n'.format(key, value)
+
+
+# The language for content autogenerated by Sphinx. Refer to documentation
+# for a list of supported languages.
+#
+# This is also used if you do content translation via gettext catalogs.
+# Usually you set "language" from the command line for these cases.
+language = None
+
+# There are two options for replacing |today|: either, you set today to some
+# non-false value, then it is used:
+#today = ''
+# Else, today_fmt is used as the format for a strftime call.
+#today_fmt = '%B %d, %Y'
+
+# List of patterns, relative to source directory, that match files and
+# directories to ignore when looking for source files.
+exclude_patterns = ['_build']
+
+# The reST default role (used for this markup: `text`) to use for all
+# documents.
+#default_role = None
+
+# If true, '()' will be appended to :func: etc. cross-reference text.
+#add_function_parentheses = True
+
+# If true, the current module name will be prepended to all description
+# unit titles (such as .. function::).
+#add_module_names = True
+
+# If true, sectionauthor and moduleauthor directives will be shown in the
+# output. They are ignored by default.
+#show_authors = False
+
+# The name of the Pygments (syntax highlighting) style to use.
+pygments_style = 'sphinx'
+
+# A list of ignored prefixes for module index sorting.
+#modindex_common_prefix = []
+
+# If true, keep warnings as "system message" paragraphs in the built documents.
+#keep_warnings = False
+
+# If true, `todo` and `todoList` produce output, else they produce nothing.
+todo_include_todos = True
+
+
+# -- Options for HTML output ----------------------------------------------
+
+# The theme to use for HTML and HTML Help pages. See the documentation for
+# a list of builtin themes.
+html_theme = 'default'
+
+# Theme options are theme-specific and customize the look and feel of a theme
+# further. For a list of options available for each theme, see the
+# documentation.
+html_theme_options = {
+ 'sidebarbgcolor': '#374249'
+}
+
+# Add any paths that contain custom themes here, relative to this directory.
+#html_theme_path = []
+
+# The name for this set of Sphinx documents. If None, it defaults to
+# "<project> v<release> documentation".
+#html_title = None
+
+# A shorter title for the navigation bar. Default is the same as html_title.
+#html_short_title = None
+
+# The name of an image file (relative to this directory) to place at the top
+# of the sidebar.
+html_logo = '../figures/frr-icon.svg'
+
+# The name of an image file (within the static path) to use as favicon of the
+# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
+# pixels large.
+html_favicon = '../figures/frr-logo-icon.png'
+
+# Add any paths that contain custom static files (such as style sheets) here,
+# relative to this directory. They are copied after the builtin static files,
+# so a file named "default.css" will overwrite the builtin "default.css".
+#html_static_path = ['_static']
+
+# Add any extra paths that contain custom files (such as robots.txt or
+# .htaccess) here, relative to this directory. These files are copied
+# directly to the root of the documentation.
+#html_extra_path = []
+
+# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
+# using the given strftime format.
+#html_last_updated_fmt = '%b %d, %Y'
+
+# If true, SmartyPants will be used to convert quotes and dashes to
+# typographically correct entities.
+#html_use_smartypants = True
+
+# Custom sidebar templates, maps document names to template names.
+#html_sidebars = {}
+
+# Additional templates that should be rendered to pages, maps page names to
+# template names.
+#html_additional_pages = {}
+
+# If false, no module index is generated.
+#html_domain_indices = True
+
+# If false, no index is generated.
+#html_use_index = True
+
+# If true, the index is split into individual pages for each letter.
+#html_split_index = False
+
+# If true, links to the reST sources are added to the pages.
+#html_show_sourcelink = True
+
+# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
+#html_show_sphinx = True
+
+# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
+#html_show_copyright = True
+
+# If true, an OpenSearch description file will be output, and all pages will
+# contain a <link> tag referring to it. The value of this option must be the
+# base URL from which the finished HTML is served.
+#html_use_opensearch = ''
+
+# This is the file name suffix for HTML files (e.g. ".xhtml").
+#html_file_suffix = None
+
+# Language to be used for generating the HTML full-text search index.
+# Sphinx supports the following languages:
+# 'da', 'de', 'en', 'es', 'fi', 'fr', 'hu', 'it', 'ja'
+# 'nl', 'no', 'pt', 'ro', 'ru', 'sv', 'tr'
+#html_search_language = 'en'
+
+# A dictionary with options for the search language support, empty by default.
+# Now only 'ja' uses this config value
+#html_search_options = {'type': 'default'}
+
+# The name of a javascript file (relative to the configuration directory) that
+# implements a search results scorer. If empty, the default will be used.
+#html_search_scorer = 'scorer.js'
+
+# Output file base name for HTML help builder.
+htmlhelp_basename = 'FRRdoc'
+
+# -- Options for LaTeX output ---------------------------------------------
+
+latex_elements = {
+# The paper size ('letterpaper' or 'a4paper').
+#'papersize': 'letterpaper',
+
+# The font size ('10pt', '11pt' or '12pt').
+#'pointsize': '10pt',
+
+# Additional stuff for the LaTeX preamble.
+#'preamble': '',
+
+# Latex figure (float) alignment
+#'figure_align': 'htbp',
+}
+
+# Grouping the document tree into LaTeX files. List of tuples
+# (source start file, target name, title,
+# author, documentclass [howto, manual, or own class]).
+latex_documents = [
+ (master_doc, 'FRR.tex', u"FRR Developer's Manual",
+ u'FRR', 'manual'),
+]
+
+# The name of an image file (relative to this directory) to place at the top of
+# the title page.
+latex_logo = '../figures/frr-logo-medium.png'
+
+# For "manual" documents, if this is true, then toplevel headings are parts,
+# not chapters.
+#latex_use_parts = False
+
+# If true, show page references after internal links.
+#latex_show_pagerefs = False
+
+# If true, show URL addresses after external links.
+#latex_show_urls = False
+
+# Documents to append as an appendix to all manuals.
+#latex_appendices = []
+
+# If false, no module index is generated.
+#latex_domain_indices = True
+
+
+# -- Options for manual page output ---------------------------------------
+
+# One entry per manual page. List of tuples
+# (source start file, name, description, authors, manual section).
+man_pages = [
+ (master_doc, 'frr', u"FRR Developer's Manual",
+ [author], 1)
+]
+
+# If true, show URL addresses after external links.
+#man_show_urls = False
+
+
+# -- Options for Texinfo output -------------------------------------------
+
+# Grouping the document tree into Texinfo files. List of tuples
+# (source start file, target name, title, author,
+# dir menu entry, description, category)
+texinfo_documents = [
+ (master_doc, 'frr', u"FRR Developer's Manual",
+ author, 'FRR', 'One line description of project.',
+ 'Miscellaneous'),
+]
+
+# Documents to append as an appendix to all manuals.
+#texinfo_appendices = []
+
+# If false, no module index is generated.
+#texinfo_domain_indices = True
+
+# How to display URL addresses: 'footnote', 'no', or 'inline'.
+#texinfo_show_urls = 'footnote'
+
+# If true, do not generate a @detailmenu in the "Top" node's menu.
+#texinfo_no_detailmenu = False
+
+# custom extensions here
+def setup(app):
+ # object type for FRR CLI commands, can be extended to document parent CLI
+ # node later on
+ app.add_object_type('clicmd', 'clicmd')
diff --git a/doc/developer/dev-modules.md b/doc/developer/dev-modules.md
new file mode 100644
index 0000000000..87bc963188
--- /dev/null
+++ b/doc/developer/dev-modules.md
@@ -0,0 +1,119 @@
+# Module and Hook support (developer docs)
+
+## What it does
+
+It uses `dlopen()` to load DSOs at startup.
+
+
+## Limitations
+
+* can't load, unload, or reload during runtime. This just needs some work
+ and can probably be done in the future.
+* doesn't fix any of the "things need to be changed in the code in the library"
+ issues. Most prominently, you can't add a CLI node because CLI nodes are
+ listed in the library...
+* if your module crashes, the daemon crashes. Should be obvious.
+* **does not provide a stable API or ABI**. Your module must match a version
+ of FRR and you may have to update it frequently to match changes.
+* **does not create a license boundary**. Your module will need to link
+ libzebra and include header files from the daemons, meaning it will be
+ GPL-encumbered.
+
+
+## Installation
+
+Look for `moduledir` in `configure.ac`, default is normally
+`/usr/lib64/frr/modules` but depends on `--libdir` / `--prefix`.
+
+The daemon's name is prepended when looking for a module, e.g. "snmp" tries
+to find "zebra_snmp" first when used in zebra. This is just to make it nicer
+for the user, with the snmp module having the same name everywhere.
+
+Modules can be packaged separately from FRR. The SNMP and FPM modules are
+good candidates for this because they have dependencies (net-snmp / protobuf)
+that are not FRR dependencies. However, any distro packages should have an
+"exact-match" dependency onto the FRR package. Using a module from a
+different FRR version will probably blow up nicely.
+
+For snapcraft (and during development), modules can be loaded with full path
+(e.g. -M `$SNAP/lib/frr/modules/zebra_snmp.so`). Note that libtool puts output
+files in the .libs directory, so during development you have to use
+`./zebra -M .libs/zebra_snmp.so`.
+
+
+## Creating a module
+
+... best to look at the existing SNMP or FPM modules.
+
+Basic boilerplate:
+
+```
+#include "hook.h"
+#include "module.h"
+
+static int
+module_init (void)
+{
+ hook_register(frr_late_init, module_late_init);
+ return 0;
+}
+
+FRR_MODULE_SETUP(
+ .name = "my module",
+ .version = "0.0",
+ .description = "my module",
+ .init = module_init,
+)
+```
+
+The `frr_late_init` hook will be called after the daemon has finished its
+other startup and is about to enter the main event loop; this is the best
+place for most initialisation.
+
+
+## Compiler & Linker magic
+
+There's a `THIS_MODULE` (like in the Linux kernel), which uses `visibility`
+attributes to restrict it to the current module. If you get a linker error
+with `_frrmod_this_module`, there is some linker SNAFU. This shouldn't be
+possible, though one way to get it would be to not include libzebra (which
+provides a fallback definition for the symbol).
+
+libzebra and the daemons each have their own `THIS_MODULE`, as do all loadable
+modules. In any other libraries (e.g. `libfrrsnmp`), `THIS_MODULE` will use
+the definition in libzebra; same applies if the main executable doesn't use
+`FRR_DAEMON_INFO` (e.g. all testcases).
+
+The deciding factor here is "what dynamic linker unit are you using the symbol
+from." If you're in a library function and want to know who called you, you
+can't use `THIS_MODULE` (because that'll just tell you you're in the library).
+Put a macro around your function that adds `THIS_MODULE` in the *caller's
+code calling your function*.
+
+The idea is to use this in the future for module unloading. Hooks already
+remember which module they were installed by, as groundwork for a function
+that removes all of a module's installed hooks.
+
+There's also the `frr_module` symbol in modules, pretty much a standard entry
+point for loadable modules.
+
+
+## Hooks
+
+Hooks are just points in the code where you can register your callback to
+be called. The parameter list is specific to the hook point. Since there is
+no stable API, the hook code has some extra type safety checks making sure
+you get a compiler warning when the hook parameter list doesn't match your
+callback. Don't ignore these warnings.
+
+
+## Relation to MTYPE macros
+
+The MTYPE macros, while primarily designed to decouple MTYPEs from the library
+and beautify the code, also work very nicely with loadable modules -- both
+constructors and destructors are executed when loading/unloading modules.
+
+This means there is absolutely no change required to MTYPEs, you can just use
+them in a module and they will even clean up themselves when we implement
+module unloading and an unload happens. In fact, it's impossible to create
+a bug where unloading fails to de-register a MTYPE.
diff --git a/doc/developer/draft-zebra-00.ms b/doc/developer/draft-zebra-00.ms
new file mode 100644
index 0000000000..25994727a0
--- /dev/null
+++ b/doc/developer/draft-zebra-00.ms
@@ -0,0 +1,209 @@
+.pl 10.0i
+.po 0
+.ll 7.2i
+.lt 7.2i
+.nr LL 7.2i
+.nr LT 7.2i
+.ds LF Ishiguro
+.ds RF FORMFEED[Page %]
+.ds CF
+.ds LH RFC DRAFT
+.ds RH March 1998
+.ds CH
+.hy 0
+.ad l
+Network Working Group K. Ishiguro
+Request for Comments: DRAFT Digital Magic Labs, Inc.
+ March 1998
+.sp 2
+.ce
+Zebra Protocol Draft
+.sp 2
+.fi
+.ne 4
+Status of this Memo
+.sp
+.in 3
+This draft is very eary beta version.
+.sp
+.in 0
+.ne 4
+Introduction
+.sp
+.in 3
+The zebra protocol is a communication protocol between kernel
+routing table manager and routing protocol daemon. It is built over
+TCP/IP protocol suite.
+.sp
+.in 0
+.ne 4
+Request message formats
+.sp
+.in 3
+zebra is TCP-based protocol.
+.sp
+Below is request packet format.
+.sp
+.in 0
+.DS
+0 1 2 3
+0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Length (2) | Command (1) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+.DE
+.sp
+.in 3
+Length is total packet length.
+.sp
+Here is summary of command list.
+.sp
+.in 0
+.DS
+1 - ZEBRA_IPV4_ROUTE_ADD
+2 - ZEBRA_IPV4_ROUTE_DELETE
+3 - ZEBRA_IPV6_ROUTE_ADD
+4 - ZEBRA_IPV6_ROUTE_DELETE
+5 - ZEBRA_GET_ONE_INTERFACE
+6 - ZEBRA_GET_ALL_INTERFACE
+7 - ZEBRA_GET_HOSTINFO
+.DE
+.sp
+.in 0
+.ne 4
+IPv4 reply message formats
+.sp
+.in 0
+.DS
+0 1 2 3
+0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
++-+-+-+-+-+-+-+-+
+| Type (1) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Gateway (4) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+.DE
+.sp
+.in 3
+Type field specify route's origin type.
+.sp
+.in 0
+.DS
+1 - ZEBRA_ROUTE_RESERVE
+2 - ZEBRA_ROUTE_CONNECT
+3 - ZEBRA_ROUTE_STATIC
+4 - ZEBRA_ROUTE_RIP
+5 - ZEBRA_ROUTE_RIPNG
+6 - ZEBRA_ROUTE_BGP
+7 - ZEBRA_ROUTE_RADIX
+.DE
+.sp
+.in 3
+After above message there can be variale length IPv4 prefix data.
+Each IPv4 prefix is encoded as a two tuple of the form <masklength,
+prefix>
+.sp
+.in 0
+.DS
++----------------------+
+|Subnet mask (1 octet) |
++----------------------+
+|IPv4 prefix (variable)|
++----------------------+
+.DE
+.sp
+.in 0
+.ne 4
+IPv6 reply message formats
+.sp
+.in 0
+.DS
+0 1 2 3
+0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
++-+-+-+-+-+-+-+-+
+| Type (1) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Gateway (16) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+.DE
+.sp
+.in 3
+Type field specify route's origin type.
+.sp
+.in 0
+.DS
+1 - ZEBRA_ROUTE_RESERVE
+2 - ZEBRA_ROUTE_CONNECT
+3 - ZEBRA_ROUTE_STATIC
+4 - ZEBRA_ROUTE_RIP
+5 - ZEBRA_ROUTE_RIPNG
+6 - ZEBRA_ROUTE_BGP
+7 - ZEBRA_ROUTE_RADIX
+.DE
+.sp
+.in 0
+.DS
++----------------------+
+| ifindex (4 octet) |
++----------------------+
+| prefixlen (1 octet)|
++----------------------+
+|IPv6 prefix (variable)|
++----------------------+
+.DE
+.sp
+.in 3
+I am not sure but it seems some operation systems IPv6
+implementation may need interface index when add and delete
+linklocal routes.
+.sp
+I have added ifindex field to specify IPv6 routes interface
+index. If this index is value zero, it will ignored.
+.sp
+.in 0
+.ne 4
+Interface information message format.
+.sp
+.in 0
+.DS
+0 1 2 3
+0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Interface name (20) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Index (1) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Inteface flag (4) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Inteface metric (4) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Inteface MTU (4) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+| Inteface Address count (4) |
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+.DE
+.sp
+.in 3
+Address message format.
+.sp
+.in 0
+.ne 4
+Host inforamtion message format.
+.sp
+.in 0
+.DS
+0 1 2 3
+0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+|IPv4 forwarding|IPv6 forwarding|
++-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+.DE
+.sp
+.in 3
+Host information contain IPv4/IPv6 forwarding information.
diff --git a/doc/developer/hooks.rst b/doc/developer/hooks.rst
new file mode 100644
index 0000000000..0afa297aa7
--- /dev/null
+++ b/doc/developer/hooks.rst
@@ -0,0 +1,171 @@
+.. highlight:: c
+
+Hooks
+=====
+
+Libfrr provides type-safe subscribable hook points where other pieces of
+code can add one or more callback functions. "type-safe" in this case
+applies to the function pointers used for subscriptions. The
+implementations checks (at compile-time) wheter a callback to be added has
+the appropriate function signature (parameters) for the hook.
+
+Example:
+
+.. code-block:: c
+ :caption: mydaemon.h
+
+ #include "hook.h"
+ DECLARE_HOOK(some_update_event, (struct eventinfo *info), (info))
+
+.. code-block:: c
+ :caption: mydaemon.c
+
+ #include "mydaemon.h"
+ DEFINE_HOOK(some_update_event, (struct eventinfo *info), (info))
+ ...
+ hook_call(some_update_event, info);
+
+.. code-block:: c
+ :caption: mymodule.c
+
+ #include "mydaemon.h"
+ static int event_handler(struct eventinfo *info);
+ ...
+ hook_register(some_update_event, event_handler);
+
+Do not use parameter names starting with "hook", these can collide with
+names used by the hook code itself.
+
+
+Return values
+-------------
+
+Callbacks to be placed on hooks always return "int" for now; hook_call will
+sum up the return values from each called function. (The default is 0 if no
+callbacks are registered.)
+
+There are no pre-defined semantics for the value, in most cases it is
+ignored. For success/failure indication, 0 should be success, and
+handlers should make sure to only return 0 or 1 (not -1 or other values).
+
+There is no built-in way to abort executing a chain after a failure of one
+of the callbacks. If this is needed, the hook can use an extra
+``bool *aborted`` argument.
+
+
+Priorities
+----------
+
+Hooks support a "priority" value for ordering registered calls
+relative to each other. The priority is a signed integer where lower
+values are called earlier. There are also "Koohs", which is hooks with
+reverse priority ordering (for cleanup/deinit hooks, so you can use the
+same priority value).
+
+Recommended priority value ranges are:
+
+======================== ===================================================
+Range Usage
+------------------------ ---------------------------------------------------
+ -999 ... 0 ... 999 main executable / daemon, or library
+
+-1999 ... -1000 modules registering calls that should run before
+ the daemon's bits
+
+1000 ... 1999 modules' calls that should run after daemon's
+ (includes default value: 1000)
+======================== ===================================================
+
+Note: the default value is 1000, based on the following 2 expectations:
+
+- most hook_register() usage will be in loadable modules
+- usage of hook_register() in the daemon itself may need relative ordering
+ to itself, making an explicit value the expected case
+
+The priority value is passed as extra argument on hook_register_prio() /
+hook_register_arg_prio(). Whether a hook runs in reverse is determined
+solely by the code defining / calling the hook. (DECLARE_KOOH is actually
+the same thing as DECLARE_HOOK, it's just there to make it obvious.)
+
+
+Definition
+----------
+
+.. c:macro:: DECLARE_HOOK(name, arglist, passlist)
+.. c:macro:: DECLARE_KOOH(name, arglist, passlist)
+
+ :param name: Name of the hook to be defined
+ :param arglist: Function definition style parameter list in braces.
+ :param passlist: List of the same parameters without their types.
+
+ Note: the second and third macro args must be the hook function's
+ parameter list, with the same names for each parameter. The second
+ macro arg is with types (used for defining things), the third arg is
+ just the names (used for passing along parameters).
+
+ This macro must be placed in a header file; this header file must be
+ included to register a callback on the hook.
+
+ Examples:
+
+ .. code-block:: c
+
+ DECLARE_HOOK(foo, (), ())
+ DECLARE_HOOK(bar, (int arg), (arg))
+ DECLARE_HOOK(baz, (const void *x, in_addr_t y), (x, y))
+
+.. c:macro:: DEFINE_HOOK(name, arglist, passlist)
+
+ Implements an hook. Each ``DECLARE_HOOK`` must have be accompanied by
+ exactly one ``DEFINE_HOOK``, which needs to be placed in a source file.
+ **The hook can only be called from this source file.** This is intentional
+ to avoid overloading and/or misusing hooks for distinct purposes.
+
+ The compiled source file will include a global symbol with the name of the
+ hook prefixed by `_hook_`. Trying to register a callback for a hook that
+ doesn't exist will therefore result in a linker error, or a module
+ load-time error for dynamic modules.
+
+.. c:macro:: DEFINE_KOOH(name, arglist, passlist)
+
+ Same as ``DEFINE_HOOK``, but the sense of priorities / order of callbacks
+ is reversed. This should be used for cleanup hooks.
+
+.. c:function:: int hook_call(name, ...)
+
+ Calls the specified named hook. Parameters to the hook are passed right
+ after the hook name, e.g.:
+
+ .. code-block:: c
+
+ hook_call(foo);
+ hook_call(bar, 0);
+ hook_call(baz, NULL, INADDR_ANY);
+
+ Returns the sum of return values from all callbacks. The ``DEFINE_HOOK``
+ statement for the hook must be placed in the file before any ``hook_call``
+ use of the hook.
+
+
+Callback registration
+---------------------
+
+.. c:function:: void hook_register(name, int (*callback)(...))
+.. c:function:: void hook_register_prio(name, int priority, int (*callback)(...))
+.. c:function:: void hook_register_arg(name, int (*callback)(void *arg, ...), void *arg)
+.. c:function:: void hook_register_arg_prio(name, int priority, int (*callback)(void *arg, ...), void *arg)
+
+ Register a callback with an hook. If the caller needs to pass an extra
+ argument to the callback, the _arg variant can be used and the extra
+ parameter will be passed as first argument to the callback. There is no
+ typechecking for this argument.
+
+ The priority value is used as described above. The variants without a
+ priority parameter use 1000 as priority value.
+
+.. c:function:: void hook_unregister(name, int (*callback)(...))
+.. c:function:: void hook_unregister_arg(name, int (*callback)(void *arg, ...), void *arg)
+
+ Removes a previously registered callback from a hook. Note that there
+ is no _prio variant of these calls. The priority value is only used during
+ registration.
diff --git a/doc/developer/index.rst b/doc/developer/index.rst
new file mode 100644
index 0000000000..acb05e2814
--- /dev/null
+++ b/doc/developer/index.rst
@@ -0,0 +1,13 @@
+Welcome to FRR's documentation!
+===============================
+
+.. toctree::
+ :maxdepth: 2
+
+ workflow
+ library
+ bgpd
+ building
+ ospf-api
+ ospf-sr
+
diff --git a/doc/developer/ldpd-basic-test-setup.md b/doc/developer/ldpd-basic-test-setup.md
new file mode 100644
index 0000000000..b25a2b6d4b
--- /dev/null
+++ b/doc/developer/ldpd-basic-test-setup.md
@@ -0,0 +1,681 @@
+## Topology
+
+The goal of this test is to verify that the all the basic functionality
+of ldpd is working as expected, be it running on Linux or OpenBSD. In
+addition to that, more advanced features are also tested, like LDP
+sessions over IPv6, MD5 authentication and pseudowire signaling.
+
+In the topology below there are 3 PE routers, 3 CE routers and one P
+router (not attached to any consumer site).
+
+All routers have IPv4 addresses and OSPF is used as the IGP. The
+three routers from the bottom of the picture, P, PE2 and PE3, are also
+configured for IPv6 (dual-stack) and static IPv6 routes are used to
+provide connectivity among them.
+
+The three CEs share the same VPLS membership. LDP is used to set up the
+LSPs among the PEs and to signal the pseudowires. MD5 authentication is
+used to protect all LDP sessions.
+
+```
+ CE1 172.16.1.1/24
+ +
+ |
+ +---+---+
+ | PE1 |
+ | IOS XE|
+ | |
+ +---+---+
+ |
+ | 10.0.1.0/24
+ |
+ +---+---+
+ | P |
+ +------+ IOS XR+------+
+ | | | |
+ | +-------+ |
+ 10.0.2.0/24 | | 10.0.3.0/24
+2001:db8:2::/64 | | 2001:db8:3::/64
+ | |
+ +---+---+ +---+---+
+ | PE2 | | PE3 |
+ |OpenBSD+-------------+ Linux |
+ | | | |
+ +---+---+ 10.0.4.0/24 +---+---+
+ | 2001:db8:4::/64 |
+ + +
+ 172.16.1.2/24 CE2 CE3 172.16.1.3/24
+```
+
+## Configuration
+
+#### Linux
+1 - Enable IPv4/v6 forwarding:
+```
+# sysctl -w net.ipv4.ip_forward=1
+# sysctl -w net.ipv6.conf.all.forwarding=1
+```
+
+2 - Enable MPLS forwarding:
+```
+# modprobe mpls-router
+# modprobe mpls-iptunnel
+# echo 100000 > /proc/sys/net/mpls/platform_labels
+# echo 1 > /proc/sys/net/mpls/conf/eth1/input
+# echo 1 > /proc/sys/net/mpls/conf/eth2/input
+```
+
+3 - Set up the interfaces:
+```
+# ip link add name lo1 type dummy
+# ip link set dev lo1 up
+# ip addr add 4.4.4.4/32 dev lo1
+# ip -6 addr add 4:4:4::4/128 dev lo1
+# ip link set dev eth1 up
+# ip addr add 10.0.4.4/24 dev eth1
+# ip -6 addr add 2001:db8:4::4/64 dev eth1
+# ip link set dev eth2 up
+# ip addr add 10.0.3.4/24 dev eth2
+# ip -6 addr add 2001:db8:3::4/64 dev eth2
+```
+
+4 - Set up the bridge and pseudowire interfaces:
+```
+# ip link add type bridge
+# ip link set dev bridge0 up
+# ip link set dev eth0 up
+# ip link set dev eth0 master bridge0
+# ip link add name mpw0 type dummy
+# ip link set dev mpw0 up
+# ip link set dev mpw0 master bridge0
+# ip link add name mpw1 type dummy
+# ip link set dev mpw1 up
+# ip link set dev mpw1 master bridge0
+```
+
+> NOTE: MPLS support in the Linux kernel is very recent and it still
+doesn't support pseudowire interfaces. We are using here dummy interfaces
+just to show how the VPLS configuration should look like in the future.
+
+5 - Add static IPv6 routes for the remote loopbacks:
+```
+# ip -6 route add 2:2:2::2/128 via 2001:db8:3::2
+# ip -6 route add 3:3:3::3/128 via 2001:db8:4::3
+```
+
+6 - Edit /etc/frr/ospfd.conf:
+```
+router ospf
+ network 4.4.4.4/32 area 0.0.0.0
+ network 10.0.3.4/24 area 0.0.0.0
+ network 10.0.4.4/24 area 0.0.0.0
+!
+```
+
+7 - Edit /etc/frr/ldpd.conf:
+```
+debug mpls ldp messages recv
+debug mpls ldp messages sent
+debug mpls ldp zebra
+!
+mpls ldp
+ router-id 4.4.4.4
+ dual-stack cisco-interop
+ neighbor 1.1.1.1 password opensourcerouting
+ neighbor 2.2.2.2 password opensourcerouting
+ neighbor 3.3.3.3 password opensourcerouting
+ !
+ address-family ipv4
+ discovery transport-address 4.4.4.4
+ label local advertise explicit-null
+ !
+ interface eth2
+ !
+ interface eth1
+ !
+ !
+ address-family ipv6
+ discovery transport-address 4:4:4::4
+ ttl-security disable
+ !
+ interface eth2
+ !
+ interface eth1
+ !
+ !
+!
+l2vpn ENG type vpls
+ bridge br0
+ member interface eth0
+ !
+ member pseudowire mpw0
+ neighbor lsr-id 1.1.1.1
+ pw-id 100
+ !
+ member pseudowire mpw1
+ neighbor lsr-id 3.3.3.3
+ neighbor address 3:3:3::3
+ pw-id 100
+ !
+!
+```
+
+> NOTE: We have to disable ttl-security under the ipv6 address-family
+in order to interoperate with the IOS-XR router. GTSM is mandatory for
+LDPv6 but the IOS-XR implementation is not RFC compliant in this regard.
+
+8 - Run zebra, ospfd and ldpd.
+
+#### OpenBSD
+1 - Enable IPv4/v6 forwarding:
+```
+# sysctl net.inet.ip.forwarding=1
+# sysctl net.inet6.ip6.forwarding=1
+```
+
+2 - Enable MPLS forwarding:
+```
+# ifconfig em2 10.0.2.3/24 mpls
+# ifconfig em3 10.0.4.3/24 mpls
+```
+
+3 - Set up the interfaces:
+```
+# ifconfig lo1 alias 3.3.3.3 netmask 255.255.255.255
+# ifconfig lo1 inet6 3:3:3::3/128
+# ifconfig em2 inet6 2001:db8:2::3/64
+# ifconfig em3 inet6 2001:db8:4::3/64
+```
+
+4 - Set up the bridge and pseudowire interfaces:
+```
+# ifconfig bridge0 create
+# ifconfig bridge0 up
+# ifconfig em1 up
+# ifconfig bridge0 add em1
+# ifconfig mpw0 create
+# ifconfig mpw0 up
+# ifconfig bridge0 add mpw0
+# ifconfig mpw1 create
+# ifconfig mpw1 up
+# ifconfig bridge0 add mpw1
+```
+
+5 - Add static IPv6 routes for the remote loopbacks:
+```
+# route -n add 4:4:4::4/128 2001:db8:4::4
+# route -n add 2:2:2::2/128 2001:db8:2::2
+```
+
+6 - Edit /etc/frr/ospfd.conf:
+```
+router ospf
+ network 10.0.2.3/24 area 0
+ network 10.0.4.3/24 area 0
+ network 3.3.3.3/32 area 0
+!
+```
+
+7 - Edit /etc/frr/ldpd.conf:
+```
+debug mpls ldp messages recv
+debug mpls ldp messages sent
+debug mpls ldp zebra
+!
+mpls ldp
+ router-id 3.3.3.3
+ dual-stack cisco-interop
+ neighbor 1.1.1.1 password opensourcerouting
+ neighbor 2.2.2.2 password opensourcerouting
+ neighbor 4.4.4.4 password opensourcerouting
+ !
+ address-family ipv4
+ discovery transport-address 3.3.3.3
+ label local advertise explicit-null
+ !
+ interface em3
+ !
+ interface em2
+ !
+ !
+ address-family ipv6
+ discovery transport-address 3:3:3::3
+ ttl-security disable
+ !
+ interface em3
+ !
+ interface em2
+ !
+ !
+!
+l2vpn ENG type vpls
+ bridge br0
+ member interface em1
+ !
+ member pseudowire mpw0
+ neighbor lsr-id 1.1.1.1
+ pw-id 100
+ !
+ member pseudowire mpw1
+ neighbor lsr-id 4.4.4.4
+ neighbor address 4:4:4::4
+ pw-id 100
+ !
+!
+```
+
+8 - Run zebra, ospfd and ldpd.
+
+#### Cisco routers
+CE1 (IOS):
+```
+interface FastEthernet0/0
+ ip address 172.16.1.1 255.255.255.0
+ !
+!
+```
+
+CE2 (IOS):
+```
+interface FastEthernet0/0
+ ip address 172.16.1.2 255.255.255.0
+ !
+!
+```
+
+CE3 (IOS):
+```
+interface FastEthernet0/0
+ ip address 172.16.1.3 255.255.255.0
+ !
+!
+```
+
+PE1 - IOS-XE (1):
+```
+mpls ldp neighbor 2.2.2.2 password opensourcerouting
+mpls ldp neighbor 3.3.3.3 password opensourcerouting
+mpls ldp neighbor 4.4.4.4 password opensourcerouting
+!
+l2vpn vfi context VFI
+ vpn id 1
+ member pseudowire2
+ member pseudowire1
+!
+bridge-domain 1
+ member GigabitEthernet1 service-instance 1
+ member vfi VFI
+!
+interface Loopback1
+ ip address 1.1.1.1 255.255.255.255
+!
+interface pseudowire1
+ encapsulation mpls
+ neighbor 3.3.3.3 100
+!
+interface pseudowire2
+ encapsulation mpls
+ neighbor 4.4.4.4 100
+!
+interface GigabitEthernet3
+ ip address 10.0.1.1 255.255.255.0
+ mpls ip
+!
+router ospf 1
+ network 0.0.0.0 255.255.255.255 area 0
+!
+```
+
+P - IOS-XR (2):
+```
+interface Loopback1
+ ipv4 address 2.2.2.2 255.255.255.255
+ ipv6 address 2:2:2::2/128
+!
+interface GigabitEthernet0/0/0/0
+ ipv4 address 10.0.1.2 255.255.255.0
+!
+interface GigabitEthernet0/0/0/1
+ ipv4 address 10.0.2.2 255.255.255.0
+ ipv6 address 2001:db8:2::2/64
+ ipv6 enable
+!
+interface GigabitEthernet0/0/0/2
+ ipv4 address 10.0.3.2 255.255.255.0
+ ipv6 address 2001:db8:3::2/64
+ ipv6 enable
+!
+router static
+ address-family ipv6 unicast
+ 3:3:3::3/128 2001:db8:2::3
+ 4:4:4::4/128 2001:db8:3::4
+ !
+!
+router ospf 1
+ router-id 2.2.2.2
+ address-family ipv4 unicast
+ area 0
+ interface Loopback1
+ !
+ interface GigabitEthernet0/0/0/0
+ !
+ interface GigabitEthernet0/0/0/1
+ !
+ interface GigabitEthernet0/0/0/2
+ !
+ !
+!
+mpls ldp
+ router-id 2.2.2.2
+ neighbor
+ 1.1.1.1:0 password clear opensourcerouting
+ 3.3.3.3:0 password clear opensourcerouting
+ 4.4.4.4:0 password clear opensourcerouting
+ !
+ address-family ipv4
+ !
+ address-family ipv6
+ discovery transport-address 2:2:2::2
+ !
+ interface GigabitEthernet0/0/0/0
+ address-family ipv4
+ !
+ !
+ interface GigabitEthernet0/0/0/1
+ address-family ipv4
+ !
+ address-family ipv6
+ !
+ !
+ interface GigabitEthernet0/0/0/2
+ address-family ipv4
+ !
+ address-family ipv6
+ !
+ !
+!
+```
+
+## Verification - Control Plane
+
+Using the CLI on the Linux box, the goal is to ensure that everything
+is working as expected.
+
+First, verify that all the required adjacencies and neighborships sessions
+were established:
+
+```
+linux# show mpls ldp discovery
+Local LDP Identifier: 4.4.4.4:0
+Discovery Sources:
+ Interfaces:
+ eth1: xmit/recv
+ LDP Id: 3.3.3.3:0, Transport address: 3.3.3.3
+ Hold time: 15 sec
+ LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3
+ Hold time: 15 sec
+ eth2: xmit/recv
+ LDP Id: 2.2.2.2:0, Transport address: 2.2.2.2
+ Hold time: 15 sec
+ LDP Id: 2.2.2.2:0, Transport address: 2:2:2::2
+ Hold time: 15 sec
+ Targeted Hellos:
+ 4.4.4.4 -> 1.1.1.1: xmit/recv
+ LDP Id: 1.1.1.1:0, Transport address: 1.1.1.1
+ Hold time: 45 sec
+ 4:4:4::4 -> 3:3:3::3: xmit/recv
+ LDP Id: 3.3.3.3:0, Transport address: 3:3:3::3
+ Hold time: 45 sec
+
+linux# show mpls ldp neighbor
+Peer LDP Identifier: 1.1.1.1:0
+ TCP connection: 4.4.4.4:40921 - 1.1.1.1:646
+ Session Holdtime: 180 sec
+ State: OPERATIONAL; Downstream-Unsolicited
+ Up time: 00:06:02
+ LDP Discovery Sources:
+ IPv4:
+ Targeted Hello: 1.1.1.1
+
+Peer LDP Identifier: 2.2.2.2:0
+ TCP connection: 4:4:4::4:52286 - 2:2:2::2:646
+ Session Holdtime: 180 sec
+ State: OPERATIONAL; Downstream-Unsolicited
+ Up time: 00:06:02
+ LDP Discovery Sources:
+ IPv4:
+ Interface: eth2
+ IPv6:
+ Interface: eth2
+
+Peer LDP Identifier: 3.3.3.3:0
+ TCP connection: 4:4:4::4:60575 - 3:3:3::3:646
+ Session Holdtime: 180 sec
+ State: OPERATIONAL; Downstream-Unsolicited
+ Up time: 00:05:57
+ LDP Discovery Sources:
+ IPv4:
+ Interface: eth1
+ IPv6:
+ Targeted Hello: 3:3:3::3
+ Interface: eth1
+```
+
+Note that the neighborships with the P and PE2 routers were established
+over IPv6, since this is the default behavior for dual-stack LSRs, as
+specified in RFC 7552. If desired, the **dual-stack transport-connection
+prefer ipv4** command can be used to establish these sessions over IPv4
+(the command should be applied an all routers).
+
+Now, verify that there's a remote label for each PE address:
+```
+linux# show mpls ldp binding
+1.1.1.1/32
+ Local binding: label: 20
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 imp-null
+ 2.2.2.2 24000
+ 3.3.3.3 20
+2.2.2.2/32
+ Local binding: label: 21
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 18
+ 2.2.2.2 imp-null
+ 3.3.3.3 21
+3.3.3.3/32
+ Local binding: label: 22
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 21
+ 2.2.2.2 24003
+ 3.3.3.3 imp-null
+4.4.4.4/32
+ Local binding: label: imp-null
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 22
+ 2.2.2.2 24001
+ 3.3.3.3 22
+10.0.1.0/24
+ Local binding: label: 23
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 imp-null
+ 2.2.2.2 imp-null
+ 3.3.3.3 23
+10.0.2.0/24
+ Local binding: label: 24
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 20
+ 2.2.2.2 imp-null
+ 3.3.3.3 imp-null
+10.0.3.0/24
+ Local binding: label: imp-null
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 19
+ 2.2.2.2 imp-null
+ 3.3.3.3 24
+10.0.4.0/24
+ Local binding: label: imp-null
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 1.1.1.1 23
+ 2.2.2.2 24002
+ 3.3.3.3 imp-null
+2:2:2::2/128
+ Local binding: label: 18
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 2.2.2.2 imp-null
+ 3.3.3.3 18
+3:3:3::3/128
+ Local binding: label: 19
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 2.2.2.2 24007
+4:4:4::4/128
+ Local binding: label: imp-null
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 2.2.2.2 24006
+ 3.3.3.3 19
+2001:db8:2::/64
+ Local binding: label: -
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 2.2.2.2 imp-null
+ 3.3.3.3 imp-null
+2001:db8:3::/64
+ Local binding: label: imp-null
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 2.2.2.2 imp-null
+2001:db8:4::/64
+ Local binding: label: imp-null
+ Remote bindings:
+ Peer Label
+ ----------------- ---------
+ 3.3.3.3 imp-null
+```
+
+Check if the pseudowires are up:
+```
+linux# show l2vpn atom vc
+Interface Peer ID VC ID Name Status
+--------- --------------- ---------- ---------------- ----------
+mpw1 3.3.3.3 100 ENG UP
+mpw0 1.1.1.1 100 ENG UP
+```
+
+Check the label bindings of the pseudowires:
+```
+linux# show l2vpn atom binding
+ Destination Address: 1.1.1.1, VC ID: 100
+ Local Label: 25
+ Cbit: 1, VC Type: Ethernet, GroupID: 0
+ MTU: 1500
+ Remote Label: 16
+ Cbit: 1, VC Type: Ethernet, GroupID: 0
+ MTU: 1500
+ Destination Address: 3.3.3.3, VC ID: 100
+ Local Label: 26
+ Cbit: 1, VC Type: Ethernet, GroupID: 0
+ MTU: 1500
+ Remote Label: 26
+ Cbit: 1, VC Type: Ethernet, GroupID: 0
+ MTU: 1500
+```
+
+## Verification - Data Plane
+
+Verify that all the exchanged label mappings were installed in zebra:
+```
+linux# show mpls table
+ Inbound Outbound
+ Label Type Nexthop Label
+-------- ------- --------------- --------
+ 17 LDP 2001:db8:3::2 3
+ 19 LDP 2001:db8:3::2 24005
+ 20 LDP 10.0.3.2 24000
+ 21 LDP 10.0.3.2 3
+ 22 LDP 10.0.3.2 24001
+ 23 LDP 10.0.3.2 3
+ 24 LDP 10.0.3.2 3
+ 25 LDP 10.0.3.2 3
+
+linux# show ip route ldp
+Codes: K - kernel route, C - connected, S - static, R - RIP,
+ O - OSPF, I - IS-IS, B - BGP, P - PIM, A - Babel, L - LDP,
+ > - selected route, * - FIB route
+
+L>* 1.1.1.1/32 [0/0] via 10.0.3.2, eth2 label 24000
+L>* 3.3.3.3/32 [0/0] via 10.0.3.2, eth2 label 24001
+```
+
+Verify that all the exchanged label mappings were installed in the kernel:
+```
+$ ip -M ro
+17 via inet6 2001:db8:3::2 dev eth2 proto zebra
+19 as to 24005 via inet6 2001:db8:3::2 dev eth2 proto zebra
+20 as to 24000 via inet 10.0.3.2 dev eth2 proto zebra
+21 via inet 10.0.3.2 dev eth2 proto zebra
+22 as to 24001 via inet 10.0.3.2 dev eth2 proto zebra
+23 via inet 10.0.3.2 dev eth2 proto zebra
+24 via inet 10.0.3.2 dev eth2 proto zebra
+25 via inet 10.0.3.2 dev eth2 proto zebra
+$
+$ ip route | grep mpls
+1.1.1.1 encap mpls 24000 via 10.0.3.2 dev eth2 proto zebra metric 20
+3.3.3.3 encap mpls 24001 via 10.0.3.2 dev eth2 proto zebra metric 20
+```
+
+Now ping PE1's loopback using lo1's address as a source address:
+```
+$ ping -c 5 -I 4.4.4.4 1.1.1.1
+PING 1.1.1.1 (1.1.1.1) from 4.4.4.4 : 56(84) bytes of data.
+64 bytes from 1.1.1.1: icmp_seq=1 ttl=253 time=3.02 ms
+64 bytes from 1.1.1.1: icmp_seq=2 ttl=253 time=3.13 ms
+64 bytes from 1.1.1.1: icmp_seq=3 ttl=253 time=3.19 ms
+64 bytes from 1.1.1.1: icmp_seq=4 ttl=253 time=3.07 ms
+64 bytes from 1.1.1.1: icmp_seq=5 ttl=253 time=3.27 ms
+
+--- 1.1.1.1 ping statistics ---
+5 packets transmitted, 5 received, 0% packet loss, time 4005ms
+rtt min/avg/max/mdev = 3.022/3.140/3.278/0.096 ms
+```
+
+Verify that the ICMP echo request packets are leaving with the MPLS
+label advertised by the P router. Also, verify that the ICMP echo reply
+packets are arriving with an explicit-null MPLS label:
+```
+# tcpdump -n -i eth2 mpls and icmp
+tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
+listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes
+10:01:40.758771 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 1, length 64
+10:01:40.761777 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 1, length 64
+10:01:41.760343 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 2, length 64
+10:01:41.763448 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 2, length 64
+10:01:42.761758 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 3, length 64
+10:01:42.764924 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 3, length 64
+10:01:43.763193 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 4, length 64
+10:01:43.766237 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 4, length 64
+10:01:44.764552 MPLS (label 24000, exp 0, [S], ttl 64) IP 4.4.4.4 > 1.1.1.1: ICMP echo request, id 13370, seq 5, length 64
+10:01:44.767803 MPLS (label 0, exp 0, [S], ttl 254) IP 1.1.1.1 > 4.4.4.4: ICMP echo reply, id 13370, seq 5, length 64
+```
diff --git a/doc/developer/library.rst b/doc/developer/library.rst
new file mode 100644
index 0000000000..791aedb624
--- /dev/null
+++ b/doc/developer/library.rst
@@ -0,0 +1,12 @@
+libfrr library facilities
+=========================
+
+.. toctree::
+ :maxdepth: 2
+
+ memtypes
+ hooks
+ cli
+ modules
+
+
diff --git a/doc/developer/memtypes.rst b/doc/developer/memtypes.rst
new file mode 100644
index 0000000000..d40ebe31cd
--- /dev/null
+++ b/doc/developer/memtypes.rst
@@ -0,0 +1,117 @@
+.. highlight:: c
+
+Memtypes
+========
+
+FRR includes wrappers arround ``malloc()`` and ``free()`` that count the number
+of objects currently allocated, for each of a defined ``MTYPE``.
+
+To this extent, there are `memory groups` and `memory types`. Each memory
+type must belong to a memory group, this is used just to provide some basic
+structure.
+
+Example:
+
+.. code-block:: c
+ :caption: mydaemon.h
+
+ DECLARE_MGROUP(MYDAEMON)
+ DECLARE_MTYPE(MYNEIGHBOR)
+
+.. code-block:: c
+ :caption: mydaemon.c
+
+ DEFINE_MGROUP( MYDAEMON, "My daemon's memory")
+ DEFINE_MTYPE( MYDAEMON, MYNEIGHBOR, "Neighbor entry")
+ DEFINE_MTYPE_STATIC(MYDAEMON, MYNEIGHBORNAME, "Neighbor name")
+
+ struct neigh *neighbor_new(const char *name)
+ {
+ struct neigh *n = XMALLOC(MYNEIGHBOR, sizeof(*n));
+ n->name = XSTRDUP(MYNEIGHBORNAME, name);
+ return n;
+ }
+
+ void neighbor_free(struct neigh *n)
+ {
+ XFREE(MYNEIGHBORNAME, n->name);
+ XFREE(MYNEIGHBOR, n);
+ }
+
+
+Definition
+----------
+
+.. c:macro:: DECLARE_MGROUP(name)
+
+ This macro forward-declares a memory group and should be placed in a
+ ``.h`` file. It expands to an ``extern struct memgroup`` statement.
+
+.. c:macro:: DEFINE_MGROUP(mname, description)
+
+ Defines/implements a memory group. Must be placed into exactly one ``.c``
+ file (multiple inclusion will result in a link-time symbol conflict).
+
+ Contains additional logic (constructor and destructor) to register the
+ memory group in a global list.
+
+.. c:macro:: DECLARE_MTYPE(name)
+
+ Forward-declares a memory type and makes ``MTYPE_name`` available for use.
+ Note that the ``MTYPE_`` prefix must not be included in the name, it is
+ automatically prefixed.
+
+ ``MTYPE_name`` is created as a `static const` symbol, i.e. a compile-time
+ constant. It refers to an ``extern struct memtype _mt_name``, where `name`
+ is replaced with the actual name.
+
+.. c:macro:: DEFINE_MTYPE(group, name, description)
+
+ Define/implement a memory type, must be placed into exactly one ``.c``
+ file (multiple inclusion will result in a link-time symbol conflict).
+
+ Like ``DEFINE_MGROUP``, this contains actual code to register the MTYPE
+ under its group.
+
+.. c:macro:: DEFINE_MTYPE_STATIC(group, name, description)
+
+ Same as ``DEFINE_MTYPE``, but the ``DEFINE_MTYPE_STATIC`` variant places
+ the C ``static`` keyword on the definition, restricting the MTYPE's
+ availability to the current source file. This should be appropriate in
+ >80% of cases.
+
+ .. todo::
+
+ Daemons currently have ``daemon_memory.[ch]`` files listing all of
+ their MTYPEs. This is not how it should be, most of these types
+ should be moved into the appropriate files where they are used.
+ Only a few MTYPEs should remain non-static after that.
+
+
+Usage
+-----
+
+.. c:function:: void *XMALLOC(struct memtype *mtype, size_t size)
+
+.. c:function:: void *XCALLOC(struct memtype *mtype, size_t size)
+
+.. c:function:: void *XSTRDUP(struct memtype *mtype, size_t size)
+
+ Allocation wrappers for malloc/calloc/realloc/strdup, taking an extra
+ mtype parameter.
+
+.. c:function:: void *XREALLOC(struct memtype *mtype, void *ptr, size_t size)
+
+ Wrapper around realloc() with MTYPE tracking. Note that ``ptr`` may
+ be NULL, in which case the function does the same as XMALLOC (regardless
+ of whether the system realloc() supports this.)
+
+.. c:function:: void XFREE(struct memtype *mtype, void *ptr)
+
+ Wrapper around free(), again taking an extra mtype parameter. This is
+ actually a macro, with the following additional properties:
+
+ - the macro contains ``ptr = NULL``
+ - if ptr is NULL, no operation is performed (as is guaranteed by system
+ implementations.) Do not surround XFREE with ``if (ptr != NULL)``
+ checks.
diff --git a/doc/developer/modules.rst b/doc/developer/modules.rst
new file mode 100644
index 0000000000..b832413a6c
--- /dev/null
+++ b/doc/developer/modules.rst
@@ -0,0 +1,123 @@
+Modules
+=======
+
+FRR has facilities to load DSOs at startup via ``dlopen()``. These are used to
+implement modules, such as SNMP and FPM.
+
+Limitations
+-----------
+
+- can't load, unload, or reload during runtime. This just needs some
+ work and can probably be done in the future.
+- doesn't fix any of the "things need to be changed in the code in the
+ library" issues. Most prominently, you can't add a CLI node because
+ CLI nodes are listed in the library...
+- if your module crashes, the daemon crashes. Should be obvious.
+- **does not provide a stable API or ABI**. Your module must match a
+ version of FRR and you may have to update it frequently to match
+ changes.
+- **does not create a license boundary**. Your module will need to link
+ libzebra and include header files from the daemons, meaning it will
+ be GPL-encumbered.
+
+Installation
+------------
+
+Look for ``moduledir`` in ``configure.ac``, default is normally
+``/usr/lib64/frr/modules`` but depends on ``--libdir`` / ``--prefix``.
+
+The daemon's name is prepended when looking for a module, e.g. "snmp"
+tries to find "zebra\_snmp" first when used in zebra. This is just to
+make it nicer for the user, with the snmp module having the same name
+everywhere.
+
+Modules can be packaged separately from FRR. The SNMP and FPM modules
+are good candidates for this because they have dependencies (net-snmp /
+protobuf) that are not FRR dependencies. However, any distro packages
+should have an "exact-match" dependency onto the FRR package. Using a
+module from a different FRR version will probably blow up nicely.
+
+For snapcraft (and during development), modules can be loaded with full
+path (e.g. -M ``$SNAP/lib/frr/modules/zebra_snmp.so``). Note that
+libtool puts output files in the .libs directory, so during development
+you have to use ``./zebra -M .libs/zebra_snmp.so``.
+
+Creating a module
+-----------------
+
+... best to look at the existing SNMP or FPM modules.
+
+Basic boilerplate:
+
+::
+
+ #include "hook.h"
+ #include "module.h"
+
+ static int
+ module_init (void)
+ {
+ hook_register(frr_late_init, module_late_init);
+ return 0;
+ }
+
+ FRR_MODULE_SETUP(
+ .name = "my module",
+ .version = "0.0",
+ .description = "my module",
+ .init = module_init,
+ )
+
+The ``frr_late_init`` hook will be called after the daemon has finished
+its other startup and is about to enter the main event loop; this is the
+best place for most initialisation.
+
+Compiler & Linker magic
+-----------------------
+
+There's a ``THIS_MODULE`` (like in the Linux kernel), which uses
+``visibility`` attributes to restrict it to the current module. If you
+get a linker error with ``_frrmod_this_module``, there is some linker
+SNAFU. This shouldn't be possible, though one way to get it would be to
+not include libzebra (which provides a fallback definition for the
+symbol).
+
+libzebra and the daemons each have their own ``THIS_MODULE``, as do all
+loadable modules. In any other libraries (e.g. ``libfrrsnmp``),
+``THIS_MODULE`` will use the definition in libzebra; same applies if the
+main executable doesn't use ``FRR_DAEMON_INFO`` (e.g. all testcases).
+
+The deciding factor here is "what dynamic linker unit are you using the
+symbol from." If you're in a library function and want to know who
+called you, you can't use ``THIS_MODULE`` (because that'll just tell you
+you're in the library). Put a macro around your function that adds
+``THIS_MODULE`` in the *caller's code calling your function*.
+
+The idea is to use this in the future for module unloading. Hooks
+already remember which module they were installed by, as groundwork for
+a function that removes all of a module's installed hooks.
+
+There's also the ``frr_module`` symbol in modules, pretty much a
+standard entry point for loadable modules.
+
+Hooks
+-----
+
+Hooks are just points in the code where you can register your callback
+to be called. The parameter list is specific to the hook point. Since
+there is no stable API, the hook code has some extra type safety checks
+making sure you get a compiler warning when the hook parameter list
+doesn't match your callback. Don't ignore these warnings.
+
+Relation to MTYPE macros
+------------------------
+
+The MTYPE macros, while primarily designed to decouple MTYPEs from the
+library and beautify the code, also work very nicely with loadable
+modules -- both constructors and destructors are executed when
+loading/unloading modules.
+
+This means there is absolutely no change required to MTYPEs, you can
+just use them in a module and they will even clean up themselves when we
+implement module unloading and an unload happens. In fact, it's
+impossible to create a bug where unloading fails to de-register a MTYPE.
diff --git a/doc/developer/next-hop-tracking.rst b/doc/developer/next-hop-tracking.rst
new file mode 100644
index 0000000000..284db1ef53
--- /dev/null
+++ b/doc/developer/next-hop-tracking.rst
@@ -0,0 +1,352 @@
+Next Hop Tracking
+==================
+
+Next hop tracking is an optimization feature that reduces the processing time
+involved in the BGP bestpath algorithm by monitoring changes to the routing
+table.
+
+Background
+-----------
+
+Recursive routes are of the form:
+
+::
+
+ p/m --> n
+ [Ex: 1.1.0.0/16 --> 2.2.2.2]
+
+where 'n' itself is resolved through another route as follows:
+
+::
+
+ p2/m --> h, interface
+ [Ex: 2.2.2.0/24 --> 3.3.3.3, eth0]
+
+Usually, BGP routes are recursive in nature and BGP nexthops get resolved
+through an IGP route. IGP usually adds its routes pointing to an interface
+(these are called non-recursive routes).
+
+When BGP receives a recursive route from a peer, it needs to validate the
+nexthop. The path is marked valid or invalid based on the reachability status
+of the nexthop. Nexthop validation is also important for BGP decision process
+as the metric to reach the nexthop is a parameter to best path selection
+process.
+
+As it goes with routing, this is a dynamic process. Route to the nexthop can
+change. The nexthop can become unreachable or reachable. In the current BGP
+implementation, the nexthop validation is done periodically in the scanner run.
+The default scanner run interval is one minute. Every minute, the scanner task
+walks the entire BGP table. It checks the validity of each nexthop with Zebra
+(the routing table manager) through a request and response message exchange
+between BGP and Zebra process. BGP process is blocked for that duration. The
+mechanism has two major drawbacks:
+
+- The scanner task runs to completion. That can potentially starve the other
+ tasks for long periods of time, based on the BGP table size and number of
+ nexthops.
+
+- Convergence around routing changes that affect the nexthops can be long
+ (around a minute with the default intervals). The interval can be shortened
+ to achieve faster reaction time, but it makes the first problem worse, with
+ the scanner task consuming most of the CPU resources.
+
+The next-hop tracking feature makes this process event-driven. It eliminates
+periodic nexthop validation and introduces an asynchronous communication path
+between BGP and Zebra for route change notifications that can then be acted
+upon.
+
+Goal
+----
+
+Stating the obvious, the main goal is to remove the two limitations we
+discussed in the previous section. The goals, in a constructive tone,
+are the following:
+
+- **Fairness**: the scanner run should not consume an unjustly high amount of
+ CPU time. This should give an overall good performance and response time to
+ other events (route changes, session events, IO/user interface).
+
+- **Convergence**: BGP must react to nexthop changes instantly and provide
+ sub-second convergence. This may involve diverting the routes from one
+ nexthop to another.
+
+Overview of changes
+------------------------
+
+The changes are in both BGP and Zebra modules. The short summary is
+the following:
+
+- Zebra implements a registration mechanism by which clients can
+ register for next hop notification. Consequently, it maintains a
+ separate table, per (VRF, AF) pair, of next hops and interested
+ client-list per next hop.
+
+- When the main routing table changes in Zebra, it evaluates the next
+ hop table: for each next hop, it checks if the route table
+ modifications have changed its state. If so, it notifies the
+ interested clients.
+
+- BGP is one such client. It registers the next hops corresponding to
+ all of its received routes/paths. It also threads the paths against
+ each nexthop structure.
+
+- When BGP receives a next hop notification from Zebra, it walks the
+ corresponding path list. It makes them valid or invalid depending
+ on the next hop notification. It then re-computes best path for the
+ corresponding destination. This may result in re-announcing those
+ destinations to peers.
+
+Design
+------
+
+Modules
+~~~~~~~
+
+The core design introduces an "nht" (next hop tracking) module in BGP
+and "rnh" (recursive nexthop) module in Zebra. The "nht" module
+provides the following APIs:
+
++----------------------------+--------------------------------------------------+
+| Function | Action |
++============================+==================================================+
+| bgp_find_or_add_nexthop() | find or add a nexthop in BGP nexthop table |
++----------------------------+--------------------------------------------------+
+| bgp_find_nexthop() | find a nexthop in BGP nexthop table |
++----------------------------+--------------------------------------------------+
+| bgp_parse_nexthop_update() | parse a nexthop update message coming from zebra |
++----------------------------+--------------------------------------------------+
+
+The "rnh" module provides the following APIs:
+
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| Function | Action |
++============================+==========================================================================================================+
+| zebra_add_rnh() | add a recursive nexthop |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| zebra_delete_rnh() | delete a recursive nexthop |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| zebra_lookup_rnh() | lookup a recursive nexthop |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| zebra_add_rnh_client() | register a client for nexthop notifications against a recursive nexthop |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| zebra_remove_rnh_client() | remove the client registration for a recursive nexthop |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| zebra_evaluate_rnh_table() | (re)evaluate the recursive nexthop table (most probably because the main routing table has changed). |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+| zebra_cleanup_rnh_client() | Cleanup a client from the "rnh" module data structures (most probably because the client is going away). |
++----------------------------+----------------------------------------------------------------------------------------------------------+
+
+4.2. Control flow
+
+The next hop registration control flow is the following:
+
+::
+
+ <==== BGP Process ====>|<==== Zebra Process ====>
+ |
+ receive module nht module | zserv module rnh module
+ ----------------------------------------------------------------------
+ | | |
+ bgp_update_ | | |
+ main() | bgp_find_or_add_ | |
+ | nexthop() | |
+ | | |
+ | | zserv_nexthop_ |
+ | | register() |
+ | | | zebra_add_rnh()
+ | | |
+
+
+The next hop notification control flow is the following:
+
+::
+
+ <==== Zebra Process ====>|<==== BGP Process ====>
+ |
+ rib module rnh module | zebra module nht module
+ ----------------------------------------------------------------------
+ | | |
+ meta_queue_ | | |
+ process() | zebra_evaluate_ | |
+ | rnh_table() | |
+ | | |
+ | | bgp_read_nexthop_ |
+ | | update() |
+ | | | bgp_parse_
+ | | | nexthop_update()
+ | | |
+
+
+zclient message format
+~~~~~~~~~~~~~~~~~~~~~~
+
+ZEBRA_NEXTHOP_REGISTER and ZEBRA_NEXTHOP_UNREGISTER messages are
+encoded in the following way:
+
+::
+
+ . 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | AF | prefix len |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . Nexthop prefix .
+ . .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . .
+ . .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | AF | prefix len |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . Nexthop prefix .
+ . .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+
+``ZEBRA_NEXTHOP_UPDATE`` message is encoded as follows:
+
+::
+
+ . 0 1 2 3
+ 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | AF | prefix len |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . Nexthop prefix getting resolved .
+ . .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | metric |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | #nexthops |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | nexthop type |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . resolving Nexthop details .
+ . .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ | nexthop type |
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ . resolving Nexthop details .
+ +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+
+BGP data structure
+~~~~~~~~~~~~~~~~~~
+Legend:
+
+::
+
+ /\ struct bgp_node: a BGP destination/route/prefix
+ \/
+
+ [ ] struct bgp_info: a BGP path (e.g. route received from a peer)
+
+ _
+ (_) struct bgp_nexthop_cache: a BGP nexthop
+
+ /\ NULL
+ \/--+ ^
+ | :
+ +--[ ]--[ ]--[ ]--> NULL
+ /\ :
+ \/--+ :
+ | :
+ +--[ ]--[ ]--> NULL
+ :
+ _ :
+ (_)...........
+
+
+Zebra data structure
+~~~~~~~~~~~~~~~~~~~~
+
+RNH table::
+
+ O
+ / \
+ O O
+ / \
+ O O
+
+ struct rnh
+ {
+ u_char flags;
+ struct route_entry *state;
+ struct list *client_list;
+ struct route_node *node;
+ };
+
+User interface changes
+~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ frr# show ip nht
+ 3.3.3.3
+ resolved via kernel
+ via 11.0.0.6, swp1
+ Client list: bgp(fd 12)
+ 11.0.0.10
+ resolved via connected
+ is directly connected, swp2
+ Client list: bgp(fd 12)
+ 11.0.0.18
+ resolved via connected
+ is directly connected, swp4
+ Client list: bgp(fd 12)
+ 11.11.11.11
+ resolved via kernel
+ via 10.0.1.2, eth0
+ Client list: bgp(fd 12)
+
+ frr# show ip bgp nexthop
+ Current BGP nexthop cache:
+ 3.3.3.3 valid [IGP metric 0], #paths 3
+ Last update: Wed Oct 16 04:43:49 2013
+
+ 11.0.0.10 valid [IGP metric 1], #paths 1
+ Last update: Wed Oct 16 04:43:51 2013
+
+ 11.0.0.18 valid [IGP metric 1], #paths 2
+ Last update: Wed Oct 16 04:43:47 2013
+
+ 11.11.11.11 valid [IGP metric 0], #paths 1
+ Last update: Wed Oct 16 04:43:47 2013
+
+ frr# show ipv6 nht
+ frr# show ip bgp nexthop detail
+
+ frr# debug bgp nht
+ frr# debug zebra nht
+
+ 6. Sample test cases
+
+ r2----r3
+ / \ /
+ r1----r4
+
+ - Verify that a change in IGP cost triggers NHT
+ + shutdown the r1-r4 and r2-r4 links
+ + no shut the r1-r4 and r2-r4 links and wait for OSPF to come back
+ up
+ + We should be back to the original nexthop via r4 now
+ - Verify that a NH becoming unreachable triggers NHT
+ + Shutdown all links to r4
+ - Verify that a NH becoming reachable triggers NHT
+ + no shut all links to r4
+
+Future work
+~~~~~~~~~~~
+
+- route-policy for next hop validation (e.g. ignore default route)
+- damping for rapid next hop changes
+- prioritized handling of nexthop changes ((un)reachability vs. metric
+ changes)
+- handling recursion loop, e.g::
+
+ 11.11.11.11/32 -> 12.12.12.12
+ 12.12.12.12/32 -> 11.11.11.11
+ 11.0.0.0/8 -> <interface>
+- better statistics
diff --git a/doc/developer/ospf-api.rst b/doc/developer/ospf-api.rst
new file mode 100644
index 0000000000..5f0b830c8c
--- /dev/null
+++ b/doc/developer/ospf-api.rst
@@ -0,0 +1,388 @@
+OSPF API Documentation
+======================
+
+Disclaimer
+----------
+
+The OSPF daemon contains an API for application access to the LSA
+database. This API was created by Ralph Keller, originally as patch for
+Zebra. Unfortunately, the page containing documentation of the API is no
+longer online. This page is an attempt to recreate documentation for the
+API (with lots of help of the WayBackMachine)
+
+Introduction
+------------
+
+This page describes an API that allows external applications to access
+the link-state database (LSDB) of the OSPF daemon. The implementation is
+based on the OSPF code from FRRouting (forked from Quagga and formerly
+Zebra) routing protocol suite and is subject to the GNU General Public
+License. The OSPF API provides you with the following functionality:
+
+- Retrieval of the full or partial link-state database of the OSPF
+ daemon. This allows applications to obtain an exact copy of the LSDB
+ including router LSAs, network LSAs and so on. Whenever a new LSA
+ arrives at the OSPF daemon, the API module immediately informs the
+ application by sending a message. This way, the application is always
+ synchronized with the LSDB of the OSPF daemon.
+- Origination of own opaque LSAs (of type 9, 10, or 11) which are then
+ distributed transparently to other routers within the flooding scope
+ and received by other applications through the OSPF API.
+
+Opaque LSAs, which are described in RFC 2370 , allow you to distribute
+application-specific information within a network using the OSPF
+protocol. The information contained in opaque LSAs is transparent for
+the routing process but it can be processed by other modules such as
+traffic engineering (e.g., MPLS-TE).
+
+Architecture
+------------
+
+The following picture depicts the architecture of the Quagga/Zebra
+protocol suite. The OSPF daemon is extended with opaque LSA capabilities
+and an API for external applications. The OSPF core module executes the
+OSPF protocol by discovering neighbors and exchanging neighbor state.
+The opaque module, implemented by Masahiko Endo, provides functions to
+exchange opaque LSAs between routers. Opaque LSAs can be generated by
+several modules such as the MPLS-TE module or the API server module.
+These modules then invoke the opaque module to flood their data to
+neighbors within the flooding scope.
+
+The client, which is an application potentially running on a different
+node than the OSPF daemon, links against the OSPF API client library.
+This client library establishes a socket connection with the API server
+module of the OSPF daemon and uses this connection to retrieve LSAs and
+originate opaque LSAs.
+
+.. figure:: ../figures/ospf_api_architecture.png
+ :alt: image
+
+ image
+
+The OSPF API server module works like any other internal opaque module
+(such as the MPLS-TE module), but listens to connections from external
+applications that want to communicate with the OSPF daemon. The API
+server module can handle multiple clients concurrently.
+
+One of the main objectives of the implementation is to make as little
+changes to the existing Zebra code as possible.
+
+Installation & Configuration
+----------------------------
+
+Download FRRouting and unpack
+
+Configure your frr version (note that --enable-opaque-lsa also enables
+the ospfapi server and ospfclient).
+
+::
+
+ % update-autotools
+ % sh ./configure --enable-opaque-lsa
+ % make
+
+This should also compile the client library and sample application in
+ospfclient.
+
+Make sure that you have enabled opaque LSAs in your configuration. Add
+the ospf opaque-lsa statement to your ospfd.conf:
+
+::
+
+ ! -*- ospf -*-
+ !
+ ! OSPFd sample configuration file
+ !
+ !
+ hostname xxxxx
+ password xxxxx
+
+ router ospf
+ router-id 10.0.0.1
+ network 10.0.0.1/24 area 1
+ neighbor 10.0.0.2
+ network 10.0.1.2/24 area 1
+ neighbor 10.0.1.1
+ ospf opaque-lsa <============ add this statement!
+
+Usage
+-----
+
+In the following we describe how you can use the sample application to
+originate opaque LSAs. The sample application first registers with the
+OSPF daemon the opaque type it wants to inject and then waits until the
+OSPF daemon is ready to accept opaque LSAs of that type. Then the client
+application originates an opaque LSA, waits 10 seconds and then updates
+the opaque LSA with new opaque data. After another 20 seconds, the
+client application deletes the opaque LSA from the LSDB. If the clients
+terminates unexpectedly, the OSPF API module will remove all the opaque
+LSAs that the application registered. Since the opaque LSAs are flooded
+to other routers, we will see the opaque LSAs in all routers according
+to the flooding scope of the opaque LSA.
+
+We have a very simple demo setup, just two routers connected with an ATM
+point-to-point link. Start the modified OSPF daemons on two adjacent
+routers. First run on msr2:
+
+::
+
+ > msr2:/home/keller/ospfapi/zebra/ospfd# ./ospfd -f /usr/local/etc/ospfd.conf
+
+And on the neighboring router msr3:
+
+::
+
+ > msr3:/home/keller/ospfapi/zebra/ospfd# ./ospfd -f /usr/local/etc/ospfd.conf
+
+Now the two routers form adjacency and start exchanging their databases.
+Looking at the OSPF daemon of msr2 (or msr3), you see this:
+
+::
+
+ ospfd> show ip ospf database
+
+ OSPF Router with ID (10.0.0.1)
+
+ Router Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum Link count
+ 10.0.0.1 10.0.0.1 55 0x80000003 0xc62f 2
+ 10.0.0.2 10.0.0.2 55 0x80000003 0xe3e4 3
+
+ Net Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum
+ 10.0.0.2 10.0.0.2 60 0x80000001 0x5fcb
+
+Now we start the sample main application that originates an opaque LSA.
+
+::
+
+ > cd ospfapi/apiclient
+ > ./main msr2 10 250 20 0.0.0.0 0.0.0.1
+
+This originates an opaque LSA of type 10 (area local), with opaque type
+250 (experimental), opaque id of 20 (chosen arbitrarily), interface
+address 0.0.0.0 (which is used only for opaque LSAs type 9), and area
+0.0.0.1
+
+Again looking at the OSPF database you see:
+
+::
+
+ ospfd> show ip ospf database
+
+ OSPF Router with ID (10.0.0.1)
+
+ Router Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum Link count
+ 10.0.0.1 10.0.0.1 437 0x80000003 0xc62f 2
+ 10.0.0.2 10.0.0.2 437 0x80000003 0xe3e4 3
+
+ Net Link States (Area 0.0.0.1)
+
+ Link ID ADV Router Age Seq# CkSum
+ 10.0.0.2 10.0.0.2 442 0x80000001 0x5fcb
+
+ Area-Local Opaque-LSA (Area 0.0.0.1)
+
+ Opaque-Type/Id ADV Router Age Seq# CkSum
+ 250.0.0.20 10.0.0.1 0 0x80000001 0x58a6 <=== opaque LSA
+
+You can take a closer look at this opaque LSA:
+
+::
+
+ ospfd> show ip ospf database opaque-area
+
+ OSPF Router with ID (10.0.0.1)
+
+
+ Area-Local Opaque-LSA (Area 0.0.0.1)
+
+ LS age: 4
+ Options: 66
+ LS Type: Area-Local Opaque-LSA
+ Link State ID: 250.0.0.20 (Area-Local Opaque-Type/ID)
+ Advertising Router: 10.0.0.1
+ LS Seq Number: 80000001
+ Checksum: 0x58a6
+ Length: 24
+ Opaque-Type 250 (Private/Experimental)
+ Opaque-ID 0x14
+ Opaque-Info: 4 octets of data
+ Added using OSPF API: 4 octets of opaque data
+ Opaque data: 1 0 0 0 <==== counter is 1
+
+Note that the main application updates the opaque LSA after 10 seconds,
+then it looks as follows:
+
+::
+
+ ospfd> show ip ospf database opaque-area
+
+ OSPF Router with ID (10.0.0.1)
+
+
+ Area-Local Opaque-LSA (Area 0.0.0.1)
+
+ LS age: 1
+ Options: 66
+ LS Type: Area-Local Opaque-LSA
+ Link State ID: 250.0.0.20 (Area-Local Opaque-Type/ID)
+ Advertising Router: 10.0.0.1
+ LS Seq Number: 80000002
+ Checksum: 0x59a3
+ Length: 24
+ Opaque-Type 250 (Private/Experimental)
+ Opaque-ID 0x14
+ Opaque-Info: 4 octets of data
+ Added using OSPF API: 4 octets of opaque data
+ Opaque data: 2 0 0 0 <==== counter is now 2
+
+Note that the payload of the opaque LSA has changed as you can see
+above.
+
+Then, again after another 20 seconds, the opaque LSA is flushed from the
+LSDB.
+
+Important note:
+^^^^^^^^^^^^^^^
+
+In order to originate an opaque LSA, there must be at least one active
+opaque-capable neighbor. Thus, you cannot originate opaque LSAs of no
+neighbors are present. If you try to originate even so no neighbor is
+ready, you will receive a not ready error message. The reason for this
+restriction is that it might be possible that some routers have an
+identical opaque LSA from a previous origination in their LSDB that
+unfortunately could not be flushed due to a crash, and now if the router
+comes up again and starts originating a new opaque LSA, the new opaque
+LSA is considered older since it has a lower sequence number and is
+ignored by other routers (that consider the stalled opaque LSA as more
+recent). However, if the originating router first synchronizes the
+database before originating opaque LSAs, it will detect the older opaque
+LSA and can flush it first.
+
+Protocol and Message Formats
+----------------------------
+
+If you are developing your own client application and you don't want to
+make use of the client library (due to the GNU license restriction or
+whatever reason), you can implement your own client-side message
+handling. The OSPF API uses two connections between the client and the
+OSPF API server: One connection is used for a synchronous request /reply
+protocol and another connection is used for asynchronous notifications
+(e.g., LSA update, neighbor status change).
+
+Each message begins with the following header:
+
+.. figure:: ../figures/ospf_api_msghdr.png
+ :alt: image
+
+ image
+
+The message type field can take one of the following values:
+
++-------------------------------+---------+
+| Messages to OSPF deamon | Value |
++===============================+=========+
+| MSG\_REGISTER\_OPAQUETYPE | 1 |
++-------------------------------+---------+
+| MSG\_UNREGISTER\_OPAQUETYPE | 2 |
++-------------------------------+---------+
+| MSG\_REGISTER\_EVENT | 3 |
++-------------------------------+---------+
+| MSG\_SYNC\_LSDB | 4 |
++-------------------------------+---------+
+| MSG\_ORIGINATE\_REQUEST | 5 |
++-------------------------------+---------+
+| MSG\_DELETE\_REQUEST | 6 |
++-------------------------------+---------+
+
++-----------------------------+---------+
+| Messages from OSPF deamon | Value |
++=============================+=========+
+| MSG\_REPLY | 10 |
++-----------------------------+---------+
+| MSG\_READY\_NOTIFY | 11 |
++-----------------------------+---------+
+| MSG\_LSA\_UPDATE\_NOTIFY | 12 |
++-----------------------------+---------+
+| MSG\_LSA\_DELETE\_NOTIFY | 13 |
++-----------------------------+---------+
+| MSG\_NEW\_IF | 14 |
++-----------------------------+---------+
+| MSG\_DEL\_IF | 15 |
++-----------------------------+---------+
+| MSG\_ISM\_CHANGE | 16 |
++-----------------------------+---------+
+| MSG\_NSM\_CHANGE | 17 |
++-----------------------------+---------+
+
+The synchronous requests and replies have the following message formats:
+
+.. figure:: ../figures/ospf_api_msgs1.png
+ :alt: image
+
+ image
+
+The origin field allows to select according to the following types of
+origins:
+
++-------------------------+---------+
+| Origin | Value |
++=========================+=========+
+| NON\_SELF\_ORIGINATED | 0 |
++-------------------------+---------+
+| SELF\_ORIGINATED | 1 |
++-------------------------+---------+
+| ANY\_ORIGIN | 2 |
++-------------------------+---------+
+
+The reply message has on of the following error codes:
+
++--------------------------+---------+
+| Error code | Value |
++==========================+=========+
+| API\_OK | 0 |
++--------------------------+---------+
+| API\_NOSUCHINTERFACE | -1 |
++--------------------------+---------+
+| API\_NOSUCHAREA | -2 |
++--------------------------+---------+
+| API\_NOSUCHLSA | -3 |
++--------------------------+---------+
+| API\_ILLEGALSATYPE | -4 |
++--------------------------+---------+
+| API\_ILLEGALOPAQUETYPE | -5 |
++--------------------------+---------+
+| API\_OPAQUETYPEINUSE | -6 |
++--------------------------+---------+
+| API\_NOMEMORY | -7 |
++--------------------------+---------+
+| API\_ERROR | -99 |
++--------------------------+---------+
+| API\_UNDEF | -100 |
++--------------------------+---------+
+
+The asynchronous notifications have the following message formats:
+
+.. figure:: ../figures/ospf_api_msgs2.png
+ :alt: image
+
+ image
+
+Original Acknowledgments from Ralph Keller
+------------------------------------------
+
+I would like to thank Masahiko Endo, the author of the opaque LSA
+extension module, for his great support. His wonderful ASCII graphs
+explaining the internal workings of this code, and his invaluable input
+proved to be crucial in designing a useful API for accessing the link
+state database of the OSPF daemon. Once, he even decided to take the
+plane from Tokyo to Zurich so that we could actually meet and have
+face-to-face discussions, which was a lot of fun. Clearly, without
+Masahiko no API would ever be completed. I also would like to thank
+Daniel Bauer who wrote an opaque LSA implementation too and was willing
+to test the OSPF API code in one of his projects.
diff --git a/doc/developer/ospf-sr.rst b/doc/developer/ospf-sr.rst
new file mode 100644
index 0000000000..3f8b549df3
--- /dev/null
+++ b/doc/developer/ospf-sr.rst
@@ -0,0 +1,285 @@
+OSPF Segment Routing
+====================
+
+This is an EXPERIMENTAL support of draft
+`draft-ietf-ospf-segment-routing-extensions-24`.
+DON'T use it for production network.
+
+Supported Features
+------------------
+
+* Automatic computation of Primary and Backup Adjacency SID with
+ Cisco experimental remote IP address
+* SRGB configuration
+* Prefix configuration for Node SID with optional NO-PHP flag (Linux
+ kernel support both mode)
+* Node MSD configuration (with Linux Kernel >= 4.10 a maximum of 32 labels
+ could be stack)
+* Automatic provisioning of MPLS table
+* Static route configuration with label stack up to 32 labels
+
+Interoperability
+----------------
+
+* Tested on various topology including point-to-point and LAN interfaces
+ in a mix of Free Range Routing instance and Cisco IOS-XR 6.0.x
+* Check OSPF LSA conformity with latest wireshark release 2.5.0-rc
+
+Implementation details
+----------------------
+
+Concepts
+~~~~~~~~
+
+Segment Routing used 3 differents OPAQUE LSA in OSPF to carry the various
+information:
+
+* **Router Information:** flood the Segment Routing capabilities of the node.
+ This include the supported algorithms, the Segment Routing Global Block
+ (SRGB) and the Maximum Stack Depth (MSD).
+* **Extended Link:** flood the Adjaceny and Lan Adjacency Segment Identifier
+* **Extended Prefix:** flood the Prefix Segment Identifier
+
+The implementation follow previous TE and Router Information codes. It used the
+OPAQUE LSA functions defined in ospf_opaque.[c,h] as well as the OSPF API. This
+latter is mandatory for the implementation as it provides the Callback to
+Segment Routing functions (see below) when an Extended Link / Prefix or Router
+Information LSA s are received.
+
+Overview
+~~~~~~~~
+
+Following files where modified or added:
+
+* ospd_ri.[c,h] have been modified to add the new TLVs for Segment Routing.
+* ospf_ext.[c,h] implement RFC7684 as base support of Extended Link and Prefix
+ Opaque LSA.
+* ospf_sr.[c,h] implement the earth of Segment Routing. It adds a new Segment
+ Routing database to manage Segment Identifiers per Link and Prefix and
+ Segment Routing enable node, Callback functions to process incoming LSA and
+ install MPLS FIB entry through Zebra.
+
+The figure below shows the relation between the various files:
+
+* ospf_sr.c centralized all the Segment Routing processing. It receives Opaque
+ LSA Router Information (4.0.0.0) from ospf_ri.c and Extended Prefix
+ (7.0.0.X) Link (8.0.0.X) from ospf_ext.c. Once received, it parse TLVs and
+ SubTLVs and store information in SRDB (which is defined in ospf_sr.h). For
+ each received LSA, NHLFE is computed and send to Zebra to add/remove new
+ MPLS labels entries and FEC. New CLI configurations are also centralized in
+ ospf_sr.c. This CLI will trigger the flooding of new LSA Router Information
+ (4.0.0.0), Extended Prefix (7.0.0.X) and Link (8.0.0.X) by ospf_ri.c,
+ respectively ospf_ext.c.
+* ospf_ri.c send back to ospf_sr.c received Router Information LSA and update
+ Self Router Information LSA with paramters provided by ospf_sr.c i.e. SRGB
+ and MSD. It use ospf_opaque.c functions to send/received these Opaque LSAs.
+* ospf_ext.c send back to ospf_sr.c received Extended Prefix and Link Opaque
+ LSA and send self Extended Prefix and Link Opaque LSA through ospf_opaque.c
+ functions.
+
+::
+
+ +-----------+ +-------+
+ | | | |
+ | ospf_sr.c +-----+ SRDB |
+ +-----------+ +--+ | |
+ | +-^-------^-+ | +-------+
+ | | | | |
+ | | | | |
+ | | | | +--------+
+ | | | | |
+ +---v----------+ | | | +-----v-------+
+ | | | | | | |
+ | ospf_ri.c +--+ | +-------+ ospf_ext.c |
+ | LSA 4.0.0.0 | | | LSA 7.0.0.X |
+ | | | | LSA 8.0.0.X |
+ +---^----------+ | | |
+ | | +-----^-------+
+ | | |
+ | | |
+ | +--------v------------+ |
+ | | | |
+ | | ZEBRA: Labels + FEC | |
+ | | | |
+ | +---------------------+ |
+ | |
+ | |
+ | +---------------+ |
+ | | | |
+ +---------> ospf_opaque.c <---------+
+ | |
+ +---------------+
+
+ Figure 1: Overview of Segment Routing interaction
+
+Module interactions
+~~~~~~~~~~~~~~~~~~~
+
+To process incoming LSA, the code is based on the capability to call `hook()`
+functions when LSA are inserted or delete to / from the LSDB and the
+possibility to register particular treatment for Opaque LSA. The first point
+is provided by the OSPF API feature and the second by the Opaque implementation
+itself. Indeed, it is possible to register callback function for a given Opaque
+LSA ID (see `ospf_register_opaque_functab()` function defined in
+`ospf_opaque.c`). Each time a new LSA is added to the LSDB, the
+`new_lsa_hook()` function previously register for this LSA type is called. For
+Opaque LSA it is the `ospf_opaque_lsa_install_hook()`. For deletion, it is
+`ospf_opaque_lsa_delete_hook()`.
+
+Note that incoming LSA which is already present in the LSDB will be inserted
+after the old instance of this LSA remove from the LSDB. Thus, after the first
+time, each incoming LSA will trigger a `delete` following by an `install`. This
+is not very helpfull to handle real LSA deletion. In fact, LSA deletion is done
+by Flushing LSA i.e. flood LSA after seting its age to MAX_AGE. Then, a garbage
+function has the role to remove all LSA with `age == MAX_AGE` in the LSDB. So,
+to handle LSA Flush, the best is to look to the LSA age to determine if it is
+an installation or a future deletion i.e. the flushed LSA is first store in the
+LSDB with MAX_AGE waiting for the garbage collector function.
+
+Router Information LSAs
+^^^^^^^^^^^^^^^^^^^^^^^
+
+To activate Segment Routing, new CLI command `segment-routing on` has been
+introduced. When this command is activated, function
+`ospf_router_info_update_sr()` is called to indicate to Router Information
+process that Segment Routing TLVs must be flood. Same function is called to
+modify the Segment Routing Global Block (SRGB) and Maximum Stack Depth (MSD)
+TLV. Only Shortest Path First (SPF) Algorithm is supported, so no possiblity
+to modify this TLV is offer by the code.
+
+When Opaque LSA Tyep 4 i.e. Router Information are stored in LSDB, function
+`ospf_opaque_lsa_install_hook()` will call the previously registered function
+`ospf_router_info_lsa_update()`. In turn, the function will simply trigger
+`ospf_sr_ri_lsa_update()` or `ospf_sr_ri_lsa_delete` in function of the LSA
+age. Before, it verifies that the LSA Opaque Type is 4 (Router Information).
+Self Opaque LSA are not send back to the Segment Routing functions as
+information are already stored.
+
+Extended Link Prefix LSAs
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Like for Router Information, Segment Routing is activate at the Extended
+Link/Prefix level with new `segment-routing on` command. This trigger
+automtically the flooding of Extended Link LSA for all ospf interface where
+adjacency is full. For Extended Prefix LSA, the new CLI command
+`segment-routing prefix ...` will trigger the flooding of Prefix SID
+TLV/SubTLVs.
+
+When Opaque LSA Type 7 i.e. Extended Prefix and Type 8 i.e. Extended Link are
+store in the LSDB, `ospf_ext_pref_update_lsa()` respectively
+`ospf_ext_link_update_lsa()` are called like for Router Information LSA. In
+turn, they respectively trigger `ospf_sr_ext_prefix_lsa_update()` /
+`ospf_sr_ext_link_lsa_update()` or `ospf_sr_ext_prefix_lsa_delete()` /
+`ospf_sr_ext_link_lsa_delete()` if the LSA age is equal to MAX_AGE.
+
+Zebra
+^^^^^
+
+When a new MPLS entry or new Forwarding Equivalent Class (FEC) must be added or
+deleted in the data plane, `add_sid_nhlfe()` respectively `del_sid_nhlfe()` are
+called. Once check the validity of labels, they are send to ZEBRA layer through
+`ZEBRA_MPLS_LABELS_ADD` command, respectively `ZEBRA_MPLS_LABELS_DELETE`
+command for deletion. This is completed by a new labelled route through
+`ZEBRA_ROUTE_ADD` command, respectively `ZEBRA_ROUTE_DELETE` command.
+
+Configuration
+-------------
+
+Linux Kernel
+~~~~~~~~~~~~
+
+In order to use OSPF Segment Routing, you must setup MPLS data plane. Up to
+know, only Linux Kernel version >= 4.5 is supported.
+
+First, the MPLS modules aren't loaded by default, so you'll need to load them
+yourself:
+
+::
+
+ modprobe mpls_router
+ modprobe mpls_gso
+ modprobe mpls_iptunnel
+
+Then, you must activate MPLS on the interface you would used:
+
+::
+
+ sysctl -w net.mpls.conf.enp0s9.input=1
+ sysctl -w net.mpls.conf.lo.input=1
+ sysctl -w net.mpls.platform_labels=1048575
+
+The last line fix the maximum MPLS label value.
+
+Once OSPFd start with Segment Routing, you could check that MPLS routes are
+enable with:
+
+::
+
+ ip -M route
+ ip route
+
+The first command show the MPLS LFIB table while the second show the FIB
+table which contains route with MPLS label encapsulation.
+
+If you disable Penultimate Hop Popping with the `no-php-flag` (see below), you
+MUST check that RP filter is not enable for the interface you intend to use,
+especially the `lo` one. For that purpose, disable RP filtering with:
+
+::
+
+ systcl -w net.ipv4.conf.all.rp_filter=0
+ sysctl -w net.ipv4.conf.lo.rp_filter=0
+
+OSPFd
+~~~~~
+
+Here it is a simple example of configuration to enable Segment Routing. Note
+that `opaque capability` and `router information` must be set to activate
+Opaque LSA prior to Segment
+Routing.
+
+::
+
+ router ospf
+ ospf router-id 192.168.1.11
+ capability opaque
+ mpls-te on
+ mpls-te router-address 192.168.1.11
+ router-info area 0.0.0.0
+ segment-routing on
+ segment-routing global-block 10000 19999
+ segment-routing node-msd 8
+ segment-routing prefix 192.168.1.11/32 index 1100
+
+The first segment-routing statement enable it. The Second one set the SRGB,
+third line the MSD and finally, set the Prefix SID index for a given prefix.
+Note that only prefix of Loopback interface could be configured with a Prefix
+SID. It is possible to add `no-php-flag` at the end of the prefix command to
+disbale Penultimate Hop Popping. This advertises peers that they MUST NOT pop
+the MPLS label prior to sending the packet.
+
+Known limitations
+-----------------
+
+* Runs only within default VRF
+* Only single Area is supported. ABR is not yet supported
+* Only SPF algorithm is supported
+* Extended Prefix Range is not supported
+* MPLS table are not flush at startup. Thus, restarting zebra process is
+ mandatory to remove old MPLS entries in the data plane after a crash of
+ ospfd daemon
+* With NO Penultimate Hop Popping, it is not possible to express a Segment
+ Path with an Adjacency SID due to the impossibility for the Linux Kernel to
+ perform double POP instruction.
+
+Credits
+-------
+
+* Author: Anselme Sawadogo <anselmesawadogo@gmail.com>
+* Author: Olivier Dugeon <olivier.dugeon@orange.com>
+* Copyright (C) 2016 - 2018 Orange Labs http://www.orange.com
+
+This work has been performed in the framework of the H2020-ICT-2014
+project 5GEx (Grant Agreement no. 671636), which is partially funded
+by the European Commission.
+
diff --git a/doc/developer/workflow.rst b/doc/developer/workflow.rst
new file mode 100644
index 0000000000..e1101fd76a
--- /dev/null
+++ b/doc/developer/workflow.rst
@@ -0,0 +1,657 @@
+.. _process-and-workflow:
+
+*******************
+Process & Workflow
+*******************
+
+FRR is a large project developed by many different groups. This section
+documents standards for code style & quality, commit messages, pull requests
+and best practices that all contributors are asked to follow.
+
+This chapter is "descriptive/post-factual" in that it documents pratices that
+are in use; it is not "definitive/pre-factual" in prescribing practices. This
+means that when a procedure changes, it is agreed upon, then put into practice,
+and then documented here. If this document doesn't match reality, it's the
+document that needs to be updated, not reality.
+
+Mailing Lists
+=============
+
+The FRR development group maintains multiple mailing lists for use by the
+community. Italicized lists are private.
+
++----------------------------------+--------------------------------+
+| Topic | List |
++==================================+================================+
+| Development | dev@lists.frrouting.org |
++----------------------------------+--------------------------------+
+| Users & Operators | frog@lists.frrouting.org |
++----------------------------------+--------------------------------+
+| Announcements | announce@lists.frrouting.org |
++----------------------------------+--------------------------------+
+| *Security* | security@lists.frrouting.org |
++----------------------------------+--------------------------------+
+| *Technical Steering Committee* | tsc@lists.frrouting.org |
++----------------------------------+--------------------------------+
+
+The Development list is used to discuss and document general issues related to
+project development and governance. The public Slack instance,
+frrouting.slack.com, and weekly technical meetings provide a higher bandwidth
+channel for discussions. The results of such discussions must be reflected in
+updates, as appropriate, to code (i.e., merges), `Github issues`_, and for
+governance or process changes, updates to the Development list and either this
+file or information posted at https://frrouting.org/.
+
+
+Changelog
+=========
+
+The changelog will be the base for the release notes. A changelog entry for
+your changes is usually not required and will be added based on your commit
+messages by the maintainers. However, you are free to include an update to the
+changelog with some better description.
+
+Submitting Patches and Enhancements
+===================================
+
+FRR accepts patches from two sources:
+
+- Email (git format-patch)
+- Github pull request
+
+Contributors are highly encouraged to use Github's fork-and-pr workflow. It is
+easier for us to review it, test it, try it and discuss it on Github than it is
+via email, thus your patch will get more attention more quickly on Github.
+
+The base branch for new contributions and non-critical bug fixes should be
+``master``. Please ensure your pull request is based on this branch when you
+submit it.
+
+Pre-submission Checklist
+------------------------
+
+- Format code (see `Code Formatting <#developers-guidelines>`__)
+- Verify and acknowledge license (see `License for
+ contributions <#license-for-contributions>`__)
+- Ensure you have properly signed off (see `Signing
+ Off <#signing-off>`__)
+- Test building with various configurations:
+
+ - ``buildtest.sh``
+
+- Verify building source distribution:
+
+ - ``make dist`` (and try rebuilding from the resulting tar file)
+
+- Run unit tests:
+
+ - ``make test``
+
+- Document Regression Runs and plans for continued maintenance of the
+ feature
+
+License for contributions
+-------------------------
+
+FRR is under a “GPLv2 or later” license. Any code submitted must
+be released under the same license (preferred) or any license which
+allows redistribution under this GPLv2 license (eg MIT License).
+
+Signing Off
+-----------
+
+Code submitted to FRR must be signed off. We have the same
+requirements for using the signed-off-by process as the Linux kernel. In
+short, you must include a signed-off-by tag in every patch.
+
+``Signed-off-by:`` this is a developer's certification that he or she
+has the right to submit the patch for inclusion into the project. It is
+an agreement to the Developer's Certificate of Origin (below). Code
+without a proper signoff can not and will not be merged.
+
+If you are unfamiliar with this process, you should read the `official
+policy at
+kernel.org <https://www.kernel.org/doc/html/latest/process/submitting-patches.html>`__
+and you might find this article about `participating in the Linux
+community on the Linux Foundation
+website <http://www.linuxfoundation.org/content/how-participate-linux-community-0>`__
+to be a helpful resource.
+
+In short, when you sign off on a commit, you assert your agreement to
+all of the following:
+
+::
+
+ Developer's Certificate of Origin 1.1
+
+ By making a contribution to this project, I certify that:
+
+ (a) The contribution was created in whole or in part by me and I
+ have the right to submit it under the open source license
+ indicated in the file; or
+
+ (b) The contribution is based upon previous work that, to the best
+ of my knowledge, is covered under an appropriate open source
+ license and I have the right under that license to submit that
+ work with modifications, whether created in whole or in part by
+ me, under the same open source license (unless I am permitted to
+ submit under a different license), as indicated in the file; or
+
+ (c) The contribution was provided directly to me by some other
+ person who certified (a), (b) or (c) and I have not modified it.
+
+ (d) I understand and agree that this project and the contribution
+ are public and that a record of the contribution (including all
+ personal information I submit with it, including my sign-off) is
+ maintained indefinitely and may be redistributed consistent with
+ this project or the open source license(s) involved.
+
+What do I submit my changes against?
+------------------------------------
+
+We've documented where we would like to have the different fixes applied
+at
+https://github.com/FRR/frr/wiki/Where-Do-I-create-a-Pull-Request-against%3F
+If you are unsure where your submission goes, look at that document or
+ask a project maintainer.
+
+Github pull requests
+--------------------
+
+The preferred method of submitting changes is a Github pull request.
+Code submitted by pull request will be automatically tested by one or
+more CI systems. Once the automated tests succeed, other developers will
+review your code for quality and correctness. After any concerns are
+resolved, your code will be merged into the branch it was submitted
+against.
+
+Patch submission via mailing list
+---------------------------------
+
+As an alternative submission method, a patch can be mailed to the
+development mailing list. Patches received on the mailing list will be
+picked up by Patchwork and tested against the latest development branch.
+
+The recommended way to send the patch (or series of NN patches) to the
+list is by using ``git send-email`` as follows (assuming they are the N
+most recent commit(s) in your git history:
+
+::
+
+ git send-email -NN --annotate --to=dev@lists.frrouting.org
+
+If your commits do not already contain a ``Signed-off-by`` line, then
+use the following command to add it (after making sure you agree to the
+Developer Certificate of Origin as outlined above):
+
+::
+
+ git send-email -NN --annotate --signoff --to=dev@lists.frrouting.org
+
+Submitting multi-commit patches as a Github pull request is **strongly
+encouraged** and increases the probability of your patch getting
+reviewed and merged in a timely manner.
+
+After submitting your changes
+-----------------------------
+
+- Watch for Continuous Integration (CI) Test results
+
+ - You should automatically receive an email with the test results
+ within less than 2 hrs of the submission. If you don’t get the
+ email, then check status on the Github pull request.
+ - Please notify the development mailing list if you think something
+ doesn't work.
+
+- If the tests failed:
+
+ - In general, expect the community to ignore the submission until
+ the tests pass.
+ - It is up to you to fix and resubmit.
+
+ - This includes fixing existing unit (“make test”) tests if your
+ changes broke or changed them.
+ - It also includes fixing distribution packages for the failing
+ platforms (ie if new libraries are required).
+ - Feel free to ask for help on the development list.
+
+ - Go back to the submission process and repeat until the tests pass.
+
+- If the tests pass:
+
+ - Wait for reviewers. Someone will review your code or be assigned
+ to review your code.
+ - Respond to any comments or concerns the reviewer has.
+ - After all comments and concerns are addressed, expect your patch
+ to be merged.
+
+- Watch out for questions on the mailing list. At this time there will
+ be a manual code review and further (longer) tests by various
+ community members.
+- Your submission is done once it is merged to the master branch.
+
+Git Structure
+=============
+
+.. figure:: ../figures/git_branches.png
+ :align: center
+ :scale: 55%
+ :alt: Merging Git branches into a central trunk
+
+ Rough outline of FRR development workflow
+
+The master Git for FRR resides on `GitHub`_.
+
+There is one main branch for development, ``master``. For each major release
+(2.0, 3.0 etc) a new release branch is created based on the master. Subsequent
+point releases based on a major branch are marked by tagging.
+
+Programming Languages, Tools and Libraries
+==========================================
+
+The core of FRR is written in C (gcc or clang supported) and makes
+use of GNU compiler extensions. A few non-essential scripts are
+implemented in Perl and Python. FRR requires the following tools
+to build distribution packages: automake, autoconf, texinfo, libtool and
+gawk and various libraries (i.e. libpam and libjson-c).
+
+If your contribution requires a new library or other tool, then please
+highlight this in your description of the change. Also make sure it’s
+supported by all FRR platform OSes or provide a way to build
+without the library (potentially without the new feature) on the other
+platforms.
+
+Documentation should be written in reStructuredText. Sphinx extensions may be
+utilized but pure ReST is preferred where possible. See
+:ref:`documentation`.
+
+
+Coding Practices & Style
+========================
+
+Commit messages
+---------------
+
+Commit messages should be formatted in the same way as Linux kernel
+commit messages. The format is roughly
+
+::
+
+ dir: short summary
+
+ extended summary
+
+``dir`` should be the top level source directory under which the change
+was made. For example, a change in bgpd/rfapi would be formatted as:::
+
+ bgpd: short summary
+
+The first line should be no longer than 50 characters. Subsequent lines
+should be wrapped to 72 characters.
+
+Source file header
+------------------
+
+New files need to have a Copyright header (see `License for
+contributions <#license-for-contributions>`__ above) added to the file.
+Preferred form of the header is as follows:
+
+::
+
+ /*
+ * Title/Function of file
+ * Copyright (C) YEAR Author’s Name
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the Free
+ * Software Foundation; either version 2 of the License, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; see the file COPYING; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+ #include <zebra.h>
+
+Adding copyright claims to existing files
+-----------------------------------------
+
+When adding copyright claims for modifications to an existing file,
+please preface the claim with "Portions: " on a line before it and
+indent the "Copyright ..." string. If such a case already exists, add
+your indented claim immediately after. E.g.:
+
+::
+
+ Portions:
+ Copyright (C) 2010 Entity A ....
+ Copyright (C) 2016 Your name [optional brief change description]
+
+Code formatting
+---------------
+
+FRR uses Linux kernel style except where noted below. Code which does
+not comply with these style guidelines will not be accepted.
+
+To assist with compliance, in the project root there is a .clang-format
+configuration file which can be used with the ``clang-format`` tool from
+the LLVM project. In the ``tools/`` directory there is a Python script
+named ``indent.py`` that wraps clang-format and handles some edge cases
+specific to FRR. If you are submitting a new file, it is recommended to
+run that script over the new file after ensuring that the latest stable
+release of ``clang-format`` is in your PATH.
+
+**Whitespace changes in untouched parts of the code are not acceptable
+in patches that change actual code.** To change/fix formatting issues,
+please create a separate patch that only does formatting changes and
+nothing else.
+
+Kernel and BSD styles are documented externally:
+
+- https://www.kernel.org/doc/html/latest/process/coding-style.html
+- http://man.openbsd.org/style
+
+For GNU coding style, use ``indent`` with the following invocation:
+
+::
+
+ indent -nut -nfc1 file_for_submission.c
+
+Exceptions
+^^^^^^^^^^
+
+FRR project code comes from a variety of sources, so there are some
+stylistic exceptions in place. They are organized here by branch.
+
+For ``master``
+""""""""""""""
+
+BSD coding style applies to:
+
+- ``ldpd/``
+
+``babeld`` uses, approximately, the following style:
+
+- K&R style braces
+- Indents are 4 spaces
+- Function return types are on their own line
+
+For ``stable/3.0`` and ``stable/2.0``
+"""""""""""""""""""""""""""""""""""""
+
+GNU coding style apply to the following parts:
+
+- ``lib/``
+- ``zebra/``
+- ``bgpd/``
+- ``ospfd/``
+- ``ospf6d/``
+- ``isisd/``
+- ``ripd/``
+- ``ripngd/``
+- ``vtysh/``
+
+BSD coding style applies to:
+
+- ``ldpd/``
+
+Compile-time conditional code
+-----------------------------
+
+Many users access FRR via binary packages from 3rd party sources;
+compile-time code puts inclusion/exclusion in the hands of the package
+maintainer. Please think very carefully before making code conditional
+at compile time, as it increases regression testing, maintenance
+burdens, and user confusion. In particular, please avoid gratuitous
+``--enable-…`` switches to the configure script - in general, code
+should be of high quality and in working condition, or it shouldn’t be
+in FRR at all.
+
+When code must be compile-time conditional, try have the compiler make
+it conditional rather than the C pre-processor so that it will still be
+checked by the compiler, even if disabled. For example,
+
+::
+
+ if (SOME_SYMBOL)
+ frobnicate();
+
+is preferred to
+
+::
+
+ #ifdef SOME_SYMBOL
+ frobnicate ();
+ #endif /* SOME_SYMBOL */
+
+Note that the former approach requires ensuring that ``SOME_SYMBOL``
+will be defined (watch your ``AC_DEFINE``\ s).
+
+Debug-guards in code
+--------------------
+
+Debugging statements are an important methodology to allow developers to
+fix issues found in the code after it has been released. The caveat here
+is that the developer must remember that people will be using the code
+at scale and in ways that can be unexpected for the original
+implementor. As such debugs **MUST** be guarded in such a way that they
+can be turned off. FRR has the ability to turn on/off debugs from the
+CLI and it is expected that the developer will use this convention to
+allow control of their debugs.
+
+CLI changes
+-----------
+
+CLI's are a complicated ugly beast. Additions or changes to the CLI
+should use a DEFUN to encapsulate one setting as much as is possible.
+Additionally as new DEFUN's are added to the system, documentation
+should be provided for the new commands.
+
+Backwards Compatibility
+-----------------------
+
+As a general principle, changes to CLI and code in the lib/ directory
+should be made in a backwards compatible fashion. This means that
+changes that are purely stylistic in nature should be avoided, e.g.,
+renaming an existing macro or library function name without any
+functional change. When adding new parameters to common functions, it is
+also good to consider if this too should be done in a backward
+compatible fashion, e.g., by preserving the old form in addition to
+adding the new form.
+
+This is not to say that minor or even major functional changes to CLI
+and common code should be avoided, but rather that the benefit gained
+from a change should be weighed against the added cost/complexity to
+existing code. Also, that when making such changes, it is good to
+preserve compatibility when possible to do so without introducing
+maintenance overhead/cost. It is also important to keep in mind,
+existing code includes code that may reside in private repositories (and
+is yet to be submitted) or code that has yet to be migrated from Quagga
+to FRR.
+
+That said, compatibility measures can (and should) be removed when
+either:
+
+- they become a significant burden, e.g. when data structures change
+ and the compatibility measure would need a complex adaptation layer
+ or becomes flat-out impossible
+- some measure of time (dependent on the specific case) has passed, so
+ that the compatibility grace period is considered expired.
+
+In all cases, compatibility pieces should be marked with
+compiler/preprocessor annotations to print warnings at compile time,
+pointing to the appropriate update path. A ``-Werror`` build should fail
+if compatibility bits are used.
+
+Miscellaneous
+-------------
+
+When in doubt, follow the guidelines in the Linux kernel style guide, or
+ask on the development mailing list / public Slack instance.
+
+
+.. _documentation:
+
+Documentation
+=============
+
+FRR uses Sphinx+RST as its documentation system. The document you are currently
+reading was generated by Sphinx from RST source in
+:file:`doc/developer/workflow.rst`. The documentation is structured as follows:
+
++-----------------------+--------------------------------------------------------------+
+| Directory | Contents |
++=======================+==============================================================+
+| :file:`doc/user` | User documentation; configuration guides; protocol overviews |
++-----------------------+--------------------------------------------------------------+
+| :file:`doc/developer` | Developer's documentation; API specs; datastructures; |
+| | architecture overviews; project management procedure |
++-----------------------+--------------------------------------------------------------+
+| :file:`doc/manpages` | Source for manpages |
++-----------------------+--------------------------------------------------------------+
+| :file:`doc/figures` | Images and diagrams |
++-----------------------+--------------------------------------------------------------+
+
+Each of these directories, with the exception of :file:`doc/figures`, contains
+a Sphinx-generated Makefile and configuration script :file:`conf.py` used to
+set various document parameters. The makefile can be used for a variety of
+targets; invoke `make help` in any of these directories for a listing of
+available output formats. For convenience, there is a top-level
+:file:`Makefile.am` that has targets for PDF and HTML documentation for both
+developer and user documentation, respectively. That makefile is also
+responsible for building manual pages packed with distribution builds.
+
+Indent and styling should follow existing conventions:
+
+- 3 spaces for indents under directives
+- Cross references may contain only lowercase alphanumeric characters and
+ hyphens ('-')
+- Lines wrapped to 80 characters where possible
+
+Characters for header levels should follow Python documentation guide:
+
+- ``#`` with overline, for parts
+- ``*`` with overline, for chapters
+- ``=``, for sections
+- ``-``, for subsections
+- ``^``, for subsubsections
+- ``"``, for paragraphs
+
+After you have made your changes, please make sure that you can invoke
+``make latexpdf`` and ``make html`` with no warnings.
+
+The documentation is currently incomplete and needs love. If you find a broken
+cross-reference, figure, dead hyperlink, style issue or any other nastiness we
+gladly accept documentation patches.
+
+To build the docs, please ensure you have installed a recent version of
+`Sphinx <http://www.sphinx-doc.org/en/stable/install.html>`_. If you want to
+build LaTeX or PDF docs, you will also need a full LaTeX distribution
+installed.
+
+Code
+----
+
+FRR is a large and complex software project developed by many different people
+over a long period of time. Without adequate documentation, it can be
+exceedingly difficult to understand code segments, APIs and other interfaces.
+In the interest of keeping the project healthy and maintainable, you should
+make every effort to document your code so that other people can understand
+what it does without needing to closely read the code itself.
+
+Some specific guidelines that contributors should follow are:
+
+- Functions exposed in header files should have descriptive comments above
+ their signatures in the header file. At a minimum, a function comment should
+ contain information about the return value, parameters, and a general summary
+ of the function's purpose. Documentation on parameter values can be omitted
+ if it is (very) obvious what they are used for.
+
+ Function comments must follow the style for multiline comments laid out in
+ the kernel style guide.
+
+ Example:
+
+ .. code-block:: c
+
+ /*
+ * Determines whether or not a string is cool.
+ *
+ * @param text - the string to check for coolness
+ * @param is_clccfc - whether capslock is cruise control for cool
+ * @return 7 if the text is cool, 0 otherwise
+ */
+ int check_coolness(const char *text, bool is_clccfc);
+
+ The Javadoc-style annotations are not required, but you should still strive
+ to make it equally clear what parameters and return values are used for.
+
+- Static functions should have descriptive comments in the same form as above
+ if what they do is not immediately obvious. Use good engineering judgement
+ when deciding whether a comment is necessary. If you are unsure, document
+ your code.
+- Global variables, static or not, should have a comment describing their use.
+- **For new code in lib/, these guidelines are hard requirements.**
+
+If you make significant changes to portions of the codebase covered in the
+Developer's Manual, add a major subsystem or feature, or gain arcane mastery of
+some undocumented or poorly documented part of the codebase, please document
+your work so others can benefit. If you add a major feature or introduce a new
+API, please document the architecture and API to the best of your abilities in
+the Developer's Manual, using good judgement when choosing where to place it.
+
+Finally, if you come across some code that is undocumented and feel like
+going above and beyond, document it! We absolutely appreciate and accept
+patches that document previously undocumented code.
+
+User
+----
+
+If you are contributing code that adds significant user-visible functionality
+please document how to use it in :file:`doc/user`. Use good judgement when
+choosing where to place documentation. For example, instructions on how to use
+your implementation of a new BGP draft should go in the BGP chapter instead of
+being its own chapter. If you are adding a new protocol daemon, please create a
+new chapter.
+
+When documenting CLI please use a combination of the ``.. index::`` and
+``.. clicmd::`` directives. For example, the command :clicmd:`show pony` would
+be documented as follows:
+
+.. code-block:: rest
+
+ .. index:: show pony
+ .. clicmd:: show pony
+
+ Prints an ASCII pony. Example output:::
+
+ >>\.
+ /_ )`.
+ / _)`^)`. _.---. _
+ (_,' \ `^-)"" `.\
+ | | \
+ \ / |
+ / \ /.___.'\ (\ (_
+ < ,"|| \ |`. \`-'
+ \\ () )| )/
+ hjw |_>|> /_] //
+ /_] /_]
+
+When documented this way, CLI commands can be cross referenced with the
+``:clicmd:`` inline markup like so:
+
+.. code-block:: rest
+
+ :clicmd:`show pony`
+
+This is very helpful for users who want to quickly remind themselves what a
+particular command does.
+
+.. _GitHub: https://github.com/frrouting/frr
+.. _GitHub issues: https://github.com/frrouting/frr/issues