Stefan Lendl [Fri, 5 Apr 2024 13:18:04 +0000 (15:18 +0200)]
tests: test VNets functionality as a blackbox
Add several tests for Vnets in test_vnets_blackbox. State setup as well
as testing results is done only via the API to test on the API
boundaries not not against the internal state. Internal state is mocked
to avoid requiring access to system files or pmxcfs.
Mocking is done by reading and writing to a hash that holds the entire
state of SDN. The state is reset after every test run.
Testing is done via helper functions: nic_join and nic_start.
When a nic joins a Vnet, currently it always - and only - calls
add_next_free_cidr(). The same is true if a nic starts on Vnet, which
only calles add_dhcp_mapping.
These test functions homogenize the parameter list in contrast to the
current calls to the current functions. The intention for the functions
is that they can be moved to Vnets.pm to be called from QemuServer and
LXC!
The tests are composed of a test function which can be parameterized. To
call the test function, the run_test function takes the function pointer
and passes the rest of the arguments to the test functions. It also
takes care of resetting the test state.
This allows fine-grained parameterization per-test directly in the code
instead of separated files that require the entire state to be passed
in.
The tests setup the SDN by creating a simple zone and a simple vnet. The
nic_join and nic_start function is called with different subnet
configuration wiht and without a dhcp-range configured and with or
without an already present IP in the IPAM.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:18:03 +0000 (15:18 +0200)]
debian: blackbox tests depend on libpve-access-control at build
For mocking RPCEnvironment in sbuild.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:18:02 +0000 (15:18 +0200)]
sdn: api: extract function that creates the sdn directory.
create_etc_interfaces_sdn_dir creates the /etc/pve/sdn directory.
This allows mocking in tests to prevent system fs access in tests
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:18:01 +0000 (15:18 +0200)]
sdn: dnsmasq: extract function that updates dnsmasq lease via dbus
Extract the dbus based interactions with dnsmasq so that it can be
mocked in tests.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:18:00 +0000 (15:18 +0200)]
sdn: dnsmasq: extract function that generates the ethers file path
Extracted to a function so it can be mocked in tests.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:17:59 +0000 (15:17 +0200)]
dns: dnsmasq: extract function to systemctl command.
systemctl_service() is a wrapper around PVE::Tools::run_command to allow
mocking the systemctl interactions in tests.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:17:58 +0000 (15:17 +0200)]
sdn: zones: extract function that reads datacenter config
The datacenter_config() functions in SDN::Zones::Plugin is a simple
wrapper that reads datacenter.cfg via cfs.
This allows mocking datacenter.cfg in tests.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Hanreich [Fri, 5 Apr 2024 13:17:57 +0000 (15:17 +0200)]
sdn: dhcp: rollback allocated ips on failure
If DHCP is configured for IPv4 and IPv6, failing to obtain an IPv6 IP
does not roll back the allocation made for IPv4. This patch rolls back
any changes made in case of failure, so that IP allocation is actually
atomic.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com> Reviewed-by: Stefan Lendl <s.lendl@proxmox.com> Tested-by: Stefan Lendl <s.lendl@proxmox.com> Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Stefan Hanreich [Fri, 5 Apr 2024 13:17:56 +0000 (15:17 +0200)]
sdn: dhcp: only consider subnets that have dhcp-range configured
If DHCP is enabled on a zone with subnets, but no subnet has a
dhcp-range configured, then starting a VM will fail because no IP can
be allocated. This patch fixes this by only considering subnets that
have a dhcp-range configured and only failing if there is at least one
subnet with a dhcp-range configured.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com> Reviewed-by: Stefan Lendl <s.lendl@proxmox.com> Tested-by: Stefan Lendl <s.lendl@proxmox.com> Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:17:55 +0000 (15:17 +0200)]
sdn: dhcp: request both IPv4 and IPv6 addresses on VM start
If previously an IP was allocated in the IPAM, but a new subnet added
for the other IP version, we need to allocate an IP in the new subnet.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Fri, 5 Apr 2024 13:17:54 +0000 (15:17 +0200)]
sdn: dhcp: get next free ip for a specific IP version
Specify the IP version (4|6) for which an IP shall be requested from the IPAM.
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Stefan Hanreich <s.hanreich@proxmox.com> Tested-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Tue, 2 Apr 2024 11:07:41 +0000 (13:07 +0200)]
tests: run tests in sbuild
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Tue, 2 Apr 2024 11:07:40 +0000 (13:07 +0200)]
tests: mocking more functions to avoid system access
previously extracted functions are now mocked in the zone tests
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Tue, 2 Apr 2024 11:07:39 +0000 (13:07 +0200)]
evpn: extract read_local_frr_config
to allow mocking local fs access
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Lendl [Tue, 2 Apr 2024 11:07:38 +0000 (13:07 +0200)]
controllers: extract read_etc_network_interfaces
to allow mocking local fs access
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com> Reviewed-by: Max Carrara <m.carrara@proxmox.com> Tested-by: Max Carrara <m.carrara@proxmox.com> Tested-By: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Hanreich [Mon, 4 Mar 2024 14:11:28 +0000 (15:11 +0100)]
fix #5275: remove priority field from powerdns
Since v3.4.2 the priority field has been removed and since v4.9 they
are actively rejected by PowerDNS. Stop sending this field in order to
make the PowerDNS plugin work with versions >= 4.9 again. [1]
sdn: prefer proxy from datacenter.cfg for api calls
We only setup proxies from the environment previously, but also check
the one configured in the cluster-wide datacenter.cfg and prefer that
over anything else.
controllers: evpn: bugfix: use prefix-list in route-map instead evpn match
"match evpn" in route-map is broken since 8.5.0
https://github.com/FRRouting/frr/issues/14419
the patch
https://github.com/FRRouting/frr/commit/272c6d5db128ff7450fe9fcd16c046160594deb3
is converting type-2 && type-5 evpn routes to prefix-prefix.
(fixing prefix-list not working previously, but breaking "match evpn")
So, simply use prefix-list now, as "match epvn" was a workaround anyway.
reported on the forum, where user have routing loop between the 2 exit-nodes:
https://forum.proxmox.com/threads/sdn-with-evpn-controller-routing-loop-when-using-multiple-exit-nodes.137362/
Thomas Lamprecht [Mon, 22 Jan 2024 10:51:42 +0000 (11:51 +0100)]
buildsys: add explicit default target
Currently the dinstall was the default target due to being the first
one listed, that's rather unusual compared to all of our other repos,
so align this one to our unwritten norm of using the `deb` one as
default target.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht [Wed, 29 Nov 2023 10:37:05 +0000 (11:37 +0100)]
controllers: fix interfacing with read_etc_network_interfaces
While not used currently in the parser, passing 1 as file name still
makes no sense, and we might want to rework that in general, as why
does this have to use the non-inotify read path and manually open the
file.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Hanreich [Tue, 28 Nov 2023 08:58:57 +0000 (09:58 +0100)]
dhcp: dnsmasq: untaint when deleting configuration files
The current invocation is quite unsafe and triggers the taint mode of
Perl that is enabled for our API daemons, but not pvesh used on
cluster-wide apply.
Replacing it with dir_glob_foreach solves those issues.
Reported-By: Friedrich Weber <f.weber@proxmox.com> Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com> Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
To
- start after networking.service (in order to make sure ifupdown has
created all the interfaces before dnsmasq tries to find them via the
'interfaces=' lines.
- drop the 'Requires=network.target' since it is not a *provider* of
that target
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Stefan Hanreich [Wed, 22 Nov 2023 12:29:50 +0000 (13:29 +0100)]
ipam: improve update / delete behavior
Currently when updating or deleting a mapping in the IPAM we would
delete all existing entries in the IPAM with that mac address. Now we
only delete the specific entry we are updating / deleting.
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
- don't listen to ip address, but use interface= instead
- generate 1 config file by vnet instead 1 by subnet
- enable-ra is global to server, enable it in default conf
api: take partial configs for PUT /cluster/sdn/zones/<id>
Zones previously expected a complete config, but the API schema
also contains a 'delete' parameter via the SectionConfig's
updateSchema() helper. This was not handled, and instead failed to
validate as part of the config.
The same is true for vnets and subnets, while ipams, dns and
controller entries followed our usual update procedures (but also
ignored the 'delete' parameter).
Since all of our SectionConfig based API endpoints are supposed to
take changes, rather than complete configs, this changes these
endpoints to not replace the full configuration anymore.
This is a major break for automation tools (the web UI already passed
the full config each time).
Cc: Alexandre Derumier <aderumier@odiso.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Thomas Lamprecht [Tue, 21 Nov 2023 07:26:56 +0000 (08:26 +0100)]
d/control: downgrade frr-pythontools from recommends to suggests
avoid pulling FRR in on upgrade if we'd hard-depend on
libpve-network-perl for all those systems that kept the default
install-recommends enabled as FRR is a bit involved and might announce
stuff and break some network environments if not correctly configured.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Stefan Hanreich [Mon, 20 Nov 2023 16:28:32 +0000 (17:28 +0100)]
api: refactor URL structure for Ipam
The initial URL structure was less than optimal due to Ipam as well as
Ipams being endpoints in the API, which are too similar and might be
confusing to users.
Move the listing of PVE IPAM to /ipams/pve/status
Move the create / update / delete endpoints to /vnets/{vnetid}/ips
Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
Stefan Hanreich [Fri, 17 Nov 2023 11:39:46 +0000 (12:39 +0100)]
sdn: dhcp: add dnsmasq plugin
The plugin creates a dnsmasq@<zone> service that spawns a dnsmasq
instance that handles dhcp for that zone.
The configuration files for a dnsmasq instance lie within
/etc/dnsmasq.d/<zone>
The plugin generates the following configuration files:
* 00-default.conf
Contains the default global configuration for dnsmasq. Disables DNS,
enables some specific options for Windows, does some
security-related configuration and makes dnsmasq bind only to the
interfaces it is responsible for
* 10-<subnet>.conf
Contains the default settings for a subnet. Sets dhcp options for
DNS and gateway.
* 10-<subnet>.ranges.conf
Contains the configuration of DHCP ranges for a subnet.
* ethers
Contains the MAC address to IP mappings for a zone.
Every subnet gets assigned a tag in the dnsmasq configuration that is
equal to the subnet_id. This can be used to override / set additional
configuration options on a per-subnet basis.
Additionally it creates the file /etc/default/dnsmasq.<zone> that
provides default options for the dnsmasq service.
Leases are stored in /var/lib/misc/dnsmasq.<zone>.
Co-Authored-By: Alexandre Derumier <aderumier@odiso.com> Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>