Commit Graph

3291 Commits

Author SHA1 Message Date
Joshua Boniface 4e2a1c3e52 Add worker wrapper to fix Deb incompatibility
Celery 5.x introduced a new worker argument format that is not
backwards-compatible with the older Celery 4.x format. This created a
conundrum since we use one service unit for both Debian 10 (4.x) and
Debian 11 (5.x). Instead of worse hacks, create a wrapper script to
start the worker with the correct arguments instead.
2021-06-28 12:19:29 -04:00
Joshua Boniface dbfa339cfb Ensure postinst and prerm always succeed 2021-06-23 20:35:40 -04:00
Joshua Boniface c54f66efa8 Limit match only on VM name
I can see no possible reason to want to do limits against UUIDs, but
supporting that means match is not what one would expect since a random
UUID could match the limit. So only limit based on the name.
2021-06-23 19:17:35 -04:00
Joshua Boniface cd860bae6b Optimize VM list in API
With many VMs this slows down linearly. Rework it a bit so there are
fewer calls to getInformationFromXML and so the processing could happen
in parallel at some point.
2021-06-23 19:14:26 -04:00
Joshua Boniface bbb132414c Restore shebang and don't do store if completion 2021-06-23 05:26:50 -04:00
Joshua Boniface 04fa63f081 Only hit the network endpoint once
Otherwise this is hit for every VM which gets very slow very fast.
2021-06-23 05:15:48 -04:00
Joshua Boniface f248d579df Convert pvc-client-cli into a proper Python module
Also fixes up the Debian packaging such that this works how I would
want, with proper module installation while leaving everything else
untouched. Finally implements automatic installation and removal of the
BASH completion for the PVC command.
2021-06-23 05:03:19 -04:00
Joshua Boniface f0db631947 Ignore root-level venv for testing 2021-06-23 02:15:49 -04:00
Joshua Boniface e91a597591 Merge branch 'sriov'
Implement SR-IOV support in PVC.

Closes #130
2021-06-23 00:58:44 -04:00
Joshua Boniface 8d21da9041 Add some additional interaction tests 2021-06-22 22:08:51 -04:00
Joshua Boniface 1ae34c1960 Fix bad messages in volume remove 2021-06-22 04:31:02 -04:00
Joshua Boniface 75f2560217 Add documentation on SR-IOV client networks 2021-06-22 04:20:38 -04:00
Joshua Boniface 7d2b7441c2 Mention SR-IOV in the Daemon and Ansible manuals 2021-06-22 03:55:19 -04:00
Joshua Boniface 5ec198bf98 Update API doc with remaining items 2021-06-22 03:47:27 -04:00
Joshua Boniface e6b26745ce Adjust some help messages in pvc.py 2021-06-22 03:40:21 -04:00
Joshua Boniface 3490ecbb59 Remove explicit ZK address from Patronictl command 2021-06-22 03:31:06 -04:00
Joshua Boniface 2928d695c9 Ensure migration method is updated on state changes 2021-06-22 03:20:15 -04:00
Joshua Boniface 7d2a3b5361 Ensure Macvtap NICs can use a model
Defaults to virtio like a bridged NIC. Otherwise performance is abysmal.
2021-06-22 02:38:16 -04:00
Joshua Boniface 07dbd55f03 Use list comprehension to compare against source 2021-06-22 02:31:14 -04:00
Joshua Boniface 26dd24e3f5 Ensure MTU is set on VF when starting up 2021-06-22 02:26:14 -04:00
Joshua Boniface 6cd0ccf0ad Fix network check on VM config modification 2021-06-22 02:21:55 -04:00
Joshua Boniface 1787a970ab Fix bug in address check format string 2021-06-22 02:21:32 -04:00
Joshua Boniface e623909a43 Store PHY MAC for VFs and restore after free 2021-06-22 00:56:47 -04:00
Joshua Boniface 60e1da09dd Don't try any shenannegans when updating NICs
Trying to do this on the VMInstance side had problems because we can't
differentiate the 3 types of migration there. So, just update this in
the API side and hope everything goes well.

This introduces an edge bug: if a VM is using a macvtap SR-IOV device,
and then tries to migrate, and the migrate is aborted, the NIC lists
will be inconsistent.

When I revamp the VMInstance in the future, I should be able to correct
this, but for now we'll have to live with that edgecase.
2021-06-22 00:00:50 -04:00
Joshua Boniface dc560c1dcb Better handle retcodes in migrate update 2021-06-21 23:46:47 -04:00
Joshua Boniface 68c7481aa2 Ensure offline migrations update SR-IOV NIC states 2021-06-21 23:35:52 -04:00
Joshua Boniface 7d42fba373 Ensure being in migrate doesn't abort shutdown 2021-06-21 23:28:53 -04:00
Joshua Boniface b532bc9104 Add missing managed flag for hostdev 2021-06-21 23:22:36 -04:00
Joshua Boniface 24ce361a04 Ensure SR-IOV NIC states are updated on migration 2021-06-21 23:18:34 -04:00
Joshua Boniface eeb83da97d Add support for SR-IOV NICs to VMs 2021-06-21 23:18:22 -04:00
Joshua Boniface 93c2fdec93 Swap order of networks and disks in provisioner
Done to make the resulting config match the expectations when using "vm
network add", which is that networks are below disks, not above.

Not a functional change, just ensures the VM XML is consistent after
many changes.
2021-06-21 21:59:57 -04:00
Joshua Boniface 904337b677 Fix busted changelog from previous commit 2021-06-21 21:19:06 -04:00
Joshua Boniface 64d1a37b3c Add PCIe device paths to SR-IOV VF information
This will be used when adding VM network interfaces of type hostdev.
2021-06-21 21:08:46 -04:00
Joshua Boniface 13cc0f986f Implement SR-IOV VF config set
Also fixes some random bugs, adds proper interface sorting, and assorted
tweaks.
2021-06-21 18:40:11 -04:00
Joshua Boniface e13baf8bd3 Add initial SR-IOV list/info to CLI 2021-06-21 17:12:53 -04:00
Joshua Boniface ae480d6cc1 Add SR-IOV listing/info endpoints to API 2021-06-21 17:12:45 -04:00
Joshua Boniface 33195c3c29 Ensure VF list is sorted 2021-06-21 17:11:48 -04:00
Joshua Boniface a697c2db2e Add SRIOV PF and VF listing to API 2021-06-21 01:42:55 -04:00
Joshua Boniface ca11dbf491 Sort the list of VFs for easier parsing 2021-06-21 01:40:05 -04:00
Joshua Boniface e8bd1bf2c4 Ensure used/used_by are set on creation 2021-06-21 01:25:38 -04:00
Joshua Boniface bff6d71e18 Add logging to SRIOVVFInstance and fix bug 2021-06-17 02:02:41 -04:00
Joshua Boniface 57b041dc62 Ensure default for vLAN and QOS is 0 not empty 2021-06-17 01:54:37 -04:00
Joshua Boniface 509afd4d05 Add hostdev net_type to handler as well 2021-06-17 01:52:58 -04:00
Joshua Boniface 5607a6bb62 Avoid overwriting VF data
Ensures that the configuration of a VF is not overwritten in Zookeeper
on a node restart. The SRIOVVFInstance handlers were modified to start
with None values, so that the DataWatch statements will always trigger
updates to the live system interfaces on daemon startup, thus ensuring
that the config stored in Zookeeper is applied to the system on startup
(mostly relevant after a cold boot or if the API changes them during a
daemon restart).
2021-06-17 01:45:22 -04:00
Joshua Boniface 8f1af2a642 Ignore hostdev interfaces in VM net stat gathering
Prevents errors if a SR-IOV hostdev interface is configured until this
is more defined.
2021-06-17 01:33:11 -04:00
Joshua Boniface e7b6a3eac1 Implement SR-IOV PF and VF instances
Adds support for the node daemon managing SR-IOV PF and VF instances.

PFs are added to Zookeeper automatically based on the config at startup
during network configuration, and are otherwise completely static. PFs
are automatically removed from Zookeeper, along with all coresponding
VFs, should the PF phy device be removed from the configuration.

VFs are configured based on the (autocreated) VFs of each PF device,
added to Zookeeper, and then a new class instance, SRIOVVFInstance, is
used to watch them for configuration changes. This will enable the
runtime management of VF settings by the API. The set of keys ensures
that both configuration and details of the NIC can be tracked.

Most keys are self-explanatory, especially for PFs and the basic keys
for VFs. The configuration tree is also self-explanatory, being based
entirely on the options available in the `ip link set {dev} vf` command.

Two additional keys are also present: `used` and `used_by`, which will
be able to track the (boolean) state of usage, as well as the VM that
uses a given VIF. Since the VM side implementation will support both
macvtap and direct "hostdev" assignments, this will ensure that this
state can be tracked on both the VF and the VM side.
2021-06-17 01:33:03 -04:00
Joshua Boniface 0ad6d55dff Add initial SR-IOV support to node daemon
Adds configuration values for enabled flag and SR-IOV devices to the
configuration and sets up the initial SR-IOV configuration on daemon
startup (inserting the module, configuring the VF count, etc.).
2021-06-15 22:56:09 -04:00
Joshua Boniface eada5db5e4 Add diagram and info about invalid georedundancy 2021-06-15 10:20:42 -04:00
Joshua Boniface 164becd3ef Fix info and list matching 2021-06-15 02:32:34 -04:00
Joshua Boniface e4a65230a1 Just do the shutdown command itself 2021-06-15 02:32:14 -04:00