parallelvirtualcluster
/
pvc
1
0
Fork 0
Code Issues 11 Pull Requests Projects Releases 111 Wiki Activity
2,510 Commits 1 Branch 113 Tags 28 MiB
Commit Graph

243 Commits

Author SHA1 Message Date
Joshua Boniface f91c07fdcf Re-add UUID limit matching for full UUIDs 
This *was* valuable when passing a full UUID in, so go back to that.
Verify first that the limit string is an actual UUID, and then compare
against it if applicable.
 2021-06-28 12:27:43 -04:00
Joshua Boniface c54f66efa8 Limit match only on VM name 
I can see no possible reason to want to do limits against UUIDs, but
supporting that means match is not what one would expect since a random
UUID could match the limit. So only limit based on the name.
 2021-06-23 19:17:35 -04:00
Joshua Boniface cd860bae6b Optimize VM list in API 
With many VMs this slows down linearly. Rework it a bit so there are
fewer calls to getInformationFromXML and so the processing could happen
in parallel at some point.
 2021-06-23 19:14:26 -04:00
Joshua Boniface 07dbd55f03 Use list comprehension to compare against source 2021-06-22 02:31:14 -04:00
Joshua Boniface 6cd0ccf0ad Fix network check on VM config modification 2021-06-22 02:21:55 -04:00
Joshua Boniface e623909a43 Store PHY MAC for VFs and restore after free 2021-06-22 00:56:47 -04:00
Joshua Boniface 60e1da09dd Don't try any shenannegans when updating NICs 
Trying to do this on the VMInstance side had problems because we can't
differentiate the 3 types of migration there. So, just update this in
the API side and hope everything goes well.

This introduces an edge bug: if a VM is using a macvtap SR-IOV device,
and then tries to migrate, and the migrate is aborted, the NIC lists
will be inconsistent.

When I revamp the VMInstance in the future, I should be able to correct
this, but for now we'll have to live with that edgecase.
 2021-06-22 00:00:50 -04:00
Joshua Boniface dc560c1dcb Better handle retcodes in migrate update 2021-06-21 23:46:47 -04:00
Joshua Boniface 68c7481aa2 Ensure offline migrations update SR-IOV NIC states 2021-06-21 23:35:52 -04:00
Joshua Boniface 24ce361a04 Ensure SR-IOV NIC states are updated on migration 2021-06-21 23:18:34 -04:00
Joshua Boniface eeb83da97d Add support for SR-IOV NICs to VMs 2021-06-21 23:18:22 -04:00
Joshua Boniface 64d1a37b3c Add PCIe device paths to SR-IOV VF information 
This will be used when adding VM network interfaces of type hostdev.
 2021-06-21 21:08:46 -04:00
Joshua Boniface 13cc0f986f Implement SR-IOV VF config set 
Also fixes some random bugs, adds proper interface sorting, and assorted
tweaks.
 2021-06-21 18:40:11 -04:00
Joshua Boniface 33195c3c29 Ensure VF list is sorted 2021-06-21 17:11:48 -04:00
Joshua Boniface a697c2db2e Add SRIOV PF and VF listing to API 2021-06-21 01:42:55 -04:00
Joshua Boniface 509afd4d05 Add hostdev net_type to handler as well 2021-06-17 01:52:58 -04:00
Joshua Boniface e7b6a3eac1 Implement SR-IOV PF and VF instances 
Adds support for the node daemon managing SR-IOV PF and VF instances.

PFs are added to Zookeeper automatically based on the config at startup
during network configuration, and are otherwise completely static. PFs
are automatically removed from Zookeeper, along with all coresponding
VFs, should the PF phy device be removed from the configuration.

VFs are configured based on the (autocreated) VFs of each PF device,
added to Zookeeper, and then a new class instance, SRIOVVFInstance, is
used to watch them for configuration changes. This will enable the
runtime management of VF settings by the API. The set of keys ensures
that both configuration and details of the NIC can be tracked.

Most keys are self-explanatory, especially for PFs and the basic keys
for VFs. The configuration tree is also self-explanatory, being based
entirely on the options available in the `ip link set {dev} vf` command.

Two additional keys are also present: `used` and `used_by`, which will
be able to track the (boolean) state of usage, as well as the VM that
uses a given VIF. Since the VM side implementation will support both
macvtap and direct "hostdev" assignments, this will ensure that this
state can be tracked on both the VF and the VM side.
 2021-06-17 01:33:03 -04:00
Joshua Boniface f540dd320b Allow VNI for "direct" type vNICs 2021-06-15 00:27:01 -04:00
Joshua Boniface 23318524b9 Ensure validate writes a valid schema version 2021-06-14 21:27:37 -04:00
Joshua Boniface 5f11b3198b Fix base schema None issue in handler too 2021-06-14 21:13:40 -04:00
Joshua Boniface 20c773413c Fix bug in snapshot rename 2021-06-14 00:55:26 -04:00
Joshua Boniface 49f4feb482 Fix typo bug in key rename 2021-06-14 00:51:45 -04:00
Joshua Boniface 30a160d5ff Fix invalid type_key 2021-06-13 21:20:10 -04:00
Joshua Boniface 1cbc66dccf Fix bugs in lease listing 2021-06-13 21:10:42 -04:00
Joshua Boniface bbd903e568 Fix bad schema name 2021-06-13 21:02:44 -04:00
Joshua Boniface 9511dc9864 Correct issue with invalid ACL ordering 2021-06-13 20:55:28 -04:00
Joshua Boniface 3013973975 Fix bad schema names 2021-06-13 20:32:41 -04:00
Joshua Boniface 8269930d40 Fix bad entry in network add 2021-06-13 18:22:13 -04:00
Joshua Boniface ae79113f7c Correct key typo and add error handler 2021-06-13 15:49:30 -04:00
Joshua Boniface 3bad3de720 Verify if key exists before reading 2021-06-13 15:39:43 -04:00
Joshua Boniface 680c62a6e4 Fix schema path call and version check 2021-06-13 14:46:30 -04:00
Joshua Boniface 88a1d89501 Fix bad key name 2021-06-13 14:29:54 -04:00
Joshua Boniface 7110a42e5f Add final schema elements after refactoring 2021-06-13 14:26:17 -04:00
Joshua Boniface 01c82f5d19 Move backup and restore into common 2021-06-13 14:25:51 -04:00
Joshua Boniface 059230d369 Convert vm.py to new ZK schema handler 2021-06-13 13:41:21 -04:00
Joshua Boniface f6e37906a9 Convert node.py to new ZK schema handler 2021-06-13 13:18:34 -04:00
Joshua Boniface 0a162b304a Convert network.py to new ZK schema handler 2021-06-12 18:40:25 -04:00
Joshua Boniface f071343333 Add DHCP lease schema and temp workaround 2021-06-12 18:22:43 -04:00
Joshua Boniface 01c762a362 Convert common.py to new ZK schema handler 2021-06-12 17:59:09 -04:00
Joshua Boniface 9b1bd8476f Convert cluster.py to new ZK schema handler 2021-06-12 17:11:32 -04:00
Joshua Boniface 6d00ec07b5 Convert ceph.py to new ZK schema handler 2021-06-12 17:09:29 -04:00
Joshua Boniface 247ae4fe2d Fix pre-refactor path bug 2021-06-10 01:18:33 -04:00
Joshua Boniface b1c13c9fc1 Fix another bug with read call 2021-06-10 01:08:18 -04:00
Joshua Boniface 75fc40a1e8 Fix bug with nkipath 2021-06-10 01:00:40 -04:00
Joshua Boniface 2aa7f87ca9 Fix bug in creating child path keys 2021-06-10 00:55:54 -04:00
Joshua Boniface 5273c4ebfa Fix bug with encoding raw creates 2021-06-10 00:52:07 -04:00
Joshua Boniface 8dc9fd6dcb Fix bug with sub self command path/key 2021-06-10 00:49:01 -04:00
Joshua Boniface f030ed974c Correct schema and handling of network subkeys 
Required a bit of refactoring in the validation code to ensure we have
direct access, without relying on the translations done in the normal
zkhandler functions.
 2021-06-10 00:35:42 -04:00
Joshua Boniface 9985e1dadd Add support for 2-level dynamic keys 2021-06-09 23:52:21 -04:00
Joshua Boniface 7e42118e6f Adjust lock schema in NodeInstance and VMInstance 
Removes a superfluous lock and puts the sync_lock keys in more usable
places.
 2021-06-09 22:51:00 -04:00
Joshua Boniface 24663a3333 Add missing VM schema entry 2021-06-09 22:12:24 -04:00
Joshua Boniface a9a57533a7 Integrate schema handling within ZKHandler 
Abstracts away the schema management, especially when doing actions, to
prevent duplication in other areas.
 2021-06-09 13:23:57 -04:00
Joshua Boniface 76c37e6628 Tweak some field names slightly and add missing 2021-06-09 09:58:18 -04:00
Joshua Boniface 0a04adf8f9 Allow empty sub_paths 2021-06-09 01:54:29 -04:00
Joshua Boniface f2b55ba937 Fix some bugs with migrations 2021-06-09 00:04:16 -04:00
Joshua Boniface 5540bdc86b Add automatic schema upgrade to nodes 
Performs an automatic schema upgrade when all nodes are updated to the
latest version.

Addresses #129
 2021-06-08 23:35:39 -04:00
Joshua Boniface 3c102b3769 Add per-node schema tracking 
This will allow nodes to start with their own schema versions, and then
be updated simultaneously by the API.

References #129
 2021-06-08 23:35:39 -04:00
Joshua Boniface 602dd7b714 Update version 0 schema and add full validation 
Addresses #129
 2021-06-08 23:35:39 -04:00
Joshua Boniface 126f0742cd Add Zookeeper schema manager to zkhandler 
Adds a new class, ZKSchema, to handle schema management in Zookeeper in
an automated and consistent way. This should solve several issues:

1. Pain in managing changes to ZK keys
2. Pain in handling those changes during live upgrades
3. Simplifying the codebase to remove hardcoded ZK paths

The current master schema for PVC 0.9.19 is committed as version 0.

Addresses #129
 2021-06-08 23:35:39 -04:00
Joshua Boniface f0dc0fc782 Avoid duplicating maintenance state change 
This makes no functional difference, but is technically more correct.
 2021-06-05 01:36:40 -04:00
Joshua Boniface 5d88e92acc Avoid returning errors with duplicate router mode 
Like the previous (new) flush change, these shouldn't be errors, but
simply information "what you want is already done" messages.
 2021-06-05 01:14:31 -04:00
Joshua Boniface 505c109875 Avoid re-flush or re-ready nodes if unnecessary 2021-06-05 01:08:32 -04:00
Joshua Boniface 7dea5d2fac Move logger to common, fix buffering 2021-06-01 18:50:26 -04:00
Joshua Boniface 9764090d6d Merge node common with daemon common 2021-06-01 12:22:11 -04:00
Joshua Boniface f73c433fc7 Remove useless try and import 2021-06-01 12:05:17 -04:00
Joshua Boniface a48bf2d71e More gracefully handle none selectors 
Allow selection of "none" as the node selector, and handle this by
always using the cluster default instead of writing it in.
 2021-06-01 11:13:13 -04:00
Joshua Boniface 33a54cf7f2 Move configuration keys to /config tree 2021-06-01 10:48:55 -04:00
Joshua Boniface d3b5b5236a Remove transactional delete 
This just doesn't work due to the darn limit on recursive deletes in
transactions.
 2021-05-31 19:22:01 -04:00
Joshua Boniface 8625e9bd3e Update Delete to recursive method 2021-05-31 03:14:09 -04:00
Joshua Boniface ed4f84a3ec Add log handling and persistent listener 2021-05-31 03:14:09 -04:00
Joshua Boniface 9be426507a Fix erroneous lock calls 2021-05-30 15:31:17 -04:00
Joshua Boniface 58a5b00aa1 Remove extraneous zkhandler reference 2021-05-30 01:01:40 -04:00
Joshua Boniface b4f2cf879e Rework vm library for new zkhandler 2021-05-29 21:17:19 -04:00
Joshua Boniface 3603b782c0 Rework node library for new zkhandler 2021-05-29 20:56:21 -04:00
Joshua Boniface 62cb72b62f Rework network library for new zkhandler 2021-05-29 20:53:42 -04:00
Joshua Boniface b186a75b4e Rework common library for new zkhandler 2021-05-29 20:35:28 -04:00
Joshua Boniface 6205dba451 Rework cluster library for new zkhandler 2021-05-29 20:32:20 -04:00
Joshua Boniface 688d1a6ae2 Rework ceph library for new zkhandler 2021-05-29 20:29:51 -04:00
Joshua Boniface f82da03a62 Add first wrappers and exceptions 2021-05-28 00:19:39 -04:00
Joshua Boniface fef230ad98 Implement class-based version of zkhander 2021-05-27 22:50:00 -04:00
Joshua Boniface f46c2e7f6a Implement VM rename functionality 
Closes #125
 2021-05-23 17:21:19 -04:00
Joshua Boniface 0bf276fd51 Update copyright year in headers 2021-03-25 17:01:55 -04:00
Joshua Boniface f4ec161aa2 Update file copyright header. 
Remove the option to select a later version of the GPL.
 2021-03-25 16:58:02 -04:00
Joshua Boniface 8c4c1aba38 Revert "Add object-map to default RBD volume features" 
This reverts commit 00e9cca6f0.

This causes RBD mapping to fail on Debian 10. Reverting but keeping in
history for future reference.
 2021-02-28 01:57:30 -05:00
Joshua Boniface 00e9cca6f0 Add object-map to default RBD volume features 
This allows for much quicker image deletes especially for empty images.
 2021-02-17 13:57:28 -05:00
Joshua Boniface ab05e0f3db Add B suffix back to bare bytes values 2021-02-17 11:37:36 -05:00
Joshua Boniface 9291ce6ffc Correct output of fromhuman and add size compare 
Ensures that the bytes_tohuman returns an integer to avoid the hacky
workaround of stripping off the B.

Adds a verification on the size of a new volume, that it is not larger
than the free space of the pool to prevent errors/excessively-large
volumes from being created.

Closes #120
 2021-02-17 11:27:26 -05:00
Joshua Boniface 0e4bece441 Add missing inc/dec of snapshot_count 2021-02-14 17:02:49 -05:00
Joshua Boniface b33c0ab0e2 Final final fix to snapshot ordering 2021-02-14 16:48:42 -05:00
Joshua Boniface 150c61d226 Actually fix sorting issue 2021-02-14 16:41:59 -05:00
Joshua Boniface 26b0a8b5c1 Fix sorting bug with snapshot list 2021-02-14 16:34:43 -05:00
Joshua Boniface 1b6613c280 Add live VNC information to domain output 
Sets in the node daemon, returns via the API, and shows in the CLI,
information about the live VNC listen address and port for VNC-enabled
VMs.

Closes #115
 2020-12-20 16:00:55 -05:00
Joshua Boniface f58e95e4c1 Fix bugs in modifying networks 
1. Use a consistent "is not None" to verify records are changing.
2. Fix bug where IPv6 network had no remove setter (it is now a blank
string, the first thing I would expect).
3. 1 fixes a bug whereby it was impossible to unset DHCPv4 status.
 2020-12-04 04:15:04 -05:00
Joshua Boniface 0f8e5c6536 Add VNI to VM network API list 
Saves some processing on API clients.
 2020-12-01 04:44:33 -05:00
Joshua Boniface 593810e53e Add volume_count to pool API data 2020-12-01 03:40:41 -05:00
Joshua Boniface 6912bd7a2a Add missing return statement for vm_modify 2020-11-08 00:57:13 -05:00
Joshua Boniface 0bf130077c Lint: W605 invalid escape sequence '<char>' 
The previous attempt to correct these with character classes failed.
Instead, use the proper `r'blah'` regex formatting.
 2020-11-07 17:41:09 -05:00
Joshua Boniface 3ed97d209f Revert "Lint: W605 invalid escape sequence '\^'" 
This reverts commit 639937f9c2.
 2020-11-07 17:39:34 -05:00
Joshua Boniface f33398458e Revert "Lint: W605 invalid escape sequence '\$'" 
This reverts commit 76b1cafdcc.
 2020-11-07 17:38:03 -05:00
Joshua Boniface 78eedf0d2e Remove more dead code 2020-11-07 15:20:36 -05:00