52 Commits

Author SHA1 Message Date
80fe96b24d Add some additional docstrings 2021-07-07 12:28:08 -04:00
80f04ce8ee Remove connection renewal in state handler
Regenerating the ZK connection was fraught with issues, including
duplicate connections, strange failures to reconnect, and various other
wonkiness.

Instead let Kazoo handle states sensibly. Kazoo moves to SUSPENDED state
when it loses connectivity, and stays there indefinitely (based on
cursory tests). And Kazoo seems to always resume from this just fine on
its own. Thus all that hackery did nothing but complicate reconnection.

This therefore turns the listener into a purely informational function,
providing logs of when/why it failed, and we also add some additional
output messages during initial connection and final disconnection.
2021-07-07 11:55:12 -04:00
a8c28786dd Better handle empty ipaths in schema
When trying to write to sub-item paths that don't yet exist, the
previous method would just blindly write to whatever the root key is,
which is never what we actually want.

Instead, check explicitly for a "base path" situation, and handle that.
Then, if we try to get a subpath that isn't valid, return None. Finally
in the various functions, if the path is None, just continue (or return
false/None) and (try to) chug along.
2021-07-05 23:35:03 -04:00
c45804e8c1 Revert "Return none if a schema path is not found"
This reverts commit b1fcf6a4a5c0987ef72de54d96edc8d7bb9e4eda.
2021-07-05 23:16:39 -04:00
b1fcf6a4a5 Return none if a schema path is not found
This can cause overwriting of unintended keys, so should not be
happening. Will have to find the bugs this causes.
2021-07-05 17:15:55 -04:00
a69105569f Add node PVC version data to Node information
Allows API client to see the currently-active version of the node
daemon.
2021-07-05 09:57:38 -04:00
e093efceb1 Add NoNodeError handlers in ZK locks
Instead of looping 5+ times acquiring an impossible lock on a
nonexistent key, just fail on a different error and return failure
immediately.

This is likely a major corner case that shouldn't happen, but better to
be safe than 500.
2021-07-01 01:17:38 -04:00
a080598781 Avoid superfluous ZK exists calls
These cause a major (2x) slowdown in read calls since Zookeeper
connections are expensive/slow. Instead, just try the thing and return
None if there's no key there.

Also wrap the children command in similar error handling since that did
not exist and could likely cause some bugs at some point.
2021-07-01 01:15:51 -04:00
e623909a43 Store PHY MAC for VFs and restore after free 2021-06-22 00:56:47 -04:00
64d1a37b3c Add PCIe device paths to SR-IOV VF information
This will be used when adding VM network interfaces of type hostdev.
2021-06-21 21:08:46 -04:00
e7b6a3eac1 Implement SR-IOV PF and VF instances
Adds support for the node daemon managing SR-IOV PF and VF instances.

PFs are added to Zookeeper automatically based on the config at startup
during network configuration, and are otherwise completely static. PFs
are automatically removed from Zookeeper, along with all coresponding
VFs, should the PF phy device be removed from the configuration.

VFs are configured based on the (autocreated) VFs of each PF device,
added to Zookeeper, and then a new class instance, SRIOVVFInstance, is
used to watch them for configuration changes. This will enable the
runtime management of VF settings by the API. The set of keys ensures
that both configuration and details of the NIC can be tracked.

Most keys are self-explanatory, especially for PFs and the basic keys
for VFs. The configuration tree is also self-explanatory, being based
entirely on the options available in the `ip link set {dev} vf` command.

Two additional keys are also present: `used` and `used_by`, which will
be able to track the (boolean) state of usage, as well as the VM that
uses a given VIF. Since the VM side implementation will support both
macvtap and direct "hostdev" assignments, this will ensure that this
state can be tracked on both the VF and the VM side.
2021-06-17 01:33:03 -04:00
23318524b9 Ensure validate writes a valid schema version 2021-06-14 21:27:37 -04:00
5f11b3198b Fix base schema None issue in handler too 2021-06-14 21:13:40 -04:00
49f4feb482 Fix typo bug in key rename 2021-06-14 00:51:45 -04:00
3bad3de720 Verify if key exists before reading 2021-06-13 15:39:43 -04:00
7110a42e5f Add final schema elements after refactoring 2021-06-13 14:26:17 -04:00
f071343333 Add DHCP lease schema and temp workaround 2021-06-12 18:22:43 -04:00
b1c13c9fc1 Fix another bug with read call 2021-06-10 01:08:18 -04:00
75fc40a1e8 Fix bug with nkipath 2021-06-10 01:00:40 -04:00
2aa7f87ca9 Fix bug in creating child path keys 2021-06-10 00:55:54 -04:00
5273c4ebfa Fix bug with encoding raw creates 2021-06-10 00:52:07 -04:00
8dc9fd6dcb Fix bug with sub self command path/key 2021-06-10 00:49:01 -04:00
f030ed974c Correct schema and handling of network subkeys
Required a bit of refactoring in the validation code to ensure we have
direct access, without relying on the translations done in the normal
zkhandler functions.
2021-06-10 00:35:42 -04:00
9985e1dadd Add support for 2-level dynamic keys 2021-06-09 23:52:21 -04:00
7e42118e6f Adjust lock schema in NodeInstance and VMInstance
Removes a superfluous lock and puts the sync_lock keys in more usable
places.
2021-06-09 22:51:00 -04:00
24663a3333 Add missing VM schema entry 2021-06-09 22:12:24 -04:00
a9a57533a7 Integrate schema handling within ZKHandler
Abstracts away the schema management, especially when doing actions, to
prevent duplication in other areas.
2021-06-09 13:23:57 -04:00
76c37e6628 Tweak some field names slightly and add missing 2021-06-09 09:58:18 -04:00
0a04adf8f9 Allow empty sub_paths 2021-06-09 01:54:29 -04:00
f2b55ba937 Fix some bugs with migrations 2021-06-09 00:04:16 -04:00
5540bdc86b Add automatic schema upgrade to nodes
Performs an automatic schema upgrade when all nodes are updated to the
latest version.

Addresses #129
2021-06-08 23:35:39 -04:00
3c102b3769 Add per-node schema tracking
This will allow nodes to start with their own schema versions, and then
be updated simultaneously by the API.

References #129
2021-06-08 23:35:39 -04:00
602dd7b714 Update version 0 schema and add full validation
Addresses #129
2021-06-08 23:35:39 -04:00
126f0742cd Add Zookeeper schema manager to zkhandler
Adds a new class, ZKSchema, to handle schema management in Zookeeper in
an automated and consistent way. This should solve several issues:

1. Pain in managing changes to ZK keys
2. Pain in handling those changes during live upgrades
3. Simplifying the codebase to remove hardcoded ZK paths

The current master schema for PVC 0.9.19 is committed as version 0.

Addresses #129
2021-06-08 23:35:39 -04:00
d3b5b5236a Remove transactional delete
This just doesn't work due to the darn limit on recursive deletes in
transactions.
2021-05-31 19:22:01 -04:00
8625e9bd3e Update Delete to recursive method 2021-05-31 03:14:09 -04:00
ed4f84a3ec Add log handling and persistent listener 2021-05-31 03:14:09 -04:00
f82da03a62 Add first wrappers and exceptions 2021-05-28 00:19:39 -04:00
fef230ad98 Implement class-based version of zkhander 2021-05-27 22:50:00 -04:00
0bf276fd51 Update copyright year in headers 2021-03-25 17:01:55 -04:00
f4ec161aa2 Update file copyright header.
Remove the option to select a later version of the GPL.
2021-03-25 16:58:02 -04:00
260b39ebf2 Lint: E302 expected 2 blank lines, found X 2020-11-07 14:45:24 -05:00
99d723dd8b Lint: F821 undefined name 'count' 2020-11-07 13:39:51 -05:00
6cf7f178a6 Lint: F821 undefined name 'time' 2020-11-07 13:38:54 -05:00
5da314902f Lint: F841 local variable '<variable>' is assigned to but never used 2020-11-06 21:13:13 -05:00
d9e7b7ec15 Lint: F401 <library> imported but unused 2020-11-06 19:22:49 -05:00
42c5f84ba7 Do multiple lock attempts 2020-10-21 11:21:37 -04:00
3839040092 Add exclusive lock function 2020-10-21 10:46:41 -04:00
72f47f216a Revert "Add locking in common zkhander"
This reverts commit 53c0d2b4f620e10f3e0e6b0f19495d8473792563.

This resulted in a massive performance hit and some inconsistent
behaviour. Revert for now an re-investigate later.
2020-10-21 03:49:13 -04:00
53c0d2b4f6 Add locking in common zkhander
Ensures that every changed made here is locked, thus preventing
duplicate updates, etc.
2020-10-21 03:17:18 -04:00