Commit Graph

3084 Commits

Author SHA1 Message Date
Joshua Boniface a8c28786dd Better handle empty ipaths in schema
When trying to write to sub-item paths that don't yet exist, the
previous method would just blindly write to whatever the root key is,
which is never what we actually want.

Instead, check explicitly for a "base path" situation, and handle that.
Then, if we try to get a subpath that isn't valid, return None. Finally
in the various functions, if the path is None, just continue (or return
false/None) and (try to) chug along.
2021-07-05 23:35:03 -04:00
Joshua Boniface be7b0be8ed Fix typo in schema path name 2021-07-05 23:23:23 -04:00
Joshua Boniface c45804e8c1 Revert "Return none if a schema path is not found"
This reverts commit b1fcf6a4a5.
2021-07-05 23:16:39 -04:00
Joshua Boniface b1fcf6a4a5 Return none if a schema path is not found
This can cause overwriting of unintended keys, so should not be
happening. Will have to find the bugs this causes.
2021-07-05 17:15:55 -04:00
Joshua Boniface 47f39a1a2a Fix ordering issue in test-cluster script 2021-07-05 15:14:34 -04:00
Joshua Boniface 54f82a3ea0 Fix bug in VM network list with SR-IOV 2021-07-05 15:14:01 -04:00
Joshua Boniface 37cd278bc2 Bump version to 0.9.22 2021-07-05 14:18:51 -04:00
Joshua Boniface 47a522f8af Use manual zkhandler creation in Benchmark job
Like the other Celery job this does not work properly with the
ZKConnection decorator due to conflicting "self", so just connect
manually exactly like the provisioner task does.
2021-07-05 14:12:56 -04:00
Joshua Boniface 087c23859c Adjust layout of Provisioner lists output
Use the same header format as the others.
2021-07-05 14:06:22 -04:00
Joshua Boniface 6c21a52714 Adjust layout of Ceph/storage lists output
Use the same header format as node, VM, and network lists.
2021-07-05 12:57:18 -04:00
Joshua Boniface afde436cd0 Adjust layout of Network lists output
Use the same header format as node and VM lists.
2021-07-05 11:48:39 -04:00
Joshua Boniface 1fe71969ca Adjust layout of VM list output
Matches the new node list output format with the additional header line,
as well as revamps some other aspects:

    1. Adjusts the UUID to be under the name in the info output.
    2. Removes the UUID from the list output to save space, because this
       is generally not needed in day-to-day quick-list output.
    3. Renames the "Node" header to "Current" to better reflect what
       that column actually means and avoid conflicting with the parent
       header.
2021-07-05 10:52:48 -04:00
Joshua Boniface 2b04df22a6 Add PVC version to node information output
Also adjusts the layout of the node list output to avoid excessively
long lines. Adds another header line with categories and spacing dashes
for easier visual parsing.
2021-07-05 10:45:20 -04:00
Joshua Boniface a69105569f Add node PVC version data to Node information
Allows API client to see the currently-active version of the node
daemon.
2021-07-05 09:57:38 -04:00
Joshua Boniface 21a1a7da9e Fix bad schema reference
Not sure how this didn't cause an issue until now, but the wrong key
path was used and this was getting unexpected data with the newly-added
version string instead of the proper mode string.
2021-07-05 09:53:51 -04:00
Joshua Boniface e44f3d623e Remove unnecessary try/except blocks from VM reads
The zkhandler read() function takes care of ensuring there is a None
value returned if these fail, so these aren't required. Makes the code a
fair bit more readable here.
2021-07-02 12:01:58 -04:00
Joshua Boniface f0fd3d3f0e Make extra sure VMs terminate when told
When doing a stop_vm or terminate_vm, check again after 0.2 seconds
and try re-terminating if it's still running. Covers cases where a VM
doesn't stop if given the 'stop' state.
2021-07-02 11:40:34 -04:00
Joshua Boniface f12de6727d Adjust logo slightly and add debug state 2021-07-02 02:32:08 -04:00
Joshua Boniface e94f5354e6 Update startup messages with new ASCII logo 2021-07-02 02:21:30 -04:00
Joshua Boniface c51023ba81 Add profiler to keepalive function 2021-07-02 01:55:15 -04:00
Joshua Boniface 61465ef38f Add profiler to several other functions in API 2021-07-02 01:53:19 -04:00
Joshua Boniface 64c6b04352 Ensure all edited files are restored 2021-07-02 01:50:25 -04:00
Joshua Boniface 20542c3653 Add profiler to cluster status function 2021-07-01 17:35:29 -04:00
Joshua Boniface 00b503824e Set unstable version in API and CLI too 2021-07-01 17:35:11 -04:00
Joshua Boniface 43009486ae Move Ceph pool/volume list assembly to thread pool
Same reasons as the VM list, though less impactful.
2021-07-01 17:33:13 -04:00
Joshua Boniface 58789f1db4 Move VM list assembly to thread pool
This helps parallelize the numerous Zookeeper calls a little bit, at
least within the bounds of the GIL, to improve performance when getting
a large list of VMs. The max_workers value is capped at 32 to avoid
causing too many threads during concurrent executions, but still
provides a noticeable speedup (on the order of 0.2-0.4 seconds with 75
VMs, scaling up further as counts grow).
2021-07-01 17:32:47 -04:00
Joshua Boniface baf4c3fbc7 Add performance profiler function
Usable anywhere that the global daemon "config" parameter can be passed
in (e.g. pvcapid/helper.py, pvcnoded/Daemon.py, etc.). Stores results in
a subdirectory of the PVC logdir called "profiler" if this directory can
be created, or prints results.

The debug config parameter ensures that the profiler can be added to
functions and not run unless the server is explicitly in debug mode.
Might not be useful as I don't initially plan to add this to every
function (only when investigating performance problems), but this
flexibility allows that to change later.
2021-07-01 14:01:33 -04:00
Joshua Boniface e093efceb1 Add NoNodeError handlers in ZK locks
Instead of looping 5+ times acquiring an impossible lock on a
nonexistent key, just fail on a different error and return failure
immediately.

This is likely a major corner case that shouldn't happen, but better to
be safe than 500.
2021-07-01 01:17:38 -04:00
Joshua Boniface a080598781 Avoid superfluous ZK exists calls
These cause a major (2x) slowdown in read calls since Zookeeper
connections are expensive/slow. Instead, just try the thing and return
None if there's no key there.

Also wrap the children command in similar error handling since that did
not exist and could likely cause some bugs at some point.
2021-07-01 01:15:51 -04:00
Joshua Boniface 39e82ee426 Cast base schema version to int
Or all our comparisons will fail later and nodes can't start.
2021-06-30 09:40:33 -04:00
Joshua Boniface fe0a1d582a Bump version to 0.9.21 2021-06-29 19:21:31 -04:00
Joshua Boniface 64feabf7c0 Fix adds in bump-version 2021-06-29 19:20:13 -04:00
Joshua Boniface cc841898b4 Pad dimensions of logos slightly 2021-06-29 19:16:35 -04:00
Joshua Boniface de5599b800 Revert "Try dark image instead"
This reverts commit 87f963df4c.
2021-06-29 19:11:38 -04:00
Joshua Boniface 87f963df4c Try dark image instead 2021-06-29 19:09:39 -04:00
Joshua Boniface de23c8c57e Add background colours to logos 2021-06-29 19:08:10 -04:00
Joshua Boniface c62a0a6c6a Revamp introduction text 2021-06-29 18:47:01 -04:00
Joshua Boniface 12212177ef Update to new logo 2021-06-29 18:41:36 -04:00
Joshua Boniface 6adaf1f669 Fix incorrect handling of deletions in init 2021-06-29 18:41:02 -04:00
Joshua Boniface b05c93e260 Fix bad return from initialize call 2021-06-29 18:31:56 -04:00
Joshua Boniface ffdd6bf3f8 Fix typo in command argument 2021-06-29 18:22:39 -04:00
Joshua Boniface aae9ae2e80 Fix incorrect handling of overwrite flag 2021-06-29 18:22:01 -04:00
Joshua Boniface f91c07fdcf Re-add UUID limit matching for full UUIDs
This *was* valuable when passing a full UUID in, so go back to that.
Verify first that the limit string is an actual UUID, and then compare
against it if applicable.
2021-06-28 12:27:43 -04:00
Joshua Boniface 4e2a1c3e52 Add worker wrapper to fix Deb incompatibility
Celery 5.x introduced a new worker argument format that is not
backwards-compatible with the older Celery 4.x format. This created a
conundrum since we use one service unit for both Debian 10 (4.x) and
Debian 11 (5.x). Instead of worse hacks, create a wrapper script to
start the worker with the correct arguments instead.
2021-06-28 12:19:29 -04:00
Joshua Boniface dbfa339cfb Ensure postinst and prerm always succeed 2021-06-23 20:35:40 -04:00
Joshua Boniface c54f66efa8 Limit match only on VM name
I can see no possible reason to want to do limits against UUIDs, but
supporting that means match is not what one would expect since a random
UUID could match the limit. So only limit based on the name.
2021-06-23 19:17:35 -04:00
Joshua Boniface cd860bae6b Optimize VM list in API
With many VMs this slows down linearly. Rework it a bit so there are
fewer calls to getInformationFromXML and so the processing could happen
in parallel at some point.
2021-06-23 19:14:26 -04:00
Joshua Boniface bbb132414c Restore shebang and don't do store if completion 2021-06-23 05:26:50 -04:00
Joshua Boniface 04fa63f081 Only hit the network endpoint once
Otherwise this is hit for every VM which gets very slow very fast.
2021-06-23 05:15:48 -04:00
Joshua Boniface f248d579df Convert pvc-client-cli into a proper Python module
Also fixes up the Debian packaging such that this works how I would
want, with proper module installation while leaving everything else
untouched. Finally implements automatic installation and removal of the
BASH completion for the PVC command.
2021-06-23 05:03:19 -04:00