Commit Graph

181 Commits

Author SHA1 Message Date
Joshua Boniface 35e27f79ef Fix uploading of non-raw image files
Adds a new API query parameter to define the file size, which is then
used for the temporary image. This is required for, at least VMDK, files
to work properly in qemu-img convert.
2023-09-29 16:19:22 -04:00
Joshua Boniface 1c68e83d98 Fix bad refs to etree library 2023-09-19 11:05:19 -04:00
Joshua Boniface c4397219da Ensure fencing states are properly reflected 2023-09-18 09:59:18 -04:00
Joshua Boniface 79d871ebc6 Ensure provisioning VMs are properly coloured 2023-09-15 15:42:46 -04:00
Joshua Boniface a5ffe373cd Add default node values to commands 2023-09-01 16:33:26 -04:00
Joshua Boniface 3d5cdf2b23 Fix missing function parameter 2023-08-27 20:57:27 -04:00
Joshua Boniface 47d7b23763 Fix missing re import 2023-08-27 14:45:51 -04:00
Joshua Boniface 0bae729a18 Better handle empty diffs 2023-08-27 14:44:21 -04:00
Joshua Boniface b968110e9f Add missing import for difflib unified_diff 2023-08-27 14:40:52 -04:00
Joshua Boniface 4898ae5082 Fix missing import of colorama Fore 2023-08-27 14:34:47 -04:00
Joshua Boniface 249e1568a1 Fix incorrect echo() calls 2023-08-27 14:33:42 -04:00
Joshua Boniface 33205273dc Fix missing YAML load 2023-08-27 11:41:10 -04:00
Joshua Boniface 65d2b7869c Restore original no-connection behavior
Previously not specifying a connection when multiple were available
would error. This restores that behaviour.
2023-08-23 01:38:50 -04:00
Joshua Boniface 66aee73f1d Fix incorrect short flags in node list 2023-08-22 09:26:35 -04:00
Joshua Boniface 2ff7a6865b Avoid none entries in VM state list 2023-08-18 12:34:27 -04:00
Joshua Boniface ae7950e9b7 Fix bad import 2023-08-17 22:45:50 -04:00
Joshua Boniface d769071799 Revamp behaviour of VM "--restart" options
Previously, either "--restart" was specified or a prompt was given, with
the prompt being ignored with "--unsafe" in favour of a reboot. This
failed to provide an explicit way to prevent VM restarts with these
commands, which might be desired in some non-interactive situations, and
the interaction of "--unsafe" with this option was an undesired bug.

This is now a complete binary flag with --restart and --no-restart
versions, while still defaulting to a prompt if neither is specified.
This allows full non-interactive control of this option.
2023-08-17 22:19:36 -04:00
Joshua Boniface 4ccdd6347e Move provisioner wait to helpers and fix 2023-08-17 10:26:19 -04:00
Joshua Boniface b32f478633 Work around strange Python anomaly
Apparently, `True` is both an instance of `int` and `bool`, which is a
change and is very strange. Instead flip the conditional here.
2023-08-17 09:55:19 -04:00
Joshua Boniface d2bcaec28f Move new CLI client into place 2023-08-16 23:55:27 -04:00
Joshua Boniface a70273dbae Move old CLI client out of the way 2023-08-16 23:54:51 -04:00
Joshua Boniface 4685ba1ec4 Move cli_lib to lib directory 2023-05-01 13:43:54 -04:00
Joshua Boniface 74c4ce3ec7 Increase timeout for connections to API 2023-03-14 09:19:13 -04:00
Joshua Boniface 7f57c6dbf7 Adjust the main location too 2023-02-23 10:32:31 -05:00
Joshua Boniface 6865979e08 Show possible version minimum 2023-02-23 10:30:45 -05:00
Joshua Boniface 5126bc3272 Handle old clusters in cluster detail list 2023-02-23 10:28:55 -05:00
Joshua Boniface 765f0ef13d Better handle N/A health from old versions 2023-02-23 10:22:00 -05:00
Joshua Boniface fe258d9d56 Correct bad health text call for old clusters 2023-02-23 10:19:18 -05:00
Joshua Boniface 93d89a2414 Fix status when connecting to old clusters 2023-02-23 10:16:29 -05:00
Joshua Boniface a49f3810d3 Set maintenance colour in cluster detail 2023-02-22 18:20:18 -05:00
Joshua Boniface 8331b7ecd8 Add cluster detail list
Adds a command to show a list of details including health and item
counts for all configured clusters in the client.
2023-02-22 18:09:11 -05:00
Joshua Boniface 94d4ee5b9b Lower default connect timeout to 1s 2023-02-22 18:09:01 -05:00
Joshua Boniface e773211293 Add PVC version to cluster status output 2023-02-22 16:09:24 -05:00
Joshua Boniface 6ddbde763e Correct lint error E741 2023-02-22 12:21:29 -05:00
Joshua Boniface 1f8561d59a Format cluster health like node healths
Make a cleaner construct here.
2023-02-16 12:33:36 -05:00
Joshua Boniface f2d2537e1c Add JSON output format for node info 2023-02-15 21:35:44 -05:00
Joshua Boniface 15ff729f83 Fix comparison in maintenance check 2023-02-15 16:47:31 -05:00
Joshua Boniface f4e8449356 Fix bugs and formatting of health messages 2023-02-15 16:28:56 -05:00
Joshua Boniface 00586074cf Modify cluster health to use new values 2023-02-15 15:45:43 -05:00
Joshua Boniface e6f9e6e0e8 Fix several bugs and optimize output 2023-02-13 16:36:15 -05:00
Joshua Boniface 9c14d84bfc Add node health value and send out API 2023-02-13 15:53:39 -05:00
Joshua Boniface 38d63d9837 Flip behaviour of memory selectors
It didn't make any sense to me for mem(prov) to be the default selector,
since this has too many caveats versus mem(free). Switch to using
mem(free) as the default (i.e. "mem") and make memprov the alternative.
2022-11-15 15:45:59 -05:00
Joshua Boniface 59f97ebbfb Better handle invalid nets in VMs
1. Error out when trying to add a new network to a VM if the network
doesn't exist on the cluster.
2. When showing the VM list, only show invalid networks in red, not the
whole list.
2022-11-01 10:24:24 -04:00
Joshua Boniface 072337f1f0 Remove VXLAN ref where it isn't correct 2022-11-01 09:40:13 -04:00
Joshua Boniface 726d0a562b Update copyright header year 2022-10-06 11:55:27 -04:00
Joshua Boniface 7a3870fc44 Add OVA script support
1. Ensure that system_template and script are not nullable in the DB.
2. Ensure that the CLI and API enforce the above and clean up CLI
arguments for profile add.
3. Ensure that, before uploading OVAs, a 'default_ova' provisioning
script is present.
4. Use the 'default_ova' script for new OVA uploads.
5. Ensure that OVA details are properly added to the vm_data dict in the
provisioner vmbuilder.
2022-10-06 10:48:12 -04:00
Joshua Boniface 096bcdfd75 Try a literal eval first
This is a breakage between the older version of Celery (Deb10) and
newer. The hard removal broke Deb10 instances.

So try that first, and on failure, assume newer Celery format.
2022-09-06 10:34:50 -04:00
Joshua Boniface d8e57a26c5 Fix bad variable name 2022-08-18 11:37:57 -04:00
Joshua Boniface 171f6ac9ed Add missing cluster_req for vm modify 2022-08-02 10:02:26 -04:00
Joshua Boniface 71ffd5a191 Add confirmation to disable command 2022-07-21 16:43:37 -04:00
Joshua Boniface 2739c27299 Remove faulty literal_eval 2022-07-18 13:35:15 -04:00
Joshua Boniface d8d3feee22 Add selector help and adjust flag name
1. Add documentation on the node selector flags. In the API, reference
the daemon configuration manual which now includes details in this
section; in the CLI, provide the help in "pvc vm define" in detail and
then reference that command's help in the other commands that use this
field.

2. Ensure the naming is consistent in the CLI, using the flag name
"--node-selector" everywhere (was "--selector" for "pvc vm" commands and
"--node-selector" for "pvc provisioner" commands).
2022-06-10 02:42:06 -04:00
Joshua Boniface b1357cafdb Add memfree to selector and use proper defaults 2022-06-10 02:03:12 -04:00
Joshua Boniface c401a1f655 Use consistent language for primary mode
I didn't call it "router" anywhere else, but the state in the list is
called "coordinator" so, call it "coordinator mode".
2022-05-06 15:40:52 -04:00
Joshua Boniface 7a40c7a55b Add support for replacing/refreshing OSDs
Adds commands to both replace an OSD disk, and refresh (reimport) an
existing OSD disk on a new node. This handles the cases where an OSD
disk should be replaced (either due to upgrades or failures) or where a
node is rebuilt in-place and an existing OSD must be re-imported to it.

This should avoid the need to do a full remove/add sequence for either
case.

Also cleans up some aspects of OSD removal that are identical between
methods (e.g. using safe-to-destroy and sleeping after stopping) and
fixes a bug if an OSD does not truly exist when the daemon starts up.
2022-05-06 15:32:06 -04:00
Joshua Boniface 8027a6efdc Improve handling of rounded values 2022-05-02 15:29:30 -04:00
Joshua Boniface d6ca74376a Fix bugs with forced removal 2022-04-29 14:03:07 -04:00
Joshua Boniface 413100a147 Ensure unresponsive OSDs still display in list
It is still useful to see such dead OSDs even if they've never checked
in or have not checked in for quite some time.
2022-04-29 12:11:52 -04:00
Joshua Boniface 4d698be34b Add OSD removal force option
Ensures a removal can continue even in situations where some step(s)
might fail, for instance removing an obsolete OSD from a replaced node.
2022-04-29 11:16:33 -04:00
Joshua Boniface 53aed0a735 Use a singular configured cluster by default
If there is...
  1. No '--cluster' passed, and
  2. No 'local' cluster, and
  3. There is exactly one cluster configured
...then use that cluster by default in the CLI.
2022-01-13 18:36:20 -05:00
Joshua Boniface 1142454934 Add pool PGs count modification
Allows an administrator to adjust the PG count of a given pool. This can
be used to increase the PGs (for example after adding more OSDs) or
decrease it (to remove OSDs, reduce CPU load, etc.).
2021-12-28 21:53:29 -05:00
Joshua Boniface bbfad340a1 Add PGs count to pool list 2021-12-28 21:12:02 -05:00
Joshua Boniface 25fe45dd28 Add device class tiers to Ceph pools
Allows specifying a particular device class ("tier") for a given pool,
for instance SSD-only or NVMe-only. This is implemented with Crush
rules on the Ceph side, and via an additional new key in the pool
Zookeeper schema which is defaulted to "default".
2021-12-28 20:58:15 -05:00
Joshua Boniface abc23ebb18 Handle detect strings as arguments for blockdevs
Allows specifying blockdevs in the OSD and OSD-DB addition commands as
detect strings rather than actual block device paths. This provides
greater flexibility for automation with pvcbootstrapd (which originates
the concept of detect strings) and in general usage as well.
2021-12-28 02:53:02 -05:00
Joshua Boniface 9f122e916f Allow bypassing confirm message for benchmarks 2021-12-23 21:00:42 -05:00
Joshua Boniface 3ce4d90693 Add auditing to local syslog from PVC client
This ensures that any client command is logged by the local system.
Helps ensure Accounting for users of the CLI. Currently logs the full
command executed along with the $USER environment variable contents.
2021-12-10 16:17:33 -05:00
Joshua Boniface 8470dfaa29 Fix bugs with legacy benchmark format 2021-11-26 11:42:35 -05:00
Joshua Boniface 195f31501c Ensure echo always has an argument 2021-11-25 09:33:26 -05:00
Joshua Boniface 135d28e60b Add 0.05s to connection timeout
This is recommended by the Python Requests documentation:

> It’s a good practice to set connect timeouts to slightly larger than a
  multiple of 3, which is the default TCP packet retransmission window.
2021-11-08 03:11:41 -05:00
Joshua Boniface e7d7378bae Use separate connect and data timeouts
This allows us to keep a very low connect timeout of 3 seconds, but also
ensure that long commands (e.g. --wait or VM disable) can take as long
as the API requires to complete.

Avoids having to explicitly set very long single-instance timeouts for
other functions which would block forever on an unreachable API.
2021-11-08 03:10:09 -05:00
Joshua Boniface ea19af6494 Allow American spelling for compatibility 2021-11-08 00:09:59 -05:00
Joshua Boniface 7069d3237c Shorten help messages slightly to fit 2021-11-08 00:07:21 -05:00
Joshua Boniface 619c3f7ff5 Add forced colour support
Allows preserving colour within e.g. watch, where Click would normally
determine that it is "not a terminal". This is done via the wrapper echo
which filters via the local config.
2021-11-08 00:04:20 -05:00
Joshua Boniface 21b4bbe51a Apply more granular timeout formatting
We don't need to wait forever if state changes aren't waiting or disable
(which does a shutdown before returning).
2021-11-06 13:34:03 -04:00
Joshua Boniface 87ec31c023 Up timeout when setting VM state
Ensures the API won't time out immediately especially during a
wait-flagged or disable action.
2021-11-06 04:15:10 -04:00
Joshua Boniface 006f40f195 Add disable forcing to CLI
References #148
2021-11-06 04:02:50 -04:00
Joshua Boniface 66bfad3109 Fix linting errors F522/F523 unused args 2021-11-06 03:24:50 -04:00
Joshua Boniface c41664d2da Reformat code with Black code formatter
Unify the code style along PEP and Black principles using the tool.
2021-11-06 03:02:43 -04:00
Joshua Boniface 12eef58d42 Fix ordering to show correct message 2021-10-27 13:37:52 -04:00
Joshua Boniface f2e6892fd2 Support adding the same network to a VM again
This is a supported configuration for some edge cases and should be
allowed.
2021-10-27 13:33:27 -04:00
Joshua Boniface d87bea4159 More gracefully handle restart + live
Instead of erroring, just use the implication that restarting a VM does
not want a live modification, and proceed from there. Update the help
text to match.
2021-10-27 13:23:39 -04:00
Joshua Boniface 3a6f442856 Support removing VM interfaces by MAC
Provides a way to handle multiple interfaces in the same network
gracefully, while making the previous behaviour explicit.
2021-10-27 13:20:05 -04:00
Joshua Boniface 3f78ca1cc9 Add explicit 3 second timeout to requests 2021-10-11 16:31:18 -04:00
Joshua Boniface e866335918 Add version function support to CLI 2021-10-11 15:34:41 -04:00
Joshua Boniface d2c18d7b46 Fix bad header length in network list 2021-10-09 18:50:32 -04:00
Joshua Boniface 8f71a6d2f6 Add MTU support to network add/modify commands
Addresses #144
2021-10-09 18:06:21 -04:00
Joshua Boniface c0f7ba0125 Add limit negation to VM list
When using the "state", "node", or "tag" arguments to a VM list, add
support for a "negate" flag to look for all VMs *not in* the state,
node, or tag state.
2021-10-07 11:50:52 -04:00
Joshua Boniface aa035a61a7 Correct latency units and format name 2021-10-03 17:06:34 -04:00
Joshua Boniface 7c8ba56561 Revamp test result display
Instead of showing CLAT percentiles, which are very hard to interpret
and understand, instead use the main latency buckets.
2021-10-03 15:49:01 -04:00
Joshua Boniface 7c122ac921 Add benchmark format to list 2021-10-03 15:05:58 -04:00
Joshua Boniface 0dbf139706 Adjust ETA for benchmarks 2021-10-02 04:51:01 -04:00
Joshua Boniface c909beaf6d Add format parsing for format 1 storage benchmarks 2021-10-02 04:46:44 -04:00
Joshua Boniface 2da49297d2 Add version 2 benchmark list formatting 2021-10-02 02:47:17 -04:00
Joshua Boniface 0058f19d88 Fix handling of array of information
With a benchmark info we only ever want test one, so pass only that to
the formatter. Simplifies the format function.
2021-10-02 01:28:39 -04:00
Joshua Boniface 056cf3740d Avoid versioning benchmark lists
This wouldn't work since each individual test is versioned. Instead add
a placeholder for later once additional format(s) are defined.
2021-10-02 01:25:18 -04:00
Joshua Boniface 58f174b87b Add format option to benchmark info
Allows specifying of raw json or json-pretty formats in addition to the
"pretty" formatted option.
2021-10-02 01:13:50 -04:00
Joshua Boniface 37b98fd54f Add benchmark format function support
Allows choosing different list and info functions based on the benchmark
version found. Currently only implements "legacy" version 0 with more to
be added.
2021-10-02 01:07:25 -04:00
Joshua Boniface ce06e4d81b Load benchmark results as JSON
Load the JSON at the API side instead of client side, because that's
what the API doc says it is and it just makes more sense.
2021-09-30 23:40:24 -04:00
Joshua Boniface bb1cca522f Revamp benchmark tests
1. Move to a time-based (60s) benchmark to avoid these taking an absurd
amount of time to show the same information.

2. Eliminate the 256k random benchmarks, since they don't really add
anything.

3. Add in a 4k single-queue benchmark as this might provide valuable
insight into latency.

4. Adjust the output to reflect the above changes.

While this does change the benchmarking, this should not invalidate any
existing benchmarks since most of the test suit is unchanged (especially
the most important 4M sequential and 4K random tests). It simply removes
an unused entry and adds a more helpful one. The time-based change
should not significantly affect the results either, just reduces the
total runtime for long-tests and increase the runtime for quick tests to
provide a better picture.
2021-09-29 20:51:30 -04:00
Joshua Boniface ee4266f8ca Tweak CLI helptext around OSD actions
Adds some more detail about OSD commands and their values.
2021-09-26 01:29:23 -04:00