Commit Graph

260 Commits

Author SHA1 Message Date
Joshua Boniface d8d3feee22 Add selector help and adjust flag name
1. Add documentation on the node selector flags. In the API, reference
the daemon configuration manual which now includes details in this
section; in the CLI, provide the help in "pvc vm define" in detail and
then reference that command's help in the other commands that use this
field.

2. Ensure the naming is consistent in the CLI, using the flag name
"--node-selector" everywhere (was "--selector" for "pvc vm" commands and
"--node-selector" for "pvc provisioner" commands).
2022-06-10 02:42:06 -04:00
Joshua Boniface b1357cafdb Add memfree to selector and use proper defaults 2022-06-10 02:03:12 -04:00
Joshua Boniface 51ad2058ed Bump version to 0.9.49 2022-05-06 15:49:39 -04:00
Joshua Boniface c401a1f655 Use consistent language for primary mode
I didn't call it "router" anywhere else, but the state in the list is
called "coordinator" so, call it "coordinator mode".
2022-05-06 15:40:52 -04:00
Joshua Boniface 7a40c7a55b Add support for replacing/refreshing OSDs
Adds commands to both replace an OSD disk, and refresh (reimport) an
existing OSD disk on a new node. This handles the cases where an OSD
disk should be replaced (either due to upgrades or failures) or where a
node is rebuilt in-place and an existing OSD must be re-imported to it.

This should avoid the need to do a full remove/add sequence for either
case.

Also cleans up some aspects of OSD removal that are identical between
methods (e.g. using safe-to-destroy and sleeping after stopping) and
fixes a bug if an OSD does not truly exist when the daemon starts up.
2022-05-06 15:32:06 -04:00
Joshua Boniface 5807351405 Bump version to 0.9.48 2022-04-29 15:03:52 -04:00
Joshua Boniface 4d698be34b Add OSD removal force option
Ensures a removal can continue even in situations where some step(s)
might fail, for instance removing an obsolete OSD from a replaced node.
2022-04-29 11:16:33 -04:00
Joshua Boniface ea709f573f Bump version to 0.9.47 2021-12-28 22:03:08 -05:00
Joshua Boniface 1142454934 Add pool PGs count modification
Allows an administrator to adjust the PG count of a given pool. This can
be used to increase the PGs (for example after adding more OSDs) or
decrease it (to remove OSDs, reduce CPU load, etc.).
2021-12-28 21:53:29 -05:00
Joshua Boniface bbfad340a1 Add PGs count to pool list 2021-12-28 21:12:02 -05:00
Joshua Boniface 25fe45dd28 Add device class tiers to Ceph pools
Allows specifying a particular device class ("tier") for a given pool,
for instance SSD-only or NVMe-only. This is implemented with Crush
rules on the Ceph side, and via an additional new key in the pool
Zookeeper schema which is defaulted to "default".
2021-12-28 20:58:15 -05:00
Joshua Boniface 58d57d7037 Bump version to 0.9.46 2021-12-28 15:02:14 -05:00
Joshua Boniface abc23ebb18 Handle detect strings as arguments for blockdevs
Allows specifying blockdevs in the OSD and OSD-DB addition commands as
detect strings rather than actual block device paths. This provides
greater flexibility for automation with pvcbootstrapd (which originates
the concept of detect strings) and in general usage as well.
2021-12-28 02:53:02 -05:00
Joshua Boniface d8689e6eaa Remove "and started" from message text
This is not necessarily the case.
2021-11-29 16:42:26 -05:00
Joshua Boniface bc49b5eca2 Fix bug with cloned image sizes 2021-11-29 14:56:50 -05:00
Joshua Boniface f164d898c1 Bump version to 0.9.45 2021-11-25 09:34:20 -05:00
Joshua Boniface 817dffcf30 Bump version to 0.9.44 2021-11-11 16:20:38 -05:00
Joshua Boniface 6e9fcd38a3 Bump version to 0.9.43 2021-11-08 02:29:17 -05:00
Joshua Boniface 5f193a6134 Perform automatic shutdown/stop on VM disable
Instead of requiring the VM to already be stopped, instead allow disable
state changes to perform a shutdown first. Also add a force option which
will do a hard stop instead of a shutdown.

References #148
2021-11-06 03:57:24 -04:00
Joshua Boniface 78faa90139 Reformat recent changes with Black 2021-11-06 03:27:07 -04:00
Joshua Boniface 66bfad3109 Fix linting errors F522/F523 unused args 2021-11-06 03:24:50 -04:00
Joshua Boniface c41664d2da Reformat code with Black code formatter
Unify the code style along PEP and Black principles using the tool.
2021-11-06 03:02:43 -04:00
Joshua Boniface e88147db4a Bump version to 0.9.42 2021-10-12 15:25:42 -04:00
Joshua Boniface f13cc04b89 Bump version to 0.9.41 2021-10-09 19:39:21 -04:00
Joshua Boniface 24de0f4189 Add MTU to network creation/modification
Addresses #144
2021-10-09 17:51:32 -04:00
Joshua Boniface c27359c4bf Bump version to 0.9.40 2021-10-07 14:42:04 -04:00
Joshua Boniface bdb9db8375 Bump version to 0.9.39 2021-10-07 11:52:38 -04:00
Joshua Boniface c0f7ba0125 Add limit negation to VM list
When using the "state", "node", or "tag" arguments to a VM list, add
support for a "negate" flag to look for all VMs *not in* the state,
node, or tag state.
2021-10-07 11:50:52 -04:00
Joshua Boniface da9248cfa2 Bump version to 0.9.38 2021-10-03 22:32:41 -04:00
Joshua Boniface 32b3af697c Tweak fio tests for benchmarks
1. Remove ramp_time as this was giving very strange results.

2. Up the runtime to 75 seconds to compensate.

3. Print the fio command to the console to validate.
2021-10-03 15:06:18 -04:00
Joshua Boniface 0ff9a6b8c4 Handle benchmark running state properly 2021-10-02 01:54:51 -04:00
Joshua Boniface 28377178d2 Fix missing argument in database insert 2021-10-02 01:49:47 -04:00
Joshua Boniface e06b114c48 Update to storage benchmark format 1
1. Runs `fio` with the `--format=json` option and removes all terse
format parsing from the results.

2. Adds a 15-second ramp time to minimize wonky ramp-up results.

3. Sets group_reporting, which isn't necessary with only a single job,
but is here for consistency.
2021-10-02 01:41:08 -04:00
Joshua Boniface f83a345bfe Add test format versioning to storage benchmarks
Adds a test_format database column and a value in the API return for the
test format version, starting at 0 for the existing format as of 0.9.37.

References #143
2021-10-02 00:55:27 -04:00
Joshua Boniface ce06e4d81b Load benchmark results as JSON
Load the JSON at the API side instead of client side, because that's
what the API doc says it is and it just makes more sense.
2021-09-30 23:40:24 -04:00
Joshua Boniface 23977b04fc Bump version to 0.9.37 2021-09-30 02:08:14 -04:00
Joshua Boniface bb1cca522f Revamp benchmark tests
1. Move to a time-based (60s) benchmark to avoid these taking an absurd
amount of time to show the same information.

2. Eliminate the 256k random benchmarks, since they don't really add
anything.

3. Add in a 4k single-queue benchmark as this might provide valuable
insight into latency.

4. Adjust the output to reflect the above changes.

While this does change the benchmarking, this should not invalidate any
existing benchmarks since most of the test suit is unchanged (especially
the most important 4M sequential and 4K random tests). It simply removes
an unused entry and adds a more helpful one. The time-based change
should not significantly affect the results either, just reduces the
total runtime for long-tests and increase the runtime for quick tests to
provide a better picture.
2021-09-29 20:51:30 -04:00
Joshua Boniface 9a4dce4e4c Add primary node to benchmark job name
Ensures tracking of the current primary node the job was run on, since
this may be relevant for performance reasons.
2021-09-28 09:58:22 -04:00
Joshua Boniface 65df807b09 Add support for configurable OSD DB ratios
The default of 0.05 (5%) is likely ideal in the initial implementation,
but allow this to be set explicitly for maximum flexibility in
space-constrained or performance-critical use-cases.
2021-09-24 01:06:39 -04:00
Joshua Boniface d0f3e9e285 Bump version to 0.9.36 2021-09-23 14:01:38 -04:00
Joshua Boniface adc8a5a3bc Add separate OSD DB device support
Adds in three parts:

1. Create an API endpoint to create OSD DB volume groups on a device.
Passed through to the node via the same command pipeline as
creating/removing OSDs, and creates a volume group with a fixed name
(osd-db).

2. Adds API support for specifying whether or not to use this DB volume
group when creating a new OSD via the "ext_db" flag. Naming and sizing
is fixed for simplicity and based on Ceph recommendations (5% of OSD
size). The Zookeeper schema tracks the block device to use during
removal.

3. Adds CLI support for the new and modified API endpoints, as well as
displaying the block device and DB block device in the OSD list.

While I debated supporting adding a DB device to an existing OSD, in
practice this ended up being a very complex operation involving stopping
the OSD and setting some options, so this is not supported; this can be
specified during OSD creation only.

Closes #142
2021-09-23 13:59:49 -04:00
Joshua Boniface 772807deb3 Bump version to 0.9.35 2021-09-13 02:20:46 -04:00
Joshua Boniface e962743e51 Add VM device hot attach/detach support
Adds a new API endpoint to support hot attach/detach of devices, and the
corresponding client-side logic to use this endpoint when doing VM
network/storage add/remove actions.

The live attach is now the default behaviour for these types of
additions and removals, and can be disabled if needed.

Closes #141
2021-09-12 19:33:00 -04:00
Joshua Boniface 694b8e85a0 Bump version to 0.9.34 2021-08-24 16:15:25 -04:00
Joshua Boniface eb321497ee Properly handle =-separated fsargs 2021-08-24 11:40:22 -04:00
Joshua Boniface a18cef5f25 Bump version to 0.9.33 2021-08-21 03:28:48 -04:00
Joshua Boniface afdf254297 Bump version to 0.9.32 2021-08-19 12:37:58 -04:00
Joshua Boniface 7ecc6a2635 Bump version to 0.9.31 2021-07-30 12:08:12 -04:00
Joshua Boniface c460aa051a Add missing floppy RASD type for compat 2021-07-27 16:32:32 -04:00
Joshua Boniface 2a99a27feb Bump version to 0.9.30 2021-07-20 00:01:45 -04:00