Commit Graph

374 Commits

Author SHA1 Message Date
Joshua Boniface 96defebd0b Add last item to swagger doc 2023-02-22 00:25:27 -05:00
Joshua Boniface e9aa545e9b Update API specification 2023-02-22 00:06:52 -05:00
Joshua Boniface aeb238f43c Bump version to 0.9.61 2023-02-08 10:08:05 -05:00
Joshua Boniface e945fd8590 Remove bad casting to int in string compare 2023-01-01 13:55:10 -05:00
Joshua Boniface a49510ecc8 Bump version to 0.9.60 2022-12-06 15:42:55 -05:00
Joshua Boniface 6d7730ab52 Disable RBD caching by default
Results in a massive (~2x) performance boost for random block I/O inside
VMs, and thus a worthwhile default change.
2022-12-05 17:56:59 -05:00
Joshua Boniface 8135426973 Fix bad ref in example scripts 2022-11-18 12:54:28 -05:00
Joshua Boniface 20d436a745 Update description 2022-11-16 22:48:40 -05:00
Joshua Boniface 28f6819726 Fix up remaining bugs in Rinse test script 2022-11-16 13:32:24 -05:00
Joshua Boniface 35c07f0384 Ensure transient dirs are cleaned up 2022-11-16 13:01:15 -05:00
Joshua Boniface 6127387be4 Ensure swap is skipped during cleanup too 2022-11-16 12:52:24 -05:00
Joshua Boniface 343d66875b Skip swap volumes during mounting 2022-11-16 12:42:28 -05:00
Joshua Boniface 92feeefd26 Bump version to 0.9.59 2022-11-15 15:50:15 -05:00
Joshua Boniface 38d63d9837 Flip behaviour of memory selectors
It didn't make any sense to me for mem(prov) to be the default selector,
since this has too many caveats versus mem(free). Switch to using
mem(free) as the default (i.e. "mem") and make memprov the alternative.
2022-11-15 15:45:59 -05:00
Joshua Boniface 095bcb2373 Bump version to 0.9.58 2022-11-07 12:27:48 -05:00
Joshua Boniface 91e450f399 Remove extra lower() call where not needed 2022-11-07 12:26:50 -05:00
Joshua Boniface 79eb994a5e Ensure equality of none and None for selector 2022-11-07 11:59:53 -05:00
Joshua Boniface d65f512897 Bump version to 0.9.57 2022-11-06 01:39:50 -04:00
Joshua Boniface ea7a4b2b85 Make benchmarker function as a module
1. Move the test_matrix, volume name, and size to module-level variables
so they can be accessed externally if this is imported.
2. Separate the volume creation and volume cleanup into functions.
3. Separate the individual benchmark runs into a function.

This should enable easier calling of the various subcomponents
externally, e.g. for external benchmark scripts.
2022-11-03 21:33:32 -04:00
Joshua Boniface c3bc55eff8 Bump version to 0.9.56 2022-10-27 14:21:04 -04:00
Joshua Boniface 666e02fbfd Ensure None filesystem is valid 2022-10-21 15:13:52 -04:00
Joshua Boniface 46dde055c4 Move /dev umount to cleanup step 2022-10-21 14:47:48 -04:00
Joshua Boniface ef437c3dbf Adjust help message text 2022-10-21 14:22:15 -04:00
Joshua Boniface bd2208e8f6 Add ova as valid name in addition to default_ova 2022-10-21 14:13:40 -04:00
Joshua Boniface 62d5ff11df Fix console config and domain argument 2022-10-21 14:04:17 -04:00
Joshua Boniface 0019881cfa Add cloud-init configuration to debootstrap script
Prevents errors trying to find the cloud-init metadata source.
2022-10-21 14:03:34 -04:00
Joshua Boniface d46133802b Add pfsense example provisioner script 2022-10-21 13:35:48 -04:00
Joshua Boniface fcadde057e Include /proc in chroot mounts 2022-10-20 15:00:10 -04:00
Joshua Boniface 2608f38d64 Properly handle missing source_volume from OVAs 2022-10-19 13:18:12 -04:00
Joshua Boniface 89f05ced3f Add missing ceph import 2022-10-19 13:10:40 -04:00
Joshua Boniface 729481126c Move conversion to install() step
Seems more clear to me than doing it in prepare()
2022-10-19 13:09:29 -04:00
Joshua Boniface 41eccb9c7d Add missing imports 2022-10-19 13:07:34 -04:00
Joshua Boniface e550e39a5a Add output messages during OVA prepare 2022-10-19 12:58:11 -04:00
Joshua Boniface dff156b2b0 Fix bad comparison 2022-10-19 12:46:15 -04:00
Joshua Boniface e26ff8a975 Fix missing f-string marker 2022-10-15 16:26:47 -04:00
Joshua Boniface a34d64a71b Add rinse example configuration
Provisions Rocky Linux 8 and 9 systems, and potentially older
CentOS/Fedora/Scientific Linux/SuSE systems. Depends on a custom build
of rinse (3.7.1) with Rocky 9 support.
2022-10-07 19:55:56 -04:00
Joshua Boniface 71297e0179 Add host-model to CPU config in VMs 2022-10-07 09:36:22 -04:00
Joshua Boniface 45c9909428 Add output message to debootstrap install 2022-10-07 02:27:20 -04:00
Joshua Boniface 7268592c87 Fix braces in fstring example 2022-10-06 15:57:31 -04:00
Joshua Boniface 726d0a562b Update copyright header year 2022-10-06 11:55:27 -04:00
Joshua Boniface 39e1fc50ed Fix titles 2022-10-06 11:54:36 -04:00
Joshua Boniface 7a3870fc44 Add OVA script support
1. Ensure that system_template and script are not nullable in the DB.
2. Ensure that the CLI and API enforce the above and clean up CLI
arguments for profile add.
3. Ensure that, before uploading OVAs, a 'default_ova' provisioning
script is present.
4. Use the 'default_ova' script for new OVA uploads.
5. Ensure that OVA details are properly added to the vm_data dict in the
provisioner vmbuilder.
2022-10-06 10:48:12 -04:00
Joshua Boniface bffab7a5a1 Reverse numbering of example scripts 2022-10-06 10:14:37 -04:00
Joshua Boniface 6cbaeb5dc8 Complete OVA provisioning script 2022-10-06 10:14:04 -04:00
Joshua Boniface 58ce133c8d Remove lingering OVA references 2022-10-06 00:13:36 -04:00
Joshua Boniface 3a5d8c61da Reorganize and add more comments to examples 2022-10-05 23:35:53 -04:00
Joshua Boniface 1e0b502250 Fix remaining bugs in example scripts 2022-10-05 22:37:11 -04:00
Joshua Boniface fe17d28385 Ensure inner cleanup and end message response 2022-10-05 22:36:42 -04:00
Joshua Boniface 8aaac33056 Fix bad variable reference 2022-10-05 17:43:23 -04:00
Joshua Boniface cc7952c232 Add additional import for config 2022-10-05 17:41:37 -04:00
Joshua Boniface 16915ed507 Add better exception handling with ctxtmgrs 2022-10-05 17:35:05 -04:00
Joshua Boniface 2c624ceb2c Add additional missing import in examples 2022-10-05 17:29:34 -04:00
Joshua Boniface da85480488 Improve error messages 2022-10-05 17:26:09 -04:00
Joshua Boniface 47b0704555 Fix bad ref in examples 2022-10-05 17:25:56 -04:00
Joshua Boniface 7c49967586 Add proper imports to examples 2022-10-05 17:22:04 -04:00
Joshua Boniface e3f96ac87e Better handle cleanups and fix chroot bug 2022-10-05 17:21:30 -04:00
Joshua Boniface 4df70cf086 Implement new provisioner setup 2022-10-05 16:03:05 -04:00
Joshua Boniface f1df1cfe93 Bump version to 0.9.55 2022-10-04 13:21:40 -04:00
Joshua Boniface 239c392892 Bump version to 0.9.54 2022-08-23 11:01:05 -04:00
Joshua Boniface 172d0a86e4 Use proper SSLContext and enable TLSv1
It's bad, but sometimes you need to access the API from a very old
software version. So just enable it for now and clean it up later.
2022-08-23 10:58:47 -04:00
Joshua Boniface 9b499b9f48 Bump version to 0.9.53 2022-08-12 17:47:11 -04:00
Joshua Boniface 2a21d48128 Bump version to 0.9.52 2022-08-12 11:09:25 -04:00
Joshua Boniface 645b525ad7 Bump version to 0.9.51 2022-07-25 23:25:41 -04:00
Joshua Boniface 932b3c55a3 Bump version to 0.9.50 2022-07-06 16:01:14 -04:00
Joshua Boniface d8d3feee22 Add selector help and adjust flag name
1. Add documentation on the node selector flags. In the API, reference
the daemon configuration manual which now includes details in this
section; in the CLI, provide the help in "pvc vm define" in detail and
then reference that command's help in the other commands that use this
field.

2. Ensure the naming is consistent in the CLI, using the flag name
"--node-selector" everywhere (was "--selector" for "pvc vm" commands and
"--node-selector" for "pvc provisioner" commands).
2022-06-10 02:42:06 -04:00
Joshua Boniface b1357cafdb Add memfree to selector and use proper defaults 2022-06-10 02:03:12 -04:00
Joshua Boniface 51ad2058ed Bump version to 0.9.49 2022-05-06 15:49:39 -04:00
Joshua Boniface c401a1f655 Use consistent language for primary mode
I didn't call it "router" anywhere else, but the state in the list is
called "coordinator" so, call it "coordinator mode".
2022-05-06 15:40:52 -04:00
Joshua Boniface 7a40c7a55b Add support for replacing/refreshing OSDs
Adds commands to both replace an OSD disk, and refresh (reimport) an
existing OSD disk on a new node. This handles the cases where an OSD
disk should be replaced (either due to upgrades or failures) or where a
node is rebuilt in-place and an existing OSD must be re-imported to it.

This should avoid the need to do a full remove/add sequence for either
case.

Also cleans up some aspects of OSD removal that are identical between
methods (e.g. using safe-to-destroy and sleeping after stopping) and
fixes a bug if an OSD does not truly exist when the daemon starts up.
2022-05-06 15:32:06 -04:00
Joshua Boniface 5807351405 Bump version to 0.9.48 2022-04-29 15:03:52 -04:00
Joshua Boniface 4d698be34b Add OSD removal force option
Ensures a removal can continue even in situations where some step(s)
might fail, for instance removing an obsolete OSD from a replaced node.
2022-04-29 11:16:33 -04:00
Joshua Boniface ea709f573f Bump version to 0.9.47 2021-12-28 22:03:08 -05:00
Joshua Boniface 1142454934 Add pool PGs count modification
Allows an administrator to adjust the PG count of a given pool. This can
be used to increase the PGs (for example after adding more OSDs) or
decrease it (to remove OSDs, reduce CPU load, etc.).
2021-12-28 21:53:29 -05:00
Joshua Boniface bbfad340a1 Add PGs count to pool list 2021-12-28 21:12:02 -05:00
Joshua Boniface 25fe45dd28 Add device class tiers to Ceph pools
Allows specifying a particular device class ("tier") for a given pool,
for instance SSD-only or NVMe-only. This is implemented with Crush
rules on the Ceph side, and via an additional new key in the pool
Zookeeper schema which is defaulted to "default".
2021-12-28 20:58:15 -05:00
Joshua Boniface 58d57d7037 Bump version to 0.9.46 2021-12-28 15:02:14 -05:00
Joshua Boniface abc23ebb18 Handle detect strings as arguments for blockdevs
Allows specifying blockdevs in the OSD and OSD-DB addition commands as
detect strings rather than actual block device paths. This provides
greater flexibility for automation with pvcbootstrapd (which originates
the concept of detect strings) and in general usage as well.
2021-12-28 02:53:02 -05:00
Joshua Boniface d8689e6eaa Remove "and started" from message text
This is not necessarily the case.
2021-11-29 16:42:26 -05:00
Joshua Boniface bc49b5eca2 Fix bug with cloned image sizes 2021-11-29 14:56:50 -05:00
Joshua Boniface f164d898c1 Bump version to 0.9.45 2021-11-25 09:34:20 -05:00
Joshua Boniface 817dffcf30 Bump version to 0.9.44 2021-11-11 16:20:38 -05:00
Joshua Boniface 6e9fcd38a3 Bump version to 0.9.43 2021-11-08 02:29:17 -05:00
Joshua Boniface 5f193a6134 Perform automatic shutdown/stop on VM disable
Instead of requiring the VM to already be stopped, instead allow disable
state changes to perform a shutdown first. Also add a force option which
will do a hard stop instead of a shutdown.

References #148
2021-11-06 03:57:24 -04:00
Joshua Boniface 78faa90139 Reformat recent changes with Black 2021-11-06 03:27:07 -04:00
Joshua Boniface 66bfad3109 Fix linting errors F522/F523 unused args 2021-11-06 03:24:50 -04:00
Joshua Boniface c41664d2da Reformat code with Black code formatter
Unify the code style along PEP and Black principles using the tool.
2021-11-06 03:02:43 -04:00
Joshua Boniface e88147db4a Bump version to 0.9.42 2021-10-12 15:25:42 -04:00
Joshua Boniface f13cc04b89 Bump version to 0.9.41 2021-10-09 19:39:21 -04:00
Joshua Boniface 24de0f4189 Add MTU to network creation/modification
Addresses #144
2021-10-09 17:51:32 -04:00
Joshua Boniface c27359c4bf Bump version to 0.9.40 2021-10-07 14:42:04 -04:00
Joshua Boniface bdb9db8375 Bump version to 0.9.39 2021-10-07 11:52:38 -04:00
Joshua Boniface c0f7ba0125 Add limit negation to VM list
When using the "state", "node", or "tag" arguments to a VM list, add
support for a "negate" flag to look for all VMs *not in* the state,
node, or tag state.
2021-10-07 11:50:52 -04:00
Joshua Boniface da9248cfa2 Bump version to 0.9.38 2021-10-03 22:32:41 -04:00
Joshua Boniface 32b3af697c Tweak fio tests for benchmarks
1. Remove ramp_time as this was giving very strange results.

2. Up the runtime to 75 seconds to compensate.

3. Print the fio command to the console to validate.
2021-10-03 15:06:18 -04:00
Joshua Boniface 0ff9a6b8c4 Handle benchmark running state properly 2021-10-02 01:54:51 -04:00
Joshua Boniface 28377178d2 Fix missing argument in database insert 2021-10-02 01:49:47 -04:00
Joshua Boniface e06b114c48 Update to storage benchmark format 1
1. Runs `fio` with the `--format=json` option and removes all terse
format parsing from the results.

2. Adds a 15-second ramp time to minimize wonky ramp-up results.

3. Sets group_reporting, which isn't necessary with only a single job,
but is here for consistency.
2021-10-02 01:41:08 -04:00
Joshua Boniface f83a345bfe Add test format versioning to storage benchmarks
Adds a test_format database column and a value in the API return for the
test format version, starting at 0 for the existing format as of 0.9.37.

References #143
2021-10-02 00:55:27 -04:00
Joshua Boniface ce06e4d81b Load benchmark results as JSON
Load the JSON at the API side instead of client side, because that's
what the API doc says it is and it just makes more sense.
2021-09-30 23:40:24 -04:00
Joshua Boniface 23977b04fc Bump version to 0.9.37 2021-09-30 02:08:14 -04:00