Commit Graph

3329 Commits

Author SHA1 Message Date
Joshua Boniface 3ede0c7d38 Name mirror snapshots like autobackup snapshots 2024-10-09 00:49:22 -04:00
Joshua Boniface ab9390fdb8 Fix another bad stage counting instance 2024-10-09 00:44:20 -04:00
Joshua Boniface 1c83584788 Set correct verbage 2024-10-09 00:38:59 -04:00
Joshua Boniface 7f3ab4e119 Fix stage counting in tasks 2024-10-09 00:37:13 -04:00
Joshua Boniface 16eb09dc22 Fix ordering bug with vm_detail 2024-10-09 00:33:00 -04:00
Joshua Boniface 7ba75adef4 Fix bug if destination is missing 2024-10-09 00:27:42 -04:00
Joshua Boniface a691d26c30 Add check for scheme in destination
Allows handling invalid cluster names properly.
2024-10-09 00:25:13 -04:00
Joshua Boniface 1d90b066bc Add guard rails against manipulating mirrors
Snapshot mirrors should normally be promoted using "mirror promote", and
not started manually. This adds guard rails against that to the "start",
"stop", and "disable" state commands to prevent changing mirror states
without an explicit "--force" option.
2024-10-08 23:51:48 -04:00
Joshua Boniface 3ea7421f09 Implement friendlier VM mirror commands
Adds two helper commands which automate sending and promoting VM
snapshots as "vm mirror" commands.

"vm mirror create" replicates the functionality of "snapshot create" and
"snapshot send", performing both in one single task using an
autogenerated dated snapshot name for automatic cross-cluster
replication.

"vm mirror promote" replicates the functionality of "vm shutdown",
"snapshot create", "snapshot send", "vm start" (remote), and,
optionally, "vm remove", performing in one single task an entire
cross-cluster VM move with or without retaining the copy on the local
cluster (if retained, the local copy becomes a snapshot mirror of the
remote, flipping their statuses).
2024-10-08 23:51:39 -04:00
Joshua Boniface df4d437d31 Update the description of VM define endpoint 2024-10-01 13:30:44 -04:00
Joshua Boniface 8295e2089d Add proper response schema for 202 responses 2024-10-01 13:25:11 -04:00
Joshua Boniface 4ccb570762 Enhance documentation of snapshot send command 2024-09-30 23:54:53 -04:00
Joshua Boniface 235299942a Add volume resize if changed 2024-09-30 20:51:59 -04:00
Joshua Boniface 9aa32134a9 Fix bug in API specification 2024-09-30 20:51:49 -04:00
Joshua Boniface 75eac356d5 Increase send blocksize and add total speed
It's much faster and seems to cause no issues.
2024-09-30 20:11:12 -04:00
Joshua Boniface fb8561cc5d Actually fix incremental sending 2024-09-30 17:00:18 -04:00
Joshua Boniface 5f7aa0b2d6 Improve incremental send speed 2024-09-30 04:15:17 -04:00
Joshua Boniface 7fac7a62cf Clean up debug print statements 2024-09-30 03:51:39 -04:00
Joshua Boniface b19642aa2e Fix bug where snapshot rollback was never called 2024-09-30 03:04:35 -04:00
Joshua Boniface 974e0d6ac2 Shorten progress bars to 20 characters
They were needlessly long and this limited the message size.
2024-09-30 03:04:10 -04:00
Joshua Boniface 7785166a7e Finish working implementation of send/receive
Required some significant refactoring due to issues with the diff send,
but it works.
2024-09-30 02:53:23 -04:00
Joshua Boniface 34f0a2f388 Add mostly complete implementation of VM send 2024-09-29 01:31:13 -04:00
Joshua Boniface 8fa37d21c0 Fix handling of invalid network lengths 2024-09-29 00:39:53 -04:00
Joshua Boniface f462ebbc6b Add VM snapshot send (initial) 2024-09-28 10:49:35 -04:00
Joshua Boniface 0d533f3658 Rework task output bar operation
Allows sending constant updates including changes to the message within
the same task.
2024-09-28 10:48:39 -04:00
Joshua Boniface 792d135950 Update responses for Celery tasks 2024-09-28 02:01:56 -04:00
Joshua Boniface a64e0c1985 Fix incorrect default value typos 2024-09-28 02:01:56 -04:00
Joshua Boniface 1cbadb1172 Add "mirror" VM state 2024-09-28 02:01:56 -04:00
Joshua Boniface b1c4b2e928 Add Ceph block receive (initial) 2024-09-28 02:01:56 -04:00
Joshua Boniface 7fe1262887 Fix indentation in faults 2024-09-28 02:01:33 -04:00
Joshua Boniface 0e389ba1f4 Fix bug when setting split count = 1
Would set the OSD as split in Zookeeper, even though it wasn't.
2024-09-23 13:06:05 -04:00
Joshua Boniface 41cd34ba4d Allow specifying job names for benchmarks 2024-09-18 14:55:12 -04:00
Joshua Boniface 736762901c Update benchmarks to include resource utilization
Adds additional polled information on node cpu, memory, and network
bandwidth for the node running the test. This should provide additional
useful information about the results of the test.

Also bumps the test format to 2 to ensure clients can handle the changes
properly.
2024-09-18 14:32:03 -04:00
Joshua Boniface ecb812ccac Update linting for pvcapid recent changes 2024-09-18 10:18:50 -04:00
Joshua Boniface a2e5df9f6d Add support for Gunicorn execution
Modifies pvcapid to run under Gunicorn when in non-debug mode, instead
of the Flask development server. This is proper practice for one, and
also helps increase performance slightly in some workloads (file uploads
mainly).
2024-09-09 13:20:03 -04:00
Joshua Boniface 73c0834f85 Remove headers and add util to short output 2024-09-06 11:40:39 -04:00
Joshua Boniface 2de999c700 Add total cluster utilization stats
Useful for evaluating the cluster resources as a whole.
2024-09-05 16:05:33 -04:00
Joshua Boniface 7543eb839d Add dedicated volume scan endpoint
Allows an imported volume to be scanned for stats independently.

Designed to be used as part of a snapshot import via API, to allow the
"create" to happen before the real import (to check for available space,
etc.) and then run this import after when the RBD volume actually
exists.
2024-09-03 20:32:27 -04:00
Joshua Boniface 8cb44c0c5d Bump version to 0.9.100 2024-08-30 11:03:33 -04:00
Joshua Boniface c55021f30c Update information about detect strings in CLI 2024-08-30 11:02:44 -04:00
Joshua Boniface 783c9e46c2 Only add packages to bookworm repo
Deprecates Debian 10 (Buster) and 11 (Bullseye); those versions will not
receive PVC 0.9.100 or newer.
2024-08-30 10:56:24 -04:00
Joshua Boniface b7f33c1fcb Update deprecation warning
Hotfixes throw a wrench in this, so just make them generic.
2024-08-30 10:55:24 -04:00
Joshua Boniface 0f578d7c7d Ensure decimals are captured from size regex 2024-08-30 10:51:41 -04:00
Joshua Boniface f87b96887c Add detect string parser with nvme
Some newer servers do not report NVMe device paths properly using
`lsscsi` as expected. To work around this, add an `nvme`-based detect
parser that is called if the `lsscsi` parser returns a `-` (or None).
2024-08-30 10:41:56 -04:00
Joshua Boniface 02a775c99b Bump version to 0.9.99 2024-08-28 11:15:55 -04:00
Joshua Boniface 8177d5f8b7 Use absolute path for ZK schema 2024-08-27 09:40:24 -04:00
Joshua Boniface 26d0d08873 Add is-primary command
Used by the cron to check if the node firing an autobackup is the
primary node or not, so it will not multi-fire from all coordinators.
2024-08-25 22:09:03 -04:00
Joshua Boniface f57b8d4a15 Simplify Celery event handling
It was far too cumbersome to report every possible stage here in a
consistent way. Realistically, this command will be run silently from
cron 99.95% of the time, so all this overcomplexity to handle individual
Celery state updates just isn't worth it.
2024-08-25 21:59:12 -04:00
Joshua Boniface 10de85cce3 Allow API-only builds and deploy 2024-08-25 20:45:52 -04:00
Joshua Boniface e938140414 Refactor autobackups to make more sense 2024-08-25 19:21:00 -04:00