Commit Graph

503 Commits

Author SHA1 Message Date
Joshua Boniface d7f40ba1aa Move snapshot age conversion to client and round
Moves the age human conversion logic to the client so that this value
can be used by API consumers programatically.

Rounding ensures recent snapshots are not showing as "older" than they
actually are, which is important for accuracy especially with
automirror snapshots and ages above half the next rounding value.
2024-11-16 13:50:55 -05:00
Joshua Boniface 078d48a50b Add VM automirror support
Allows shipping snapshots automatically to remote clusters on a cron,
identically to how autobackup handles local snapshot exports.

VMs are selected based on configured tags, and individual destination
clusters can be specified based on a colon-separated suffix to the
tag(s).

Automirror snapshots use the prefix "am" (analogous to "ab" for
autobackups) to differentiate them from normal "mr" mirrors.
2024-11-15 11:34:39 -05:00
Joshua Boniface cebc660fb0 Allow image uploads if file_size is None 2024-11-14 10:05:35 -05:00
Joshua Boniface 4187aacc5b Correct formatting of OpenAPI Swagger specs 2024-10-19 02:23:46 -04:00
Joshua Boniface 861fef91e3 Add modification of Monitor hosts on XML import
Missing this means clusters with different storage hosts would fail to
start silently. Ensure these are updated like the secret UUID is as
well.
2024-10-16 16:00:54 -04:00
Joshua Boniface 35153cd6b6 Fix path handling for zkhandler
Using full paths broke the local schema generator, so convert these to
proper class instance methods and use them along with a new default +
settable override.
2024-10-11 16:03:40 -04:00
Joshua Boniface 9a91767405 Add proper return codes to API handlers 2024-10-11 14:43:44 -04:00
Joshua Boniface 96cebfb42a Handle cross-cluster Ceph storage secrets 2024-10-10 00:47:50 -04:00
Joshua Boniface ac00f7c4c8 Fix boolean state of remove_on_source 2024-10-09 01:04:08 -04:00
Joshua Boniface 1d90b066bc Add guard rails against manipulating mirrors
Snapshot mirrors should normally be promoted using "mirror promote", and
not started manually. This adds guard rails against that to the "start",
"stop", and "disable" state commands to prevent changing mirror states
without an explicit "--force" option.
2024-10-08 23:51:48 -04:00
Joshua Boniface 3ea7421f09 Implement friendlier VM mirror commands
Adds two helper commands which automate sending and promoting VM
snapshots as "vm mirror" commands.

"vm mirror create" replicates the functionality of "snapshot create" and
"snapshot send", performing both in one single task using an
autogenerated dated snapshot name for automatic cross-cluster
replication.

"vm mirror promote" replicates the functionality of "vm shutdown",
"snapshot create", "snapshot send", "vm start" (remote), and,
optionally, "vm remove", performing in one single task an entire
cross-cluster VM move with or without retaining the copy on the local
cluster (if retained, the local copy becomes a snapshot mirror of the
remote, flipping their statuses).
2024-10-08 23:51:39 -04:00
Joshua Boniface df4d437d31 Update the description of VM define endpoint 2024-10-01 13:30:44 -04:00
Joshua Boniface 8295e2089d Add proper response schema for 202 responses 2024-10-01 13:25:11 -04:00
Joshua Boniface 9aa32134a9 Fix bug in API specification 2024-09-30 20:51:49 -04:00
Joshua Boniface 75eac356d5 Increase send blocksize and add total speed
It's much faster and seems to cause no issues.
2024-09-30 20:11:12 -04:00
Joshua Boniface fb8561cc5d Actually fix incremental sending 2024-09-30 17:00:18 -04:00
Joshua Boniface 7fac7a62cf Clean up debug print statements 2024-09-30 03:51:39 -04:00
Joshua Boniface 7785166a7e Finish working implementation of send/receive
Required some significant refactoring due to issues with the diff send,
but it works.
2024-09-30 02:53:23 -04:00
Joshua Boniface 34f0a2f388 Add mostly complete implementation of VM send 2024-09-29 01:31:13 -04:00
Joshua Boniface f462ebbc6b Add VM snapshot send (initial) 2024-09-28 10:49:35 -04:00
Joshua Boniface 792d135950 Update responses for Celery tasks 2024-09-28 02:01:56 -04:00
Joshua Boniface a64e0c1985 Fix incorrect default value typos 2024-09-28 02:01:56 -04:00
Joshua Boniface 1cbadb1172 Add "mirror" VM state 2024-09-28 02:01:56 -04:00
Joshua Boniface b1c4b2e928 Add Ceph block receive (initial) 2024-09-28 02:01:56 -04:00
Joshua Boniface 41cd34ba4d Allow specifying job names for benchmarks 2024-09-18 14:55:12 -04:00
Joshua Boniface 736762901c Update benchmarks to include resource utilization
Adds additional polled information on node cpu, memory, and network
bandwidth for the node running the test. This should provide additional
useful information about the results of the test.

Also bumps the test format to 2 to ensure clients can handle the changes
properly.
2024-09-18 14:32:03 -04:00
Joshua Boniface ecb812ccac Update linting for pvcapid recent changes 2024-09-18 10:18:50 -04:00
Joshua Boniface a2e5df9f6d Add support for Gunicorn execution
Modifies pvcapid to run under Gunicorn when in non-debug mode, instead
of the Flask development server. This is proper practice for one, and
also helps increase performance slightly in some workloads (file uploads
mainly).
2024-09-09 13:20:03 -04:00
Joshua Boniface 2de999c700 Add total cluster utilization stats
Useful for evaluating the cluster resources as a whole.
2024-09-05 16:05:33 -04:00
Joshua Boniface 7543eb839d Add dedicated volume scan endpoint
Allows an imported volume to be scanned for stats independently.

Designed to be used as part of a snapshot import via API, to allow the
"create" to happen before the real import (to check for available space,
etc.) and then run this import after when the RBD volume actually
exists.
2024-09-03 20:32:27 -04:00
Joshua Boniface 8cb44c0c5d Bump version to 0.9.100 2024-08-30 11:03:33 -04:00
Joshua Boniface 02a775c99b Bump version to 0.9.99 2024-08-28 11:15:55 -04:00
Joshua Boniface fd87a28eb3 Fix bug in API parameters 2024-08-25 19:13:31 -04:00
Joshua Boniface de58efdaa9 Ensure email_recipients is always a list 2024-08-25 16:18:19 -04:00
Joshua Boniface 0d918d66fe Port VM autobackups into pvcworkerd with snaps
Moves VM autobackups from being in-CLI to being handled by the
pvcworkerd system on the primary coordinator. Turns the CLI autobackup
command into an actual API client endpoint rather than having its logic
in the CLI.

In addition, modifies the new autobackup to leverage the new "pvc vm
snapshot" function set, just with special snapshot names. This helps
automate this within the new snapshot scaffolding.
2024-08-23 17:23:06 -04:00
Joshua Boniface f63c392ba6 Show primary status in node run_on 2024-08-20 17:32:33 -04:00
Joshua Boniface 9a661d0173 Convert VM snapshots to worker tasks
Improves manageability and offloads these from the API context.
2024-08-20 16:50:41 -04:00
Joshua Boniface 7cc354466f Finish implementing snapshot import 2024-08-20 11:25:09 -04:00
Joshua Boniface 44232fe3c6 Fix export swagger definition 2024-08-20 11:07:56 -04:00
Joshua Boniface faf920ac1d Fix bug where force_flag is a string 2024-08-20 10:10:33 -04:00
Joshua Boniface d060787503 Add initial implementation of snapshot export 2024-08-19 18:46:07 -04:00
Joshua Boniface 0cf229273a Add API endpoint for current primary node
This was never exposed before, so expose it for use in other functions
being built.
2024-08-19 17:15:52 -04:00
Joshua Boniface 212ecaab68 Fix Swagger doc issues 2024-08-19 16:56:18 -04:00
Joshua Boniface 33f905459a Implement VM rollback
Closes #184
2024-08-16 10:47:18 -04:00
Joshua Boniface fbd5b3cca3 Remove is_backup flag for snapshots
This won't be needed for anything.
2024-08-16 10:46:25 -04:00
Joshua Boniface 6fc7f45027 Add snapshot lists and timestamp
Adds snapshots to the list of data in VM objects
2024-08-16 10:46:25 -04:00
Joshua Boniface 0c240a5129 Add VM snapshot removal 2024-08-16 10:46:25 -04:00
Joshua Boniface 553c1e670e Add VM snapshots functionality
Adds the ability to create snapshots of an entire VM, including all its
RBD disks and the VM XML config, though not any PVC metadata.
2024-08-16 10:46:25 -04:00
Joshua Boniface 1aa5999109 Bump version to 0.9.98 2024-06-05 12:01:31 -04:00
Joshua Boniface 5d0e7931d1 Add support for rolling back snapshots
We supported creating snapshots, but not doing anything with them. This
removes the manual task of restoring a snapshot and replace it with a
PVC abstraction of rolling back to a snapshot.

While Ceph recommends cloning a snapshot instead of rolling back, due to
the time taken, in our usecase I don't think that is an optimal
strategy, as it will leave dangling clones that we'd then have to
manage.

Closes #183
2024-05-13 15:24:51 -04:00