Commit Graph

3239 Commits

Author SHA1 Message Date
Joshua Boniface efc7434143 Add safety check for 80% full size
Adds a check that a volume creation or resize won't violate the 80% full
rule for the storage cluster. This ensures a cluster won't get too full
if a storage volume fills up.

Also adds a force flag throughout the pipeline to override this check,
should an administrator really want to do so.

Closes #177
2024-02-02 11:37:00 -05:00
Joshua Boniface c473dcca81 Fix errors with autobackup email summary
How this was being done didn't work, as the backup volume was already
unmounted when we tried to read the backups from it. Instead, populate
the backup summary earlier in the run, during the actual backup.
2024-02-02 09:31:16 -05:00
Joshua Boniface 18f09196be Bump version to 0.9.93 2024-01-30 09:51:21 -05:00
Joshua Boniface 8419659e1b Ensure zkhandler is always cleaned up
Even if the subfunction of an API @ZKConnection call fails, the
zkhandler needs to terminate and clean up, or it leaves stuck threads
around.
2024-01-30 09:48:17 -05:00
Joshua Boniface df40b779af Bump version to 0.9.92 2024-01-29 09:39:10 -05:00
Joshua Boniface db4f0881a2 Improve error handling and retries
1. Use the actual response code from the server on error, or 504 on
timeouts instead of 500.
2. Retry GET requests 3 times and only error if the last fails
2024-01-29 09:35:14 -05:00
Joshua Boniface 9b51fe9f10 Use get() for newer keys in client 2024-01-29 09:21:02 -05:00
Joshua Boniface a66449541d Improve script error handling and variables 2024-01-26 15:41:34 -05:00
Joshua Boniface d28fb71f57 Fix incorrect variable set 2024-01-24 14:40:40 -05:00
Joshua Boniface e5e9c7086a Add missing restore state to colours 2024-01-24 09:34:59 -05:00
Joshua Boniface f29b4c2755 Bump version to 0.9.91 2024-01-23 10:40:59 -05:00
Joshua Boniface 0adec2be0d Use consistent and less error-prone find rm's 2024-01-23 10:40:48 -05:00
Joshua Boniface b994e1a26c Add cleanup of pycaches to CLI install 2024-01-23 10:22:50 -05:00
Joshua Boniface 6d6420a695 Add missing value to vm_define function 2024-01-23 09:58:32 -05:00
Joshua Boniface 94e0287fc4 Add missing modules to default cloud-init 2024-01-21 14:23:44 -05:00
Joshua Boniface 2886176762 Improve handling of task arg display
Shows each subarg of the task_args as its own element, if applicable,
and fits the width to the terminal using MAX_CONTENT_WIDTH instead of an
arbitrary value.
2024-01-18 13:00:48 -05:00
Joshua Boniface 4dc4c975f1 Add status messages during task query 2024-01-18 12:38:53 -05:00
Joshua Boniface 8f3120baf3 Avoid errors if task_status is a tuple 2024-01-18 12:30:31 -05:00
Joshua Boniface 86ca363697 Bump version to 0.9.90 2024-01-11 10:22:48 -05:00
Joshua Boniface a5763c9d25 Fix possible race condition applying schemas
Found an instance where two of these fired too close together, and
caused a fatal error. Use a write lock, and then catch the schema.apply
function in case it fails anyways.
2024-01-11 10:21:01 -05:00
Joshua Boniface 39ec427c42 Remove obsolete swagger files 2024-01-11 10:20:42 -05:00
Joshua Boniface 1ba21312ea Use flask manage for all versions
The legacy method does not work any longer, so just use the Flask manage
script and remove the obsolete legacy one.
2024-01-11 10:20:33 -05:00
Joshua Boniface 09269f182c Add live migrate max downtime selector meta field
Adds a new flag to VM metadata to allow setting the VM live migration
max downtime. This will enable very busy VMs that hang live migration to
have this value changed.
2024-01-11 00:05:50 -05:00
Joshua Boniface 38eeb78423 Improve output formatting for build-and-deploy
Also restore the pvcnoded restart that was erroneously removed.
2024-01-10 14:22:12 -05:00
Joshua Boniface 362edeed8c Add backup reporting and improve metrics
Major improvements to autobackup and backups, including additional
information/fields in the backup JSON itself, improved error handling,
and the ability to email reports of autobackups using a local sendmail
utility.
2024-01-10 14:18:44 -05:00
Joshua Boniface 8d74ee7273 Remove obsolete worker restart in pvcapid package 2024-01-10 13:26:14 -05:00
Joshua Boniface 39c8367723 Add additional metainfo to VM backups
Adds additional information about failures, runtime, file sizes, etc. to
the JSON output of a VM backup.

This helps enable additional reporting and summary information for
autobackup runs.
2024-01-10 10:37:29 -05:00
Joshua Boniface e9b6072fa0 Bump version to 0.9.89 2024-01-09 12:15:53 -05:00
Joshua Boniface aac306c55b Fix missing stats on old Debians 2024-01-09 12:10:16 -05:00
Joshua Boniface 3c458ca9a6 Fix broken result backend on old Celery 2024-01-09 12:06:23 -05:00
Joshua Boniface 1d480f5629 Bump version to 0.9.88 2023-12-29 14:56:33 -05:00
Joshua Boniface 138a982ffe Add tag to VMs dashboard 2023-12-29 14:56:24 -05:00
Joshua Boniface 29a740882b Update some graph positions and sizes 2023-12-29 14:37:00 -05:00
Joshua Boniface 04f6f7acab Add VM dashboard 2023-12-29 14:28:42 -05:00
Joshua Boniface c1ae571213 Add additional VM details to Prometheus 2023-12-29 14:09:39 -05:00
Joshua Boniface 88a377c1aa Update positions and IOPS field 2023-12-29 14:09:25 -05:00
Joshua Boniface 73bf256650 Update Prometheus readmes 2023-12-29 11:22:52 -05:00
Joshua Boniface 123c7ce857 Update copyright header on all files for 2024
Last release of 2023 is probably the best time to do this.
2023-12-29 11:16:59 -05:00
Joshua Boniface 3b3ffaf2d4 Add Prometheus file SD output to connection list
Allows an administrator to easily generate a Prometheus file service
discovery configuration via the CLI for all clusters they have
configured. Assumes that all the various connection details are correct,
and due to the limits of the file SD config does not include the scheme
or SSL verification options (as these are global in Prometheus).
2023-12-29 11:13:54 -05:00
Joshua Boniface 2309b9dcf0 Make final dashboard read-only 2023-12-29 10:31:48 -05:00
Joshua Boniface 51b9f062b7 Add descriptions for each panel and reset version. 2023-12-29 10:30:28 -05:00
Joshua Boniface e4ca74c201 Add Zookeeper performance to Grafana dashboard 2023-12-29 09:44:40 -05:00
Joshua Boniface 4969e90f8a Allow enable/disable of Prometheus endpoints
Since these are unauthenticated, it might be the case that an
administrator wishes to completely disable these metrics endpoints.
Provide that option via pvc.conf through pvc-ansible's existing
enable_prometheus_exporters option and the new enable_prometheus
configuration flag.

Defaults to "yes" to provide all functionality unless explicitly
disabled, as the author assumes that the PVC API is secured in other
ways as well and that metric information is not completely sensitive.
2023-12-29 09:25:10 -05:00
Joshua Boniface 52f68909f6 Update Grafana dashboard 2023-12-28 14:55:43 -05:00
Joshua Boniface 0bcf8cfe19 Add Zookeeper metrics proxy 2023-12-28 13:53:15 -05:00
Joshua Boniface 2bb24d3b57 Update Prometheus dashboard and add README 2023-12-27 15:57:12 -05:00
Joshua Boniface 8083b7a3e6 Bump version to 0.9.87 2023-12-27 13:40:51 -05:00
Joshua Boniface 3346ce9bb0 Add missing shutdown state from combinations 2023-12-27 13:40:30 -05:00
Joshua Boniface 572596c575 Fix missing f-string placeholder 2023-12-27 13:21:20 -05:00
Joshua Boniface e654fbba08 Move debug condition handling to Logger
Avoids many dozens of conditionals sprinkled throughout the code by
centralizing this check into the main Logger instance.
2023-12-27 13:01:45 -05:00