Commit Graph

268 Commits

Author SHA1 Message Date
Joshua Boniface b4c2b9bdf8 Lower sync and init tick limits
Lower both of these to 5 seconds to ensure Zookeeper doesn't linger on
startup or synchronization while pvcnoded is starting up (15s in
0.9.101).
2024-10-15 11:11:10 -04:00
Joshua Boniface 31728c0915 Fix unsafe conditional 2024-09-03 21:24:56 -04:00
Joshua Boniface 805477b8be Ignore more errors for user: module 2024-08-29 01:08:44 -04:00
Joshua Boniface e35842675f Add new is-primary condition to autobackup timers
Avoids running these except on the primary coordinator at the time of
firing, to avoid duplicate runs being passed to pvcworkerd.
2024-08-25 22:18:20 -04:00
Joshua Boniface cc2fb44689 Lower default pgnum and pgpnum to 32 2024-03-27 11:11:50 -04:00
Joshua Boniface fcb4096f19 Ensure directory is created first 2024-03-07 13:34:23 -05:00
Joshua Boniface 5a3233a905 Fix shutdown bug with keydb 2024-03-07 13:21:10 -05:00
Joshua Boniface db8c437a31 Add email reporting for autobackups (0.9.90) 2024-01-11 00:32:38 -05:00
Joshua Boniface 65dce44f72 Ignore errors when disabling units 2024-01-09 12:07:45 -05:00
Joshua Boniface b7f1c55279 Update and add default vars for autobackup 2024-01-09 09:54:38 -05:00
Joshua Boniface f6878c69ad Add enable_prometheus switch 2023-12-29 09:29:29 -05:00
Joshua Boniface 5089aec178 Use node names rather than IPs 2023-12-28 11:54:33 -05:00
Joshua Boniface ae4237c173 Use all coordinators 2023-12-28 11:47:38 -05:00
Joshua Boniface ce14cdb938 Rename file for consistency 2023-12-28 11:38:33 -05:00
Joshua Boniface 3be0da1e21 Fix incorrect source template path 2023-12-28 11:36:39 -05:00
Joshua Boniface 687d8f31bb Ignore errors when disabling sockets 2023-12-27 15:35:17 -05:00
Joshua Boniface d45bbad17a Only install libvirt exporter on Debian 11+ 2023-12-27 13:57:21 -05:00
Joshua Boniface fc1556c16c Fix incorrect flag 2023-12-14 14:05:45 -05:00
Joshua Boniface fdf856c53d Add creation of PVC directory 2023-12-14 12:53:11 -05:00
Joshua Boniface 9842f628a3 Use GO zkcli instead of trash BASH one
With Debian 12+ the interactive zkCli.sh is completely broken, failing
to load the JLine module regardless of what is done, and no one in the
project seems to care about that.

Instead let's swap this out for the (non-interactive) GO-based zkcli of
https://github.com/go-zkcli/zkcli instead, as this suits our needs
nicely.
2023-12-13 15:48:12 -05:00
Joshua Boniface b2a7eecf2c Only set exclude_perf_counts on Debian 12+ 2023-12-10 12:43:34 -05:00
Joshua Boniface 09eba9de80 Add Prometheus Zookeeper exporter 2023-12-10 12:39:35 -05:00
Joshua Boniface 1eb618e8f3 Fix typo 2023-12-10 03:15:02 -05:00
Joshua Boniface 1f34427a0c Add user as well 2023-12-10 03:05:11 -05:00
Joshua Boniface c7d914696b Add configuration for Postgres Prom exporter 2023-12-10 02:53:11 -05:00
Joshua Boniface 258b5792da Add Libvirt Prometheus exporter 2023-12-10 01:42:09 -05:00
Joshua Boniface 7a2c16e2a5 Disable Zookeeper Prometheus export
This does not work as this functionality is built out of the Debian
packages. Hopefully it will be enabled one day...
2023-12-10 01:41:06 -05:00
Joshua Boniface 23bb3cb4d9 Add JVM info to Zookepeer metric 2023-12-10 01:16:44 -05:00
Joshua Boniface 54bf70d336 Enable Prometheus metrics in Zookeeper too 2023-12-10 00:32:37 -05:00
Joshua Boniface 513313d60f Limit FRR Prom exporter to 12+
Package did not exist on Debian 10/11
2023-12-10 00:31:21 -05:00
Joshua Boniface bf10ede298 Add additional Prometheus exporters 2023-12-10 00:24:19 -05:00
Joshua Boniface 9e21aecf97 Ignore errors then check for PVC package
This helps work around apt issues when running from the oneshot
update-pvc-daemons playbook. On a new install, this will be OK. On an
upgrade, the apt tasks will fail OK but then the verification that
pvc-client-cli will ensure that things are actually sane before
proceeding.
2023-12-09 23:56:34 -05:00
Joshua Boniface 865c7d0872 Add Ceph Prometheus configurations (0.9.84) 2023-12-09 23:51:29 -05:00
Joshua Boniface 18054c01a0 Remove obsolete config templates 2023-12-09 23:05:16 -05:00
Joshua Boniface 5111ae47c4 Lower default monitoring interval to 15s
Faults are also reported on the monitoring interval, so 60s seems like
too long. Lower this to 15 seconds by default instead.
2023-12-01 16:06:24 -05:00
Joshua Boniface 73ad2a7751 Avoid removal of old versions at all
We simply shouldn't do this here. Let's leave them hanging around unless
removed in another way (e.g. in update-pvc-daemons)
2023-12-01 02:08:22 -05:00
Joshua Boniface 97b1469a70 Set ownership of pvc.conf 2023-12-01 01:57:56 -05:00
Joshua Boniface d59c9ce1ea Add safety to removal of legacy configs
This conditional will ensure that, the first time pvc.conf is installed
(or, subsequent times, until it stabilizes), the legacy configs will not
be removed. Then, on the next run in which pvc.conf does not change,
they will be removed.

This should provide a safety valve during a 0.9.83 update with the
update-pvc-daemons playbook: if the update succeeds, on the next run,
the legacy configs will be purged; otherwise, they will still be present
and can be used for fallback just in case.

This probably isn't needed, but just in case I'd rather be safe.
2023-12-01 01:45:48 -05:00
Joshua Boniface 1cfda69e5e Remove autobackup.yaml and fix quoting 2023-12-01 01:43:14 -05:00
Joshua Boniface 9408bf709c Only install pvcapid on coordinators
There should be no reason for the API to be installed on non-coordinator
hosts, so separate it out.
2023-12-01 01:40:56 -05:00
Joshua Boniface 9d2af41d3f Install new packages and remove old confs 2023-11-30 03:29:24 -05:00
Joshua Boniface b1d6915cf4 Write new pvc.conf style configuration (0.9.82+) 2023-11-28 16:10:23 -05:00
Joshua Boniface c8764159f6 Readd queue configuration with updated options 2023-11-05 23:37:49 -05:00
Joshua Boniface 523f7da71e Add KeyDB (Redis clone) to configuration
Replaces Redis for PVC >= 0.9.81
2023-11-05 19:24:30 -05:00
Joshua Boniface 2f9603c82f Adjust pvcapid.yaml for 0.9.81 worker queue config 2023-11-04 12:52:21 -04:00
Joshua Boniface e15e2dfaab Remove erroneous netmask from floating IP defaults 2023-11-04 12:51:07 -04:00
Joshua Boniface 103e9fe147 Add restart overrides for ceph-mgr
Needed because ceph-mgr seems to crash frequently under Debian 12 when
adding or removing OSDs. The default settings do not restart it
properly, so this override does.
2023-11-03 14:25:31 -04:00
Joshua Boniface 15a5b581f1 Disable failing socket services 2023-11-03 12:10:19 -04:00
Joshua Boniface 90417621d7 Add autobackup support to pvc-ansible 2023-10-27 02:08:20 -04:00
Joshua Boniface 17f819ea3f Don't set "latest" for libvirt packages
Avoids errors during runs before upgrades.
2023-10-24 10:41:47 -04:00