Commit Graph

730 Commits

Author SHA1 Message Date
Joshua Boniface 794cea4a02 Reverse ordering, run checks before starting timer 2023-09-15 22:25:37 -04:00
Joshua Boniface fa24f3ba75 Fix bad fstring in psur check 2023-09-15 22:19:49 -04:00
Joshua Boniface caadafa80d Add PSU redundancy sensor check 2023-09-15 19:07:29 -04:00
Joshua Boniface 479e156234 Run monitoring plugins once on startup 2023-09-15 17:53:16 -04:00
Joshua Boniface 86830286f3 Adjust message printing to be on one line 2023-09-15 17:00:34 -04:00
Joshua Boniface 4d51318a40 Make monitoring interval configurable 2023-09-15 16:54:51 -04:00
Joshua Boniface cba6f5be48 Fix wording of non-coordinator state 2023-09-15 16:51:04 -04:00
Joshua Boniface 254303b9d4 Use coordinator_state instead of router_state
Makes it much clearer what this variable represents.
2023-09-15 16:47:56 -04:00
Joshua Boniface 40b7d68853 Separate monitoring and move to 60s interval
Removes the dependency of the monitoring subsystem from the node
keepalives, and runs them at a 60s interval to avoid excessive backups
if a plugin takes too long.

Adds its own logs and related items as required.

Finally adds a new required argument to the run() of plugins, the
coordinator state, which can be used by a plugin to determine actions
based on whether the node is a primary, secondary, or non-coordinator.
2023-09-15 16:47:11 -04:00
Joshua Boniface a8115cafd1 Bump version to 0.9.73 2023-09-02 02:16:19 -04:00
Joshua Boniface 570da99605 Avoid failures if no children found 2023-09-02 01:36:17 -04:00
Joshua Boniface fdda47e8a2 Bump version to 0.9.72 2023-09-01 16:34:45 -04:00
Joshua Boniface bb2aac145d Bump version to 0.9.71 2023-09-01 00:36:38 -04:00
Joshua Boniface 6c407d54c3 Bump version to 0.9.70 2023-08-31 14:15:54 -04:00
Joshua Boniface cb413e5ce6 [Bookworm] Fix Ceph 16 OSD stat parsing 2023-08-31 00:45:03 -04:00
Joshua Boniface 123499f75f [Bookworm] Specify YAML loader explicitly 2023-08-31 00:16:19 -04:00
Joshua Boniface 83b8ce7b62 Bump version to 0.9.69 (nice) 2023-08-29 22:02:13 -04:00
Joshua Boniface 5e43f9bd7c Ensure Patroni failures do not block takeover 2023-08-29 22:00:11 -04:00
Joshua Boniface ed087d83c2 Found cpuload to 2 decimal places 2023-08-29 21:41:44 -04:00
Joshua Boniface 83d475bd15 Bump version to 0.9.68 2023-08-27 20:59:23 -04:00
Joshua Boniface 705ec802a3 Bump version to 0.9.67 2023-08-27 14:47:20 -04:00
Joshua Boniface 0b90f37518 Bump version to 0.9.66 2023-08-27 11:41:22 -04:00
Joshua Boniface 1e083d7652 Bump version to 0.9.65 2023-08-23 01:56:57 -04:00
Joshua Boniface 075dbe7cc9 Bump version to 0.9.64 2023-08-18 12:34:27 -04:00
Joshua Boniface b5f996febd Fix bugs for node flush for stop/shutdown/restart
Previously VMs in stop/shutdown/restart states wouldn't be properly
handled during a node flush. This fixes the bugs and ensures that the
transient VM states (shutdown/restart) are completed before proceeding,
and then avoids setting a stopped/shutdown VM to shutdown/auotstart.
2023-08-18 11:25:59 -04:00
Joshua Boniface 3a90fda109 Bump version to 0.9.63 2023-04-28 14:47:04 -04:00
Joshua Boniface 9114255af5 Add *.update-* obsolete configs to dpkg plugin 2023-04-10 15:39:40 -04:00
Joshua Boniface 2c3a3cdf52 Use try when watching health value in NodeInstance 2023-03-07 09:53:01 -05:00
Joshua Boniface 0b583bfdaf Bump IPMI timeout to 2 seconds 2023-03-07 09:25:27 -05:00
Joshua Boniface 7c07fbefff Adjust keepalive health printing and ordering 2023-02-24 11:08:30 -05:00
Joshua Boniface 202dc3ed59 Correct error handling if monitoring plugins fail 2023-02-24 10:19:41 -05:00
Joshua Boniface 4c2d99f8a6 Fix bug with SMART info 2023-02-23 13:21:23 -05:00
Joshua Boniface bcff6650d0 Set timeout on IPMI command 2023-02-23 11:10:09 -05:00
Joshua Boniface a11206253d Fix ZK check location 2023-02-23 11:04:02 -05:00
Joshua Boniface 45ad3b9a17 Bump version to 0.9.62 2023-02-22 18:13:45 -05:00
Joshua Boniface dc4e56db4b Add IPMI monitoring check 2023-02-22 15:02:08 -05:00
Joshua Boniface e45b3108a2 Add health delta change to message output 2023-02-22 15:02:08 -05:00
Joshua Boniface 118237a53b Fix bad string value for message 2023-02-22 15:02:08 -05:00
Joshua Boniface 9805681f94 Use consistent connection with other checks 2023-02-22 15:02:08 -05:00
Joshua Boniface 6c9abb2abe Add Libvirtd monitoring check 2023-02-22 15:02:08 -05:00
Joshua Boniface a1122c6e71 Add Zookeeper monitoring check 2023-02-22 15:02:08 -05:00
Joshua Boniface 3696f81597 Add PostgreSQL monitoring check 2023-02-22 15:02:08 -05:00
Joshua Boniface 5ca0d903b6 Adjust comment message 2023-02-22 15:02:08 -05:00
Joshua Boniface 626424b74a Adjust Munin threshold values 2023-02-22 10:42:43 -05:00
Joshua Boniface c9ceb3159b Remove obsolete LINKSPEED variable 2023-02-22 01:04:25 -05:00
Joshua Boniface 6525a2568b Adjust health delta of load to 50
This is a very bad situation and should be critical.
2023-02-22 01:03:12 -05:00
Joshua Boniface 09a005d3d7 Adjust health delta of EDAC Uncorrected to 50
This is a very bad situation and should be critical.
2023-02-22 01:01:54 -05:00
Joshua Boniface fb0fcc0597 Update readme for Munin plugin 2023-02-18 00:00:04 -05:00
Joshua Boniface 3009f24910 Fix typo in var and flip conditional 2023-02-17 16:18:42 -05:00
Joshua Boniface 5ae836f1c5 Fix various issues with PVC Munin plugin 2023-02-17 15:41:16 -05:00