Commit Graph

3284 Commits

Author SHA1 Message Date
Joshua Boniface eab1ae873b Ensure upstream_gateway key will exist 2023-11-27 13:37:57 -05:00
Joshua Boniface eaf93cdf96 Readd missing subsystem configurations 2023-11-27 13:33:41 -05:00
Joshua Boniface c8f4cbb39e Fix node entry keys 2023-11-27 13:24:01 -05:00
Joshua Boniface 786fae7769 Improve logo output 2023-11-27 13:01:43 -05:00
Joshua Boniface 17f81e8296 Refactor pvcapid to use new configuration 2023-11-27 12:49:26 -05:00
Joshua Boniface bcc57638a9 Refactor pvcnoded to use new configuration 2023-11-26 15:41:25 -05:00
Joshua Boniface a593ee9c2e Reorganize and add more configuration items 2023-11-26 15:32:53 -05:00
Joshua Boniface 2666e0603e Update dnsmasq script to use new config file 2023-11-26 14:18:13 -05:00
Joshua Boniface dab7396196 Move to unified pvc.conf configuration file 2023-11-26 14:16:21 -05:00
Joshua Boniface 460a2dd09f Bump version to 0.9.82 2023-11-25 15:38:50 -05:00
Joshua Boniface c30ea66d14 Add swagger document to gitignore 2023-11-25 15:37:01 -05:00
Joshua Boniface 24cabd3b99 Fix missing result_backend on Debian 10/11
For whatever reason, a Celery worker on <5.2.x was not picking these up.
Move them back to the root of the module so they are properly picked up
on these older versions but still prevents calling the routing functions
during an API doc generation.
2023-11-25 15:35:25 -05:00
Joshua Boniface 3e001b08b6 Bump version to 0.9.81 2023-11-17 01:29:41 -05:00
Joshua Boniface 7f6b3ebb6b Update test script 2023-11-17 01:03:56 -05:00
Joshua Boniface 91858fbd20 Update manage script 2023-11-16 23:12:53 -05:00
Joshua Boniface b66cfb07d8 Isolate cluster-dependent Celery startup
Avoids calling unworkable functions when generating API docs etc. by
isolating them into a Celery startup function called by Daemon.py.

Also update to Celery 4+ settings format.
2023-11-16 20:32:29 -05:00
Joshua Boniface 9885914abd Remove stray periods from messages 2023-11-16 19:56:24 -05:00
Joshua Boniface e8da3714c0 Convert benchmark to use new Celery step structure 2023-11-16 19:36:23 -05:00
Joshua Boniface 4d23d0419c Fix total stage count 2023-11-16 18:41:43 -05:00
Joshua Boniface 7223073afc Don't show stderr here 2023-11-16 18:30:32 -05:00
Joshua Boniface c1c22c81e7 Ensure script cleanup is done in chroot 2023-11-16 18:27:23 -05:00
Joshua Boniface 712a50ca27 Avoid use of fail here
It causes a reraise with a bunch of extra entries that we don't need.
2023-11-16 18:22:59 -05:00
Joshua Boniface 815041ff20 Fix bugs when main installs fail 2023-11-16 18:20:26 -05:00
Joshua Boniface 823ce8cbf2 Remove duplicate cleanups 2023-11-16 18:19:05 -05:00
Joshua Boniface fca02238d7 Adjust starting text 2023-11-16 18:06:31 -05:00
Joshua Boniface dd6a38d5ea Properly pass the name of the exception 2023-11-16 18:05:52 -05:00
Joshua Boniface 73a4795967 Avoid fail during yields
This just causes a double-exception, so don't do it.
2023-11-16 17:22:53 -05:00
Joshua Boniface 2a637c62e8 Port provisioner scripts to updated framework
Updates all the example provisioner scripts to use the new functions
exposed by the VMBuilder class as an illustration of how best to use
them.

Also adds a wrapper fail() handler to ensure the cleanup of the script,
as well as the global cleanup, are run on an exception.
2023-11-16 17:04:46 -05:00
Joshua Boniface 618a1c1c10 Add helper functions to VMBuilder instances 2023-11-16 16:17:17 -05:00
Joshua Boniface f50f170d4e Convert vmbuilder to use new Celery step structure 2023-11-16 16:08:49 -05:00
Joshua Boniface e92ed245d6 Pass proper task details into wait function 2023-11-16 15:31:25 -05:00
Joshua Boniface 9ab505ec98 Return and show task_name 2023-11-16 14:50:02 -05:00
Joshua Boniface 9958d1cfe8 Add name to task output 2023-11-16 13:23:31 -05:00
Joshua Boniface 0cb81f96e6 Use custom task IDs for Celery tasks
Full UUIDs were obnoxiously long, so switch to using just the first
8-character section of a UUID instead. Keeps the list nice and short,
makes them easier to copy, and is just generally nicer.

Could this cause uniqueness problems? Perhaps, but I don't see that
happening nearly frequently enough to matter.
2023-11-16 13:22:14 -05:00
Joshua Boniface 3651885954 Add --events to workers 2023-11-16 12:35:54 -05:00
Joshua Boniface d226e9f4e5 Enable extended Celery results 2023-11-16 12:02:57 -05:00
Joshua Boniface fa361a55d9 Explicitly use kwargs in Celery task calls 2023-11-16 11:55:30 -05:00
Joshua Boniface 8915864fa9 Lower truncation size and add elipses 2023-11-16 11:47:36 -05:00
Joshua Boniface 79f7e8f82e Skip "run_on" argument in output
This isn't required to know, it's internal.
2023-11-16 11:46:15 -05:00
Joshua Boniface 0d818017e8 Name the celery workers pvcworkerd@<hostname> 2023-11-16 11:43:17 -05:00
Joshua Boniface eb1d61a8b9 Generalize task status output 2023-11-16 11:39:08 -05:00
Joshua Boniface 262babc63d Use kwargs for all task arguments
This will help ensure that the CLI frontend can properly parse the args
in a consistent way.
2023-11-16 10:10:48 -05:00
Joshua Boniface 63773a3061 Allow watching existing task via cluster task 2023-11-16 03:06:13 -05:00
Joshua Boniface 289049d223 Properly handle a "primary" run_on value 2023-11-16 02:49:29 -05:00
Joshua Boniface e818df5dae Use enable/disable --now instead of two commands
Avoids needing two calls here especially for the stop.
2023-11-16 02:40:35 -05:00
Joshua Boniface c76a5afd04 Avoid waits during node secondary
Waiting for the daemons to stop took too much time on some nodes and
could throw off the lockstep. Instead, leverage background=True to run
the systemctl os_commands in the background (when they complete is
irrelevant), stop the Metadata API first, and don't delay during its
stop at all.
2023-11-16 02:34:12 -05:00
Joshua Boniface 0bec6abe71 Return proper run_on for ported tasks 2023-11-16 02:28:57 -05:00
Joshua Boniface 18e43a9377 Adjust name in worker log output 2023-11-16 02:25:14 -05:00
Joshua Boniface 4555f5a20a Remove warnings when switch coordinator state
Tasks are no longer bound to the primary coordinator for state updates
due to using KeyDB and a proper shared queue and result backend, so this
warning is now obsolete and no longer required.

This would interrupt "--wait" commands on provisioner tasks, but we no
longer believe that this warrants a warning, as the affected user could
simply run "pvc cluster task" to validate or resume the watcher.
2023-11-16 02:15:01 -05:00
Joshua Boniface d727764ebc Remove obsolete status and add cluster task
Removes the obsoleted "pvc provisioner status" command and replaces it
with a generalized "pvc cluster task" command to show all
currently-active or pending tasks on the cluster workers.
2023-11-16 02:13:26 -05:00