Joshua Boniface
102c3c3106
Port all Celery worker functions to discrete pkg
...
Moves all tasks run by the Celery worker into a discrete package/module
for easier installation. Also adjusts several parameters throughout to
accomplish this.
2023-11-30 02:24:54 -05:00
Joshua Boniface
03a738f878
Move config parser into daemon_lib
...
And reformat/add config values for API.
2023-11-30 00:05:37 -05:00
Joshua Boniface
11db3c5b20
Fix ordering during termination
2023-11-29 21:21:51 -05:00
Joshua Boniface
fa12a3c9b1
Permit buffered log appending
2023-11-29 21:21:51 -05:00
Joshua Boniface
787f4216b3
Expand Zookeeper log daemon prefix to match
2023-11-29 21:21:51 -05:00
Joshua Boniface
83ceb41138
Add daemon name to Logger entries
2023-11-29 15:18:37 -05:00
Joshua Boniface
2e5958640a
Remove erroneous time from message
2023-11-29 15:12:41 -05:00
Joshua Boniface
7abc697c8a
Improve Zookeeper log handling
...
Ensures that messages are fully read before each append. Adds more
Zookeeper hits, but ensures logs won't be overwritten by multiple
daemons.
Also don't use a set on the client side, to avoid "removing duplicate"
entries erroneously.
2023-11-29 15:12:41 -05:00
Joshua Boniface
dd6a38d5ea
Properly pass the name of the exception
2023-11-16 18:05:52 -05:00
Joshua Boniface
f50f170d4e
Convert vmbuilder to use new Celery step structure
2023-11-16 16:08:49 -05:00
Joshua Boniface
83c4c6633d
Readd RBD lock detection and clearing on startup
...
This is still needed due to the nature of the locks and freeing them on
startup, and to preserve lock=fail behaviour on VM startup.
Also fixes the fencing lock flush to directly use the client library
outside of Celery. I don't like this hack but it seems prudent until we
move fencing to the workers as well.
2023-11-10 01:33:48 -05:00
Joshua Boniface
b522306f87
Increase Celery wait times
...
It's a bit inefficient, but provides nicer output and a bit of settling
time between each stage.
2023-11-09 23:54:05 -05:00
Joshua Boniface
07026efb63
Ensure OSD checks in before completing
...
Avoids issues where the new OSD doesn't check in; at least the
administrator will know.
Also fixes some issues with osd_db in removal.
2023-11-09 23:51:05 -05:00
Joshua Boniface
08411708f6
Clean up dangling references to cmd pipes
...
Also removes the schema references for these CMD pipes as they are no
longer required.
2023-11-09 23:28:14 -05:00
Joshua Boniface
ce17c60a20
Port OSD on-node tasks to Celery worker system
...
Adds Celery versions of the osd_add, osd_replace, osd_refresh,
osd_remove, and osd_db_vg_add functions.
2023-11-09 23:28:08 -05:00
Joshua Boniface
89681d54b9
Port VM on-node tasks to Celery worker system
...
Adds Celery versions of the flush_locks, device_attach, and
device_detach functions.
2023-11-06 20:40:46 -05:00
Joshua Boniface
a016337f57
Remove block verify in APi
...
This doesn't work right and is handled by the node anyways.
2023-11-04 02:45:10 -04:00
Joshua Boniface
7f5dd385b5
Use right key for FSID elsewhere
2023-11-03 23:51:01 -04:00
Joshua Boniface
ec42b19d0e
Send FSID to clients too
2023-11-03 16:37:55 -04:00
Joshua Boniface
64e37ae963
Update OSD replacement functionality
...
1. Simplify this by leveraging the existing remove_osd/add_osd
functions, since its task was functionally identical to those two in
sequential order.
2. Add support for split OSDs within the command (replacing all OSDs on
the block device(s) as required).
3. Add additional configurability and flexibility around the old device,
weight, and external DB LVs.
2023-11-03 01:45:49 -04:00
Joshua Boniface
980ea6a9e9
Adjust handling of ext_db and _count options
...
Avoid the use of superfluous flag options, default them to none, and add
support for fixed-size DB LVs.
2023-11-02 13:29:47 -04:00
Joshua Boniface
526a5f4a74
Add support for split OSD adds
...
Allows creating multiple OSDs on a single (NVMe) block device,
leveraging the "ceph-volume lvm batch" command. Replaces the previous
method of creating OSDs.
Also adds a new ZK item for each OSD indicating if it is split or not.
2023-11-01 21:31:35 -04:00
Joshua Boniface
35f80e544c
Use more hierarchical backup path structure
2023-10-24 02:04:16 -04:00
Joshua Boniface
83b937654c
Avoid removing nonexistent snapshots
...
Store retain_snapshot in JSON and use that to check during delete.
2023-10-24 01:35:00 -04:00
Joshua Boniface
714bde89e6
Fix incorrect variable ref
2023-10-24 01:25:01 -04:00
Joshua Boniface
c87736eb0a
Use consistent path name and format
2023-10-24 01:20:44 -04:00
Joshua Boniface
63d0a85e29
Add backup deletion command
2023-10-24 01:18:27 -04:00
Joshua Boniface
55ca131c2c
Handle snapshots on restore and provide options
...
Also rename the retain option to remove superfluous plural.
2023-10-24 00:25:06 -04:00
Joshua Boniface
8d256a1737
Complete VM restore functionality
2023-10-23 22:23:17 -04:00
Joshua Boniface
d3b3fdfc80
Revert "Export backup images to a tar archive"
...
This reverts commit 38abd078af
2023-10-23 11:01:16 -04:00
Joshua Boniface
f1b29ea94e
Initial VM restore work
2023-10-23 11:00:54 -04:00
Joshua Boniface
38abd078af
Export backup images to a tar archive
...
This helps ensure an easier restore as the tar archive(s) can be sent
directly to the API via the normal process of image uploading, instead
of individual disks.
2023-10-23 09:56:50 -04:00
Joshua Boniface
fabb97cf48
Only split a command_string if its not a list
2023-10-23 09:50:58 -04:00
Joshua Boniface
68124db323
Remove extra spaces
2023-10-17 13:01:38 -04:00
Joshua Boniface
8921efd269
Fix incorrect tuple construct
2023-10-17 12:55:44 -04:00
Joshua Boniface
3d12915989
Further improve return messages
2023-10-17 12:53:08 -04:00
Joshua Boniface
67b0b19bca
Use better time functionality
2023-10-17 12:39:37 -04:00
Joshua Boniface
5d0c674d1d
Add runtime and adjust ordering
2023-10-17 12:32:40 -04:00
Joshua Boniface
f441b0d823
Improve missing parent message
2023-10-17 12:17:29 -04:00
Joshua Boniface
a5d0f219e4
Improve return messages
2023-10-17 12:10:55 -04:00
Joshua Boniface
0169510df0
Fix up datestring generation
2023-10-17 12:05:45 -04:00
Joshua Boniface
a58c1d5a8c
Fix bad snapshot removals
2023-10-17 12:02:24 -04:00
Joshua Boniface
a8e4b01b67
Handle return data even better
2023-10-17 11:51:03 -04:00
Joshua Boniface
45c4c86911
Handle extra return variable
2023-10-17 11:47:01 -04:00
Joshua Boniface
6448b31d2c
Improve VM list arguments
...
Use kwargs here instead of fixed args to allow default None values.
2023-10-17 11:01:38 -04:00
Joshua Boniface
b997c6f31e
Add support for full VM backups
...
Adds support for exporting full VM backups, including configuration,
metainfo, and RBD disk images, with incremental support.
2023-10-17 10:15:06 -04:00
Joshua Boniface
a0b45a2bcd
Always create RBDs with bytes value
...
Converting into human results in imprecise values when specifying bytes
directly, which in turn breaks VMDK image uploads. Instead, just use the
raw bytes value when creating the volume instead of converting it back.
2023-09-30 12:37:43 -04:00
Joshua Boniface
c4397219da
Ensure fencing states are properly reflected
2023-09-18 09:59:18 -04:00
Joshua Boniface
311bb69785
Format based on updated Black
2023-09-12 16:41:02 -04:00
Joshua Boniface
653b95ee25
Normalize return messages for node commands
2023-05-04 17:02:46 -04:00
Joshua Boniface