89681d54b9
Port VM on-node tasks to Celery worker system
...
Adds Celery versions of the flush_locks, device_attach, and
device_detach functions.
2023-11-06 20:40:46 -05:00
f0c2e9d295
Don't start pvcapid-worker on primary
...
It will be running anyways
2023-11-05 19:44:00 -05:00
2c15036f86
Add KeyDB to node startup services
...
Also ensure API worker starts on all nodes, not just coordinators.
2023-11-05 19:26:38 -05:00
42ed6f6420
Remove redis as a dependency
2023-11-05 18:23:34 -05:00
3dc1f57de2
Revert "Switch to ZK+PG over Redis for Celery queue"
...
This reverts commit 54215bab6c1c420bc284160c6c4633e0ab2e8ff2.
2023-11-05 17:10:46 -05:00
b99b4e64b2
Ensure store path is passed properly
2023-11-05 16:48:47 -05:00
91af1175ef
Fix missing CLI_CONFIG in echo()
2023-11-04 15:17:50 -04:00
af8a8d969e
Ensure queues are set up for non-coordinator nodes
...
Allows a runner to operate on every possible node, not just
coordinators, as OSDs or other things could be on any node.
Also add more comments.
2023-11-04 15:05:07 -04:00
a6caac1b78
Add Celery queue routing for tasks
...
By default, tasks will continue to run as they did, on the primary
coordinator's task runner. However this opens the possibility for
defining more tasks that will run on other nodes or coordinators.
2023-11-04 14:29:59 -04:00
30d7e49401
Start API worker with node daemon on coordinators
2023-11-04 13:08:16 -04:00
ab629f6b51
Use per-host hostname and queues in worker
...
Opens up the ability to direct tasks to specific workers.
2023-11-04 13:02:30 -04:00
54215bab6c
Switch to ZK+PG over Redis for Celery queue
...
Redis did not provide a distributed solution for the worker, which
precluded several important planned functions. So instead, move to using
Zookeeper + PostgreSQL as the broker and result backend respectively.
Should be a seamless drop-in change but for future uses requires the
database host to be the primary coordinator IP rather than localhost, so
that writes can occur to the database from non-primary hosts.
2023-11-04 12:46:34 -04:00
7490f13b7c
Check for partition tables on new devices
2023-11-04 03:13:58 -04:00
d1602f35de
Adjust split indicator
2023-11-04 02:56:21 -04:00
7cdedde2fb
Adjust wording about extdb
2023-11-04 02:54:25 -04:00
ab156b14b7
Update help messages for OSD refresh
2023-11-04 02:47:04 -04:00
a016337f57
Remove block verify in APi
...
This doesn't work right and is handled by the node anyways.
2023-11-04 02:45:10 -04:00
e32054be81
Refactor refresh as well
2023-11-04 02:44:52 -04:00
18d32fede3
Fix wording of detect strings
2023-11-04 01:37:07 -04:00
b3d13fe9be
Add log message for zap
2023-11-04 01:02:51 -04:00
48b2ccbd95
Add timeout for safe-to-destroy
...
Continuously take the OSD down and out while doing so.
2023-11-04 00:55:05 -04:00
1535078842
Fix lvremove, lvcreate, and update ZK details
2023-11-04 00:30:14 -04:00
0e45613634
Use right key with correct data
2023-11-04 00:02:00 -04:00
75135f6d5f
Avoid broken output format for new OSDs
2023-11-03 23:54:10 -04:00
7f5dd385b5
Use right key for FSID elsewhere
2023-11-03 23:51:01 -04:00
befce62925
Add OSD destroy before purge
2023-11-03 23:44:27 -04:00
b0909aed61
Get proper FSID value
2023-11-03 23:38:24 -04:00
f418b40527
Use proper FSID instead of hack
2023-11-03 16:38:19 -04:00
ec42b19d0e
Send FSID to clients too
2023-11-03 16:37:55 -04:00
dd0177ce10
Rework replacement procedure again
...
Avoid calling other functions; replicate the actual process from Ceph
docs (https://docs.ceph.com/en/pacific/rados/operations/add-or-rm-osds/ )
to ensure things work out well (e.g. preserving OSD IDs).
2023-11-03 16:31:56 -04:00
ed5bc9fb43
Fix numerous formatting and function bugs
2023-11-03 14:00:05 -04:00
94d8d2cf75
Fix skip_zap_flag anomaly and add crush rm
2023-11-03 02:35:12 -04:00
20497cf89d
Fix bugs and skip safe_to_destroy on force
2023-11-03 02:29:50 -04:00
64e37ae963
Update OSD replacement functionality
...
1. Simplify this by leveraging the existing remove_osd/add_osd
functions, since its task was functionally identical to those two in
sequential order.
2. Add support for split OSDs within the command (replacing all OSDs on
the block device(s) as required).
3. Add additional configurability and flexibility around the old device,
weight, and external DB LVs.
2023-11-03 01:45:49 -04:00
3cb8a70f04
Add forcing to OSD purge
2023-11-02 23:20:48 -04:00
44d2f98e75
Remove Var field from OSDs
...
Not super duper useful and increases length
2023-11-02 22:55:39 -04:00
cb91bf18a7
Fix incorrect variables
2023-11-02 22:39:32 -04:00
a3e3fe829a
Adjust helptext for osd add
2023-11-02 22:34:58 -04:00
f53af510c1
Avoid startup failures if OSD removed
2023-11-02 22:24:39 -04:00
d5d783fad3
Set proper split flag
2023-11-02 22:20:22 -04:00
8b8957547a
Adjust helptext for create-db-vg command
2023-11-02 22:14:25 -04:00
980ea6a9e9
Adjust handling of ext_db and _count options
...
Avoid the use of superfluous flag options, default them to none, and add
support for fixed-size DB LVs.
2023-11-02 13:29:47 -04:00
0f433bd5eb
Add wait messages for OSD commands
2023-11-02 09:31:41 -04:00
8780044be6
Ensure db_device is an empty string
2023-11-02 00:52:18 -04:00
f08c654f22
Fix missing fstring
2023-11-01 21:41:06 -04:00
80a7fd6195
Improve help text messages
2023-11-01 21:38:55 -04:00
8b93f9a80e
Handle OSD index errors during stats collection
2023-11-01 21:33:40 -04:00
526a5f4a74
Add support for split OSD adds
...
Allows creating multiple OSDs on a single (NVMe) block device,
leveraging the "ceph-volume lvm batch" command. Replaces the previous
method of creating OSDs.
Also adds a new ZK item for each OSD indicating if it is split or not.
2023-11-01 21:31:35 -04:00
aa0b1f504f
Fix output bug
2023-11-01 15:46:38 -04:00
bc425b9224
Avoid duplicate confirmations in a safer way
...
This version instead still requires --yes with --restart to avoid the
confirmation option, but avoids duplicate prompts.
This might be slightly more cumbersome, but ensures consistency: every
situation that could cause a restart is confirmed even if --restart is
given.
2023-11-01 12:05:52 -04:00
