Support DB cache devices for Ceph OSDs #142

New Issue

joshuaboniface · 2021-09-12T14:32:40-04:00

JoshuaBoniface commented

2021-09-12 14:32:40 -04:00

(Migrated from git.bonifacelabs.ca)

Support adding cache devices as an option to new Ceph OSDs. Available with the --block.db option flag to ceph-volume lvm prepare.

Potentially also add method to add this cache later on too.

Ceph docs recommend at least 4% of the size of the device for the DB cache, so let's set our default to 10% for simplicity and to provide decent WAL area.

Support adding cache devices as an option to new Ceph OSDs. Available with the `--block.db` option flag to `ceph-volume lvm prepare`. Potentially also add method to add this cache later on too. Ceph docs recommend at least 4% of the size of the device for the DB cache, so let's set our default to 10% for simplicity and to provide decent WAL area.

JoshuaBoniface commented

2021-09-23 02:40:31 -04:00

(Migrated from git.bonifacelabs.ca)

From https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#sizing:

For RBD workloads, 1% to 2% of block size is usually enough.

So 5% is probably OK instead.

We can implement this with LVM on a device first to keep things easier to manage and more automated.

So add a set of commands like so:

pvc storage osd db parent command.
pvc storage osd db vg command to create/view/manage a DB device volume group.
pvc storage osd add-db command to add a DB LV to an existing OSD.
pvc storage osd add --db flag to add a DB LV to a new OSD.

All of these will be piped through to the relevant node just like other OSD commands.

From https://docs.ceph.com/en/latest/rados/configuration/bluestore-config-ref/#sizing: >For RBD workloads, 1% to 2% of block size is usually enough. So 5% is probably OK instead. We can implement this with LVM on a device first to keep things easier to manage and more automated. So add a set of commands like so: 1. `pvc storage osd db` parent command. 2. `pvc storage osd db vg` command to create/view/manage a DB device volume group. 3. `pvc storage osd add-db` command to add a DB LV to an existing OSD. 4. `pvc storage osd add` `--db` flag to add a DB LV to a new OSD. All of these will be piped through to the relevant node just like other OSD commands.

JoshuaBoniface commented

2021-09-23 13:52:52 -04:00

(Migrated from git.bonifacelabs.ca)

closed via commit 838072d6d4890453f4e33e14636eaa9776eaf822

JoshuaBoniface commented

2021-09-23 13:55:56 -04:00

(Migrated from git.bonifacelabs.ca)

Ended up going with a requirement of fresh OSDs for this to work; it ended up being a major hassle to add a DB to an existing OSD, so the procedure must be a remove + rebalance + add-with-db-block. Otherwise, this is fully implemented in 0.9.36.

JoshuaBoniface commented

2021-09-23 14:03:05 -04:00

(Migrated from git.bonifacelabs.ca)

Actually closed via commit 6cef68d157

Actually closed via commit 6cef68d1577a4ad1173c7bb95b7d7c29c815f331

JoshuaBoniface commented

2021-10-03 22:47:55 -04:00

(Migrated from git.bonifacelabs.ca)

mentioned in commit 6cef68d157

mentioned in commit 6cef68d1577a4ad1173c7bb95b7d7c29c815f331

joshuaboniface referenced this issue from a commit

2023-09-01 15:58:46 -04:00