Move more system functonality into PVC #136

New Issue

joshuaboniface · 2021-07-06T00:32:54-04:00

JoshuaBoniface commented

2021-07-06 00:32:54 -04:00

(Migrated from git.bonifacelabs.ca)

This is a very open-ended task to pave the way for eventual hot add/remove of nodes.

Move more things that are currently statically managed by Ansible into the PVC configuration. For instance, off the top of my head:

Ceph monitor and manager configurations. We already basically handle OSDs, so generating ceph.conf and such shouldn't be too difficult.
Libvirt configuration can be handled in Zookeeper.
Patroni could feature more integration to more tightly couple it to PVC node daemons (rather than relying on patronictl external calls).
Even Zookeeper could be managed dynamically, since Kazoo can hot-update the configuration. Or at least manage/write zoo.conf.

This is a very open-ended task to pave the way for eventual hot add/remove of nodes. Move more things that are currently statically managed by Ansible into the PVC configuration. For instance, off the top of my head: 1. Ceph monitor and manager configurations. We already basically handle OSDs, so generating `ceph.conf` and such shouldn't be too difficult. 2. Libvirt configuration can be handled in Zookeeper. 3. Patroni could feature more integration to more tightly couple it to PVC node daemons (rather than relying on `patronictl` external calls). 4. Even Zookeeper could be managed dynamically, since Kazoo can hot-update the configuration. Or at least manage/write `zoo.conf`.

JoshuaBoniface commented

2021-07-23 22:41:49 -04:00

(Migrated from git.bonifacelabs.ca)

I think 3 is a good place to start, since it's also a Python application and should have a direct API to handle this.

JoshuaBoniface commented

2021-08-05 00:19:35 -04:00

(Migrated from git.bonifacelabs.ca)

Definitely ties into #128. I think refactoring pretty much the entire node daemon from scratch with lessons learned is a good idea long-term.

joshuaboniface added this to the (deleted) project 2022-09-21 09:51:50 -04:00

joshuaboniface commented

2022-10-07 23:29:00 -04:00

Revisiting this thought. After thinking more, as nice as internalizing these daemons might seem, I think from an administration perspective leaving them external is the right call at least until v2.

The biggest hurdle will be the Zookeeper bootstrap problem. Since the data is held in the Zookeeper database, and the Zookeeper database needs to know its peers to start, this seems like it would need external config anyways to function properly. On the other side, reviewing Zookeper's manual there is the ability to hot add and remove nodes, so perhaps it's possible.

In any case I'm going to close this one in favour of other more specific implementation ideas.

Revisiting this thought. After thinking more, as nice as internalizing these daemons might seem, I think from an administration perspective leaving them external is the right call at least until v2. The biggest hurdle will be the Zookeeper bootstrap problem. Since the data is held in the Zookeeper database, and the Zookeeper database needs to know its peers to start, this seems like it would need external config anyways to function properly. On the other side, reviewing Zookeper's manual there is the ability to hot add and remove nodes, so perhaps it's possible. In any case I'm going to close this one in favour of other more specific implementation ideas.

joshuaboniface closed this issue

2022-10-07 23:29:04 -04:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: parallelvirtualcluster/pvc#136