Commit Graph

2384 Commits

Author SHA1 Message Date
Joshua Boniface 65df807b09 Add support for configurable OSD DB ratios
The default of 0.05 (5%) is likely ideal in the initial implementation,
but allow this to be set explicitly for maximum flexibility in
space-constrained or performance-critical use-cases.
2021-09-24 01:06:39 -04:00
Joshua Boniface d0f3e9e285 Bump version to 0.9.36 2021-09-23 14:01:38 -04:00
Joshua Boniface adc8a5a3bc Add separate OSD DB device support
Adds in three parts:

1. Create an API endpoint to create OSD DB volume groups on a device.
Passed through to the node via the same command pipeline as
creating/removing OSDs, and creates a volume group with a fixed name
(osd-db).

2. Adds API support for specifying whether or not to use this DB volume
group when creating a new OSD via the "ext_db" flag. Naming and sizing
is fixed for simplicity and based on Ceph recommendations (5% of OSD
size). The Zookeeper schema tracks the block device to use during
removal.

3. Adds CLI support for the new and modified API endpoints, as well as
displaying the block device and DB block device in the OSD list.

While I debated supporting adding a DB device to an existing OSD, in
practice this ended up being a very complex operation involving stopping
the OSD and setting some options, so this is not supported; this can be
specified during OSD creation only.

Closes #142
2021-09-23 13:59:49 -04:00
Joshua Boniface df277edf1c Move console watcher stop try up
Could cause an exception if d_domain is not defined yet.
2021-09-22 16:02:04 -04:00
Joshua Boniface 772807deb3 Bump version to 0.9.35 2021-09-13 02:20:46 -04:00
Joshua Boniface 58db537093 Add memory and vCPU checks to VM define/modify
Ensures that a VM won't:

(a) Have provisioned more RAM than there is available on a given node.
Due to memory overprovisioning, this is simply a "is the VM memory count
more than the node count", and doesn't factor in free or used memory on
a node, total cluster usage, etc. So if a node has 64GB total RAM, the
VM limit is 64GB. It is up to an administrator to ensure sanity *below*
that value.

(b) Have provisioned more vCPUs than there are CPU cores on the node,
minus 2 to account for hypervisor/storage processes. Will ensure there
is no severe CPU contention caused by a single VM having more vCPUs than
there are actual execution threads available.

Closes #139
2021-09-13 01:51:21 -04:00
Joshua Boniface e71a6c90bf Add pool size check when resizing volumes
Closes #140
2021-09-12 19:54:51 -04:00
Joshua Boniface a8e9a56924 Increase build-and-deploy sleep 2021-09-12 19:50:58 -04:00
Joshua Boniface f3fb492633 Handle VM disk/network stats gathering exceptions 2021-09-12 19:41:07 -04:00
Joshua Boniface e962743e51 Add VM device hot attach/detach support
Adds a new API endpoint to support hot attach/detach of devices, and the
corresponding client-side logic to use this endpoint when doing VM
network/storage add/remove actions.

The live attach is now the default behaviour for these types of
additions and removals, and can be disabled if needed.

Closes #141
2021-09-12 19:33:00 -04:00
Joshua Boniface 46f1d761f6 Adjust lint script for newer linter 2021-09-12 15:40:38 -04:00
Joshua Boniface be954c1625 Don't crash cleanup if no this_node 2021-08-29 03:52:18 -04:00
Joshua Boniface fb46f5f9e9 Change default node object state to flushed 2021-08-29 03:34:08 -04:00
Joshua Boniface 694b8e85a0 Bump version to 0.9.34 2021-08-24 16:15:25 -04:00
Joshua Boniface eb321497ee Properly handle =-separated fsargs 2021-08-24 11:40:22 -04:00
Joshua Boniface 5b81e59481 Convert argument elements of task status to types 2021-08-23 14:28:12 -04:00
Joshua Boniface a4c0e0befd Fix typo in output message 2021-08-23 00:39:19 -04:00
Joshua Boniface a18cef5f25 Bump version to 0.9.33 2021-08-21 03:28:48 -04:00
Joshua Boniface f6c5aa9992 Avoid failing if no provisioner tasks 2021-08-21 03:25:16 -04:00
Joshua Boniface ffa3dd5edb Ensure pycache files are removed on deb creation 2021-08-21 03:19:18 -04:00
Joshua Boniface afb0359c20 Refactor pvcnoded to reduce Daemon.py size
This branch commit refactors the pvcnoded component to better adhere to
good programming practices. The previous Daemon.py was a massive file
which contained almost 2000 lines of direct, root-level code which was
directly imported. Not only was this poor practice, but this resulted
in a nigh-unmaintainable file which was hard even for me to understand.

This refactoring splits a large section of the code from Daemon.py into
separate small modules and functions in the `util/` directory. This will
hopefully make most of the functionality easy to find and modify without
having to dig through a single large file.

Further the existing subcomponents have been moved to the `objects/`
directory which clearly separates them.

Finally, the Daemon.py code has mostly been moved into a function,
`entrypoint()`, which is then called from the `pvcnoded.py` stub.

An additional item is that most format strings have been replaced by
f-strings to make use of the Python 3.6 features in Daemon.py and the
utility files.
2021-08-21 03:14:22 -04:00
Joshua Boniface afdf254297 Bump version to 0.9.32 2021-08-19 12:37:58 -04:00
Joshua Boniface 42e776fac1 Properly handle exceptions getting VM stats 2021-08-19 12:36:31 -04:00
Joshua Boniface dae67a1b7b Fix image dimensions and size 2021-08-18 19:51:55 -04:00
Joshua Boniface b86f8c1e09 Add screenshots to docs 2021-08-18 19:49:53 -04:00
Joshua Boniface 13e309b450 Fix colours of network status elements 2021-08-18 19:41:53 -04:00
Joshua Boniface 7ecc6a2635 Bump version to 0.9.31 2021-07-30 12:08:12 -04:00
Joshua Boniface 73e8149cb0 Remove explicit image-features from rbd cmd
This should be managed in ceph.conf with the `rbd default
features` configuration option instead, and thus can be tailored to the
underlying OS version.
2021-07-30 11:33:59 -04:00
Joshua Boniface 4a7246b8c0 Ensure RBD resize has bytes appended
If this isn't, the resize will be interpreted as a MB value and result
in an absurdly big volume instead. This is the same consistency
validation that occurs on add.
2021-07-30 11:25:13 -04:00
Joshua Boniface c49351469b Revert "Ensure consistent sizing of volumes"
This reverts commit dc03e95bbf.
2021-07-29 15:30:00 -04:00
Joshua Boniface dc03e95bbf Ensure consistent sizing of volumes
Convert from human to bytes, then to megabytes and always pass this to
the RBD command. This ensures consistency regardless of what is actually
passed by the user.
2021-07-29 15:14:25 -04:00
Joshua Boniface c460aa051a Add missing floppy RASD type for compat 2021-07-27 16:32:32 -04:00
Joshua Boniface 3ab6365a53 Adjust receive output to show proper source 2021-07-22 15:43:08 -04:00
Joshua Boniface 32613ff119 Remove obsolete Suggests lines from control 2021-07-20 00:35:21 -04:00
Joshua Boniface 2a99a27feb Bump version to 0.9.30 2021-07-20 00:01:45 -04:00
Joshua Boniface 45f23c12ea Remove logs from schema validation
These are managed entirely by the logging subsystem not by the schema
handler due to catch-22's.
2021-07-20 00:00:37 -04:00
Joshua Boniface fa1d93e933 Bump version to 0.9.29 2021-07-19 16:55:41 -04:00
Joshua Boniface b14bc7e3a3 Add retry to log writes 2021-07-19 13:11:28 -04:00
Joshua Boniface 4d6842f942 Don't bail out if write fails, keep retrying 2021-07-19 13:09:36 -04:00
Joshua Boniface 6ead21a308 Handle cleanup from a failure properly 2021-07-19 12:39:13 -04:00
Joshua Boniface b7c8c2ee3d Fix handling of this_node and d_domain in cleanup 2021-07-19 12:36:35 -04:00
Joshua Boniface d48f58930b Use harder exits and add cleanup termination 2021-07-19 12:27:16 -04:00
Joshua Boniface 7c36388c8f Add post-networking delay and adjust daemon delay 2021-07-19 12:23:45 -04:00
Joshua Boniface e9df043c0a Ensure ZK logging does not block startup 2021-07-19 12:19:59 -04:00
Joshua Boniface 71e4d0b32a Bump version to 0.9.28 2021-07-19 09:29:34 -04:00
Joshua Boniface f16bad4691 Revamp confirmation options for vm modify
Before, "-y"/"--yes" only confirmed the reboot portion. Instead, modify
this to confirm both the diff portion and the restart portion, and add
separate flags to bypass one or the other independently, ensuring the
administrator has lots of flexibility. UNSAFE mode implies "-y" so both
would be auto-confirmed if that option is set.
2021-07-19 00:25:43 -04:00
Joshua Boniface 15d92c483f Bump version to 0.9.27 2021-07-19 00:03:40 -04:00
Joshua Boniface 7dd17e71e7 Fix bug with VM editing with file
Current config is needed for the diff but it was in a conditional.
2021-07-19 00:02:19 -04:00
Joshua Boniface 5be968123f Readd 1 second queue get timeout
Otherwise daemon stops will sometimes inexplicably block.
2021-07-18 22:17:57 -04:00
Joshua Boniface 99fd7ebe63 Fix excessive CPU due to looping 2021-07-18 22:06:50 -04:00