Commit Graph

56 Commits

Author SHA1 Message Date
Joshua Boniface b1357cafdb Add memfree to selector and use proper defaults 2022-06-10 02:03:12 -04:00
Joshua Boniface 7a40c7a55b Add support for replacing/refreshing OSDs
Adds commands to both replace an OSD disk, and refresh (reimport) an
existing OSD disk on a new node. This handles the cases where an OSD
disk should be replaced (either due to upgrades or failures) or where a
node is rebuilt in-place and an existing OSD must be re-imported to it.

This should avoid the need to do a full remove/add sequence for either
case.

Also cleans up some aspects of OSD removal that are identical between
methods (e.g. using safe-to-destroy and sleeping after stopping) and
fixes a bug if an OSD does not truly exist when the daemon starts up.
2022-05-06 15:32:06 -04:00
Joshua Boniface 1142454934 Add pool PGs count modification
Allows an administrator to adjust the PG count of a given pool. This can
be used to increase the PGs (for example after adding more OSDs) or
decrease it (to remove OSDs, reduce CPU load, etc.).
2021-12-28 21:53:29 -05:00
Joshua Boniface bbfad340a1 Add PGs count to pool list 2021-12-28 21:12:02 -05:00
Joshua Boniface 25fe45dd28 Add device class tiers to Ceph pools
Allows specifying a particular device class ("tier") for a given pool,
for instance SSD-only or NVMe-only. This is implemented with Crush
rules on the Ceph side, and via an additional new key in the pool
Zookeeper schema which is defaulted to "default".
2021-12-28 20:58:15 -05:00
Joshua Boniface abc23ebb18 Handle detect strings as arguments for blockdevs
Allows specifying blockdevs in the OSD and OSD-DB addition commands as
detect strings rather than actual block device paths. This provides
greater flexibility for automation with pvcbootstrapd (which originates
the concept of detect strings) and in general usage as well.
2021-12-28 02:53:02 -05:00
Joshua Boniface 5f193a6134 Perform automatic shutdown/stop on VM disable
Instead of requiring the VM to already be stopped, instead allow disable
state changes to perform a shutdown first. Also add a force option which
will do a hard stop instead of a shutdown.

References #148
2021-11-06 03:57:24 -04:00
Joshua Boniface 24de0f4189 Add MTU to network creation/modification
Addresses #144
2021-10-09 17:51:32 -04:00
Joshua Boniface c0f7ba0125 Add limit negation to VM list
When using the "state", "node", or "tag" arguments to a VM list, add
support for a "negate" flag to look for all VMs *not in* the state,
node, or tag state.
2021-10-07 11:50:52 -04:00
Joshua Boniface f83a345bfe Add test format versioning to storage benchmarks
Adds a test_format database column and a value in the API return for the
test format version, starting at 0 for the existing format as of 0.9.37.

References #143
2021-10-02 00:55:27 -04:00
Joshua Boniface 65df807b09 Add support for configurable OSD DB ratios
The default of 0.05 (5%) is likely ideal in the initial implementation,
but allow this to be set explicitly for maximum flexibility in
space-constrained or performance-critical use-cases.
2021-09-24 01:06:39 -04:00
Joshua Boniface adc8a5a3bc Add separate OSD DB device support
Adds in three parts:

1. Create an API endpoint to create OSD DB volume groups on a device.
Passed through to the node via the same command pipeline as
creating/removing OSDs, and creates a volume group with a fixed name
(osd-db).

2. Adds API support for specifying whether or not to use this DB volume
group when creating a new OSD via the "ext_db" flag. Naming and sizing
is fixed for simplicity and based on Ceph recommendations (5% of OSD
size). The Zookeeper schema tracks the block device to use during
removal.

3. Adds CLI support for the new and modified API endpoints, as well as
displaying the block device and DB block device in the OSD list.

While I debated supporting adding a DB device to an existing OSD, in
practice this ended up being a very complex operation involving stopping
the OSD and setting some options, so this is not supported; this can be
specified during OSD creation only.

Closes #142
2021-09-23 13:59:49 -04:00
Joshua Boniface a088aa4484 Add node log functions to API and CLI 2021-07-18 18:54:28 -04:00
Joshua Boniface 75fb60b1b4 Add VM list filtering by tag
Uses same method as state or node filtering, rather than altering how
the main LIMIT field works.
2021-07-14 00:59:20 -04:00
Joshua Boniface 27f1758791 Add tags manipulation to API
Also fixes some checks for Metadata too since these two actions are
almost identical, and adds tags to define endpoint.
2021-07-13 19:05:33 -04:00
Joshua Boniface a69105569f Add node PVC version data to Node information
Allows API client to see the currently-active version of the node
daemon.
2021-07-05 09:57:38 -04:00
Joshua Boniface 5ec198bf98 Update API doc with remaining items 2021-06-22 03:47:27 -04:00
Joshua Boniface ae480d6cc1 Add SR-IOV listing/info endpoints to API 2021-06-21 17:12:45 -04:00
Joshua Boniface f46c2e7f6a Implement VM rename functionality
Closes #125
2021-05-23 17:21:19 -04:00
Joshua Boniface f22f291c8b Add additional field and info to Swagger 2021-02-09 01:49:15 -05:00
Joshua Boniface 1b6613c280 Add live VNC information to domain output
Sets in the node daemon, returns via the API, and shows in the CLI,
information about the live VNC listen address and port for VNC-enabled
VMs.

Closes #115
2020-12-20 16:00:55 -05:00
Joshua Boniface 0820cb3c5b Update swagger documentation 2020-12-01 04:45:01 -05:00
Joshua Boniface a4b80be5ed Add provisioned memory to node info
Adds a separate field to the node memory, "provisioned", which totals
the amount of memory provisioned to all VMs on the node, regardless of
state, and in contrast to "allocated" which only counts running VMs.

Allows for the detection of potential overprovisioned states when
factoring in non-running VMs.

Includes the supporting code to get this data, since the original
implementation of VM memory selection was dependent on the VM being
running and getting this from libvirt. Now, if the VM is not active, it
gets this from the domain XML instead.
2020-10-18 14:17:15 -04:00
Joshua Boniface eb06c1494e Add API spec for benchmark results 2020-08-25 12:43:16 -04:00
Joshua Boniface 37a58d35e8 Implement limiting of node output
Closes #98
2020-06-25 11:51:53 -04:00
Joshua Boniface 4cdf1f7247 Add statistics values to the API 2020-06-07 02:15:33 -04:00
Joshua Boniface 99f579e41a Add wait support to API commands
References #72
2020-02-19 09:51:42 -05:00
Joshua Boniface ca68321be3 Allow modification of system templates
Closes #82
2020-02-18 16:18:27 -05:00
Joshua Boniface db558ec91f Complete implementation of OVA handling
Add functions for uploading, listing, and removing OVA images to the API
and CLI interfaces. Includes improved parsing of the OVF and creation of
a system_template and profile for each OVA.

Also modifies some behaviour around profiles, making most components
option at creation to support both profile types (and incomplete
profiles generally).

Implementation part 2/3 - remaining: OVA VM creation

References #71
2020-02-17 23:22:50 -05:00
Joshua Boniface b29c69378d Just describe the body in words 2020-02-09 21:08:27 -05:00
Joshua Boniface ad60f4b1f1 Try again with just query 2020-02-09 21:06:33 -05:00
Joshua Boniface 68638d7760 Use in: body for body contents 2020-02-09 21:05:15 -05:00
Joshua Boniface 4fa9878e01 Update swagger.json file 2020-02-09 21:02:29 -05:00
Joshua Boniface c979fed10a Use proper requestBody description of file 2020-02-09 20:59:03 -05:00
Joshua Boniface e419855911 Support converting types during upload
Allow the user to specify other, non-raw files and upload them,
performing a conversion with qemu-img convert and a temporary block
device as a shim (since qemu-img can't use FIFOs).

Also ensures that the target volume exists before proceeding.

Addresses #68
2020-02-09 20:29:12 -05:00
Joshua Boniface 49e5ce1176 Support uploading disk images to volumes in API
Addresses #68
2020-02-09 13:45:04 -05:00
Joshua Boniface 776d4f7262 Update API docs for profile 2020-02-02 01:34:55 -05:00
Joshua Boniface 99b220b051 Add profile modification endpoint 2020-01-16 22:58:08 -05:00
Joshua Boniface 2057859b9f Add Celery task list output 2020-01-12 14:12:01 -05:00
Joshua Boniface b6474198a4 Implement cluster maintenance mode
Implements a "maintenance mode" for PVC clusters. For now, the only
thing this mode does is disable node fencing while the state is true.
This allows the administrator to tell PVC that network connectivity,
etc. might be interrupted and to avoid fencing nodes.

Closes #70
2020-01-09 10:53:27 -05:00
Joshua Boniface a60afe8ea0 Update swagger.json with recent API changes 2020-01-08 21:27:13 -05:00
Joshua Boniface 82a88f8433 Fix incorrect port in swagger docs 2020-01-06 23:50:12 -05:00
Joshua Boniface 1122589b88 Add documentation of OSD endpoints and proper return 2020-01-06 12:54:18 -05:00
Joshua Boniface e433181e34 Support source_volume options to storage volumes 2020-01-05 17:25:34 -05:00
Joshua Boniface 41766febe6 Fix name of endpoint 2020-01-04 14:21:15 -05:00
Joshua Boniface a10ce95be3 Remove a few more instances of userdata_template 2020-01-04 13:58:48 -05:00
Joshua Boniface e550d4589d Move userdata from templates
This arrangement was driving me somewhat mad; make userdata just like
script in that it is not a "template".
2019-12-30 15:01:28 -05:00
Joshua Boniface 6de65378ca Fix bad metadata documentation 2019-12-29 20:58:02 -05:00
Joshua Boniface 4af95ecfcb Add API doc for cluster status 2019-12-29 20:54:20 -05:00
Joshua Boniface c9fb9600a4 Fix two small bugs and regen docs 2019-12-26 18:45:11 -05:00