Commit Graph

240 Commits

Author SHA1 Message Date
Joshua Boniface 75d95bbef7 Fix minor bugs 2019-10-12 02:03:23 -04:00
Joshua Boniface 18fc49fc6c Use node instead of hypervisor consistently 2019-10-12 01:59:08 -04:00
Joshua Boniface 0763bd2d51 Fix minor bugs 2019-10-12 01:55:37 -04:00
Joshua Boniface e5393082b6 Use build-in selector and respect limits in client
Use the new built-in selector option, as well as respecting node limits,
when performing migrate or move actions on a VM via the clients.
2019-10-12 01:45:44 -04:00
Joshua Boniface 8dc0c8f0ac Fix minor bugs 2019-10-12 01:36:50 -04:00
Joshua Boniface 5995353597 Implement VM metadata and use it
Implements the storing of three VM metadata attributes:
1. Node limits - allows specifying a list of hosts on which the VM must
run. This limit influences the migration behaviour of VMs.
2. Per-VM node selectors - allows each VM to have its migration
autoselection method specified, to automatically allow different methods
per VM based on the administrator's preferences.
3. VM autorestart - allows a VM to be automatically restarted from a
stopped state, presumably due to a failure to find a target node (either
due to limits or otherwise) during a flush/fence recovery, on the next
node unflush/ready state of its home hypervisor. Useful mostly in
conjunction with limits to ensure that VMs which were shut down due to
there being no valid migration targets are started back up when their
node becomes ready again.

Includes the full client interaction with these metadata options,
including printing, as well as defining a new function to modify this
metadata. For the CLI it is set/modified either on `vm define` or via the
`vm meta` command. For the API it is set/modified either on a POST to
the `/vm` endpoint (during VM definition) or on POST to the `/vm/<vm>`
endpoint. For the API this replaces the previous reserved word for VM
creation from scratch as this will no longer be implemented in-daemon
(see #22).

Closes #52
2019-10-12 01:17:39 -04:00
Joshua Boniface 76e6b42389 Add clone_volume backend command 2019-10-10 14:09:07 -04:00
Joshua Boniface b0175eafcb Fix bug in output 2019-09-27 11:25:36 -04:00
Joshua Boniface 63cf90b23d Add Python header to ceph.py 2019-09-26 14:07:52 -04:00
Joshua Boniface 0c9d6db14f Gracefully handle log following output
If the Zookeeper node goes away (e.g. VM is deleted), return a
reasonable error message instead of a stack trace.
2019-09-07 12:30:31 -04:00
Joshua Boniface 3b7c1adf71 Implement configurable replcfg (client-side)
Implements administrator-selectable replication configurations for new
pools in PVC clusters, overriding the default of copies=3,mincopies=2.
2019-08-23 14:12:15 -04:00
Joshua Boniface 2880a761c0 Move Ceph command pipe to new location
Matching the new /cmd/domain pipe, move Ceph pipe to /cmd/ceph.
2019-08-07 14:47:27 -04:00
Joshua Boniface 43756fc62b Implement lock flush function for clients
Uses a similar command pathway as the Ceph commands do, but in a
different location (pending move of the Ceph commands to a similar
location).
2019-08-07 13:38:49 -04:00
Joshua Boniface 6a712ffe7a Don't crash if VM has invalid disks
Useful if storage=False and removing a VM.
2019-07-31 23:05:00 -04:00
Joshua Boniface 2dbf276e51 Implement snapshot renaming
[3/2] Implements #44
2019-07-28 23:00:35 -04:00
Joshua Boniface e2f153c3b1 Fix the same bug in a different place 2019-07-26 23:28:57 -04:00
Joshua Boniface a5ed776692 Fix missing format string 2019-07-26 20:54:52 -04:00
Joshua Boniface c464443a9c Fix snapshot lists 2019-07-26 16:44:25 -04:00
Joshua Boniface 2710ed8432 Fix up bugs in the naming 2019-07-26 16:38:35 -04:00
Joshua Boniface 4937c01075 Remove junk whitespace 2019-07-26 15:13:21 -04:00
Joshua Boniface 1872489b1d Correct duplicating fuzzy limits
These were put in a bad spot (inside the loop) and would cause
operations to take progressively longer. Only do these once.
2019-07-26 15:13:21 -04:00
Joshua Boniface d5f263bdd6 Implement Ceph volume resize and rename in clients
[1/2] Implements #44
2019-07-26 15:13:21 -04:00
Joshua Boniface 1da7462e0d Move bad-net colour outside of string
This just trashed the formatting of the string if the network didn't
exist, despite several previous attempts to get this to align. Give up;
set the colour for the whole net list if any one network is invalid.
This is not as nice as per-network colouring but saves the hassle and
complexity.
2019-07-26 11:41:51 -04:00
Joshua Boniface 8071fb87d7 Include XML in domain information 2019-07-25 16:29:29 -04:00
Joshua Boniface fb40ef5b04 Fix typo in common 2019-07-25 15:45:45 -04:00
Joshua Boniface 75f80fa4bd Include XML output in VM information 2019-07-25 15:44:29 -04:00
Joshua Boniface bcd48648b2 Add is_migrated check function 2019-07-25 14:33:50 -04:00
Joshua Boniface 2a220cd16e Nicer colour output for coordinator state client 2019-07-12 09:31:42 -04:00
Joshua Boniface e765ec7f49 Handle client connection falures gracefully 2019-07-12 00:55:06 -04:00
Joshua Boniface 0d513fea4d Add delay after deleting VM
Otherwise there might still be a watcher on the block device when we go
to delete it, which causes a failure.
2019-07-10 19:05:36 -04:00
Joshua Boniface 7a8aee9fe7 Remove flush locking functionality
This just seemed like more trouble that it was worth. Flush locks were
originally intended as a way to counteract the weird issues around
flushing that were mostly fixed by the code refactoring, so this will
help test if those issues are truly gone. If not, will look into a
cleaner solution that doesn't result in unchangeable states.
2019-07-09 23:59:17 -04:00
Joshua Boniface ffc2a6ee1b Fix display bug 2019-07-09 10:47:56 -04:00
Joshua Boniface 6426607769 Store list of RBD disks in ZK
Store a basic list of RBD disks in Zookeeper for access by the node
subsystem to handle RBD locks. This avoids the need to implement complex
parsing logic inside the fencing configuration (or elsewhere).

Also handle a malformed XML content properly during VM define.
2019-07-09 10:22:23 -04:00
Joshua Boniface e263a05237 Reorganize client functions
Move some functions to the common file for future usage.
2019-07-09 09:29:47 -04:00
Joshua Boniface b5af3a16a8 Join features list in CLI output 2019-07-08 22:47:04 -04:00
Joshua Boniface a3de9da513 Add quotes around vars in all return messages 2019-07-08 22:37:26 -04:00
Joshua Boniface dd78aa8fb8 Use quotes around variables in output 2019-07-08 22:31:24 -04:00
Joshua Boniface 1ce2cfc3e3 Convert units on CLI output and format ops too 2019-07-08 22:03:34 -04:00
Joshua Boniface dda4cc282e Fix bad element names 2019-07-08 21:41:54 -04:00
Joshua Boniface b2cc444024 Correct bad bracket type 2019-07-08 21:40:06 -04:00
Joshua Boniface 359886e335 Correct variable name bugs 2019-07-08 21:35:18 -04:00
Joshua Boniface 31b4dd9aed Show radosdf output in clients 2019-07-08 10:59:49 -04:00
Joshua Boniface f75216d946 Make force_migrate not replace existing node
Previous to this, if once force-migrated a VM, the previous_node value
would be updated to the current node, which is likely never what an
administrator would want. Change this functionality so that the previous
node value is not changed, and update the documentation to reflect this.
2019-07-07 15:10:48 -04:00
Joshua Boniface 4f6d0baf6c Fix bad ZK node entry 2019-07-06 01:42:55 -04:00
Joshua Boniface 7d56ecb13d Apply tweak to rule direction handling 2019-07-05 21:38:47 -04:00
Joshua Boniface 5ac3d55b8d Always use the passed domain in messages
Don't ever output an ugly UUID unless that's what was actually sent to
the function. Makes output much much nicer, especially on errors.
2019-07-05 16:38:54 -04:00
Joshua Boniface 18520d6c77 Support disabling fuzzy limit matching
Used for the APIs when they filter by name but we don't want a fuzzy
limit since we specified an exact name.
2019-07-05 16:30:20 -04:00
Joshua Boniface dc16f51b2a Fix variable naming bug 2019-07-05 14:18:18 -04:00
Joshua Boniface 6ee3c91a63 Use None instead of all in ceph.py
Make it like other optional args (like direction in net ACLs) and use
None instead of 'all' when specifying any option
2019-07-05 13:59:29 -04:00
Joshua Boniface d755a80fe8 Fix bugs with refactoring 2019-07-05 00:44:40 -04:00
Joshua Boniface 80ea54d40f Make bracketing and indenting consistent 2019-07-05 00:32:04 -04:00
Joshua Boniface 82bf8a8406 Remove superfluous comment 2019-07-05 00:30:11 -04:00
Joshua Boniface 84a905b7f0 Refactor ceph client for API 2019-07-05 00:30:03 -04:00
Joshua Boniface 6e2dfc1704 Reorganize file before refactoring
This is the largest of the function files, and unlike the others is
cleanly split into four types. Reorganize the file and function
definitions around those types to make it easier to navigate, and do so
separately before refactoring for API.
2019-07-04 23:09:16 -04:00
Joshua Boniface bcab11ee49 Refactor network.py for API usage 2019-07-04 23:01:22 -04:00
Joshua Boniface b195eb0668 Rename Information get function to match others 2019-07-04 23:00:53 -04:00
Joshua Boniface 10a0da6fdf Return strings only in extra details 2019-07-04 12:56:41 -04:00
Joshua Boniface 338af97a7a Handle message about wait locks a bit better 2019-06-27 16:20:30 -04:00
Joshua Boniface 77e215b0eb Don't queue flushes without --wait
Don't try to queue up a flush when there is already a flush lock; direct
the user to use --wait (which will actually wait before triggering the
new action), or try again later.
2019-06-27 14:36:58 -04:00
Joshua Boniface d28b1ba133 Support removing VMs (and disks) from PVC clients
Adds full support for removing a VM entirely, including its RBD disks,
via the PVC client(s). Avoids needing an undefine + manual removal.
2019-06-27 11:19:48 -04:00
Joshua Boniface b52cf01ecc Make conditionals more Pythonic 2019-06-25 12:36:37 -04:00
Joshua Boniface 93be983b39 Properly verify node inputs 2019-06-25 12:36:37 -04:00
Joshua Boniface d8dca50af4 Ensure domain is always set 2019-06-24 12:59:32 -04:00
Joshua Boniface 4922e973c7 Functionize duplicate code and fix net length bug 2019-06-24 10:10:07 -04:00
Joshua Boniface 1316b9ed59 Fix refactoring bugs 2019-06-24 09:56:06 -04:00
Joshua Boniface e097e9e8af Fix up status call 2019-06-21 16:40:12 -04:00
Joshua Boniface 60e6df4416 Increase wait on OSD add/remove a bit more 2019-06-21 16:01:08 -04:00
Joshua Boniface 9cec0c5b43 Remove extra bolding in lists 2019-06-21 15:57:50 -04:00
Joshua Boniface 808e53fe29 Increase wait on OSD add/remove 2019-06-21 15:56:37 -04:00
Joshua Boniface c63b5a8056 Handle a no-valid-nodes situation 2019-06-21 11:37:35 -04:00
Joshua Boniface f3e9082376 Simplify fromhuman byte formatting 2019-06-21 10:18:35 -04:00
Joshua Boniface 867ad1fc1b Support human-readable biconversion and in volumes 2019-06-21 09:23:52 -04:00
Joshua Boniface 3d3fa1b99f Add write-lock-wait to all other Ceph commands 2019-06-21 09:05:00 -04:00
Joshua Boniface 3dd56c55c0 Fix a few more bugs 2019-06-19 15:32:32 -04:00
Joshua Boniface 97a3dafa3b Fix bug in regex match for snapshots 2019-06-19 15:24:04 -04:00
Joshua Boniface 51256b4f0f Fix bad bolding in lists 2019-06-19 15:16:22 -04:00
Joshua Boniface 5d4ecafb03 Support listing of snapshots 2019-06-19 15:15:17 -04:00
Joshua Boniface 80de40252b Add shutdown colour to output 2019-06-19 14:28:04 -04:00
Joshua Boniface 39a45a1e0f Fix bug with daemon_state 2019-06-19 14:19:00 -04:00
Joshua Boniface bc38af774e Check that node is in run state before primary
Fixes #37
2019-06-19 12:49:34 -04:00
Joshua Boniface 5e211837ad Fix bug in output 2019-06-19 11:24:25 -04:00
Joshua Boniface e6a458f38c Support listing volumes 2019-06-19 11:22:49 -04:00
Joshua Boniface 98365b0fcf Correct bugs around snapshots 2019-06-19 09:41:33 -04:00
Joshua Boniface a4ab3075ab Correct some bugs around new code 2019-06-19 00:23:25 -04:00
Joshua Boniface 01959cb9e3 Implementation of RBD volumes and snapshots
Adds the ability to manage RBD volumes (add/remove) and RBD
snapshots (add/remove). (Working) list functions to come.
2019-06-19 00:12:44 -04:00
Joshua Boniface b50b2a827b Add forced delays after pool add/remove
Prevents returning immediately to give the cluster some breathing
room before the admin can do other commands. Keep the write lock
as well to prevent other clients from attempting this as well.
2019-06-18 21:56:24 -04:00
Joshua Boniface ee73676114 Fix bug with pool removal 2019-06-18 21:51:11 -04:00
Joshua Boniface 3ba3c339a7 Show vCPU count on CLI output
Showing the static, total number of CPUs was pointless. Instead,
show the number of allocated vCPUs. To preserve space, no longer
show the host CPU count in the list.
2019-06-02 22:30:26 -04:00
Joshua Boniface b9774bdf03 Increase wait sleeps in node flush/unflush 2019-05-26 23:21:01 -04:00
Joshua Boniface 6fd4710f7f Remove bad replacement 2019-05-21 19:51:23 -04:00
Joshua Boniface 79d0a2eafc Handle raw sorting properly with new list format 2019-05-21 14:44:45 -04:00
Joshua Boniface 73443ecbaf Update vm.py to allow API use 2019-05-20 22:15:28 -04:00
Joshua Boniface 91ea96b772 Actually send wait for unflush 2019-05-11 00:16:38 -04:00
Joshua Boniface 62a71af46e Implement locking for unflush as well
References #32
2019-05-11 00:13:03 -04:00
Joshua Boniface c19902d952 Implement flush locking for nodes
Implements a locking mechanism to prevent clobbering of node
flushes. When a flush begins, a global cluster lock is placed
which is freed once the flush completes. While the lock is in place,
other flush events queue waiting for the lock to free before
proceeding.

Modifies the CLI output flow when the `--wait` option is specified.
First, if a lock exists when running the command, the message is
tweaked to indicate this, and the client will wait first for the
lock to free, and then for the flush as normal. Second, the wait
depends on the active lock rather than the domain_status for
consistency purposes.

Closes #32
2019-05-10 23:52:24 -04:00
Joshua Boniface 045ad131af Update node.py to allow API use
Addresses #33
2019-05-10 23:27:34 -04:00
Joshua Boniface b6ecd36588 Implement domain log watching
Implements the ability for a client to watch almost-live domain
console logs from the hypervisors. It does this using a deque-based
"tail -f" mechanism (with a configurable buffer per-VM) that watches
the domain console logfile in the (configurable) directory every
half-second. It then stores the current buffer in Zookeeper when
changed, where a client can then request it, either as a static piece
of text in the `less` pager, or via a similar "tail -f" functionality
implemented using fixed line splitting and comparison to provide a
generally-seamless output.

Enabling this feature requires each guest VM to implement a Libvirt
serial log and write its (text) console to it, for example using the
default logging directory:

```
<serial type='pty'>
    <log file='/var/log/libvirt/vmname.log' append='off'/>
<serial>
```

The append mode can be either on or off; on grows files unbounded,
off causes the log (and hence the PVC log data) to be truncated on
initial VM startup from offline. The administrator must choose how
they best want to handle this until Libvirt implements their own
clog-type logging format.
2019-05-10 23:26:59 -04:00
Joshua Boniface 1126382ac9 Initial API including fixes to common functons
Some functions were doing `click.echo` inside themselves; don't
do that as it's not API compatible, just return everything.
2019-03-21 11:19:28 -04:00
Joshua Boniface 9f0d7c5bf8 Support VM list filtering by state 2019-03-20 11:31:54 -04:00
Joshua Boniface eec14ff177 Don't set unmigrated VMs to start 2019-03-20 10:19:01 -04:00