parallelvirtualcluster
/
pvc
1
0
Fork 0
Code Issues 11 Pull Requests Projects Releases 111 Wiki Activity
2,017 Commits 1 Branch 113 Tags 28 MiB
Commit Graph

289 Commits

Author SHA1 Message Date
Joshua Boniface c3bd6b6ecc Add missing call into cluster initialize function 2020-04-06 14:48:26 -04:00
Joshua Boniface 09aeb33d13 Don't convert non-integer bytes/ops 2020-03-30 19:09:16 -04:00
Joshua Boniface d84e94eff4 Add force_single_node script 2020-03-25 10:48:49 -04:00
Joshua Boniface ce9d0e9603 Add helper scripts to CLI client 2020-03-22 01:19:55 -04:00
Joshua Boniface 3aea5ae34b Correct invalid function call 2020-03-21 16:46:34 -04:00
Joshua Boniface 36588a3a81 Work around bad RequestArgs handling 2020-03-03 16:48:20 -05:00
Joshua Boniface 11f045f100 Support showing individual userdata and script doc 
Closes #89
 2020-02-27 14:31:08 -05:00
Joshua Boniface 9d5f50f82a Implement progress bars for file uploads 
Provide pretty status bars to indicate upload progress for tasks that
perform large file uploads to the API ('provisioner ova upload' and
'storage volume upload') so the administrator can gauge progress and
estimated time to completion.
 2020-02-20 22:42:19 -05:00
Joshua Boniface 0caea03428 Clean up redundant message output 2020-02-19 16:27:14 -05:00
Joshua Boniface 65932b20d2 Handle request failures more gracefully 2020-02-19 16:19:34 -05:00
Joshua Boniface 1b8b32b07c Don't return tuple value on error 2020-02-19 15:47:08 -05:00
Joshua Boniface 39ce704969 Implement wait for node primary/secondary in CLI 
Use a different wait method of querying the node status every
half-second during the transition, in order to wait on the transition to
complete if desired.

Closes #72
 2020-02-19 14:33:31 -05:00
Joshua Boniface 8678dedfea Revert "Implement wait for node coordinator transition" 
This reverts commit 0aefafa7f7.

This does not work since the API goes away during the transition.

References #72
 2020-02-19 10:50:21 -05:00
Joshua Boniface 0aefafa7f7 Implement wait for node coordinator transition 
References #72
 2020-02-19 10:50:04 -05:00
Joshua Boniface 78780039de Add wait support to VM CLI commands 
References #72
 2020-02-19 10:02:32 -05:00
Joshua Boniface ca68321be3 Allow modification of system templates 
Closes #82
 2020-02-18 16:18:27 -05:00
Joshua Boniface dd44f2f42b Correct formatting error in confirmation 2020-02-17 23:31:03 -05:00
Joshua Boniface db558ec91f Complete implementation of OVA handling 
Add functions for uploading, listing, and removing OVA images to the API
and CLI interfaces. Includes improved parsing of the OVF and creation of
a system_template and profile for each OVA.

Also modifies some behaviour around profiles, making most components
option at creation to support both profile types (and incomplete
profiles generally).

Implementation part 2/3 - remaining: OVA VM creation

References #71
 2020-02-17 23:22:50 -05:00
Joshua Boniface 59ca296c58 Add basic OVA profile support 2020-02-17 12:00:51 -05:00
Joshua Boniface b16e2b4925 Handle CLI wildcard addresses properly 
If the local API instance is listening on a wildcard, connect to
127.0.0.1 instead.
 2020-02-16 20:02:08 -05:00
Joshua Boniface 1de57ab6f3 Add CLI client interface to image upload 
Closes #68
 2020-02-09 20:42:56 -05:00
Joshua Boniface ce985234c3 Use consistent naming of components 
Rename "pvcd" to "pvcnoded", and "pvc-api" to "pvcapid" so names for the
daemons are fully consistent. Update the names of the configuration
files as well to match this new formatting.

References #79
 2020-02-08 19:34:07 -05:00
Joshua Boniface 4505b239eb Rename API and common Debian packages 
Closes #79
 2020-02-08 18:50:38 -05:00
Joshua Boniface e7728b8375 Allow modification of a VM profile 
And fix some bugs around this.

Fixes #76
 2020-01-30 11:45:46 -05:00
Joshua Boniface 4868625809 Add try/except around new log lines 
Fixes #73
 2020-01-30 09:28:47 -05:00
Joshua Boniface ebfd9c1882 Gracefully handle API connect timeouts 
Prevents dumping a massive stacktrace if connection fails.
 2020-01-30 09:24:10 -05:00
Joshua Boniface 8c59707cba Prevent writing invalid XML to config 2020-01-28 10:51:32 -05:00
Joshua Boniface 27e73fc6a9 Fix issues with wait on node flush/ready 
Ensure that the strings are converted to booleans on the API side, and
that we're sending a lowercase true/false (for consistency).
 2020-01-26 23:05:30 -05:00
Joshua Boniface 0892e4c842 Fix overlapping name config in VM define 2020-01-23 11:33:09 -05:00
Joshua Boniface 53819b198c Add info about disk/net ordering in templates 
Ensures that the behaviour of 650ed972a1
is reflected in the help messages of these commands.
 2020-01-20 21:20:41 -05:00
Joshua Boniface 194a642c15 Report errors in log -f 2020-01-15 00:47:14 -05:00
Joshua Boniface 3fbd5fcead Reformat output warning message 2020-01-12 16:07:26 -05:00
Joshua Boniface 003171ea18 Correct syntax error 2020-01-12 16:04:42 -05:00
Joshua Boniface 74ff8b41ea Add prov warning to primary/secondary commands 
Warn the administrator if there are active provisioning jobs while
adjusting the current primary node. This is the simplest, cleanest
solution to #69 without trying to implement any hacks or blocking
operations. The administrator can then decide to revert the action
if needed, or will at least know how many jobs are running/queued and
may need to be cancelled.
 2020-01-12 16:03:51 -05:00
Joshua Boniface c4ef73c953 Refactor provisioner data+list format to match 2020-01-12 14:23:11 -05:00
Joshua Boniface b9177c2f26 Add word Job to task ID output 2020-01-12 14:16:11 -05:00
Joshua Boniface 2057859b9f Add Celery task list output 2020-01-12 14:12:01 -05:00
Joshua Boniface 1683a638cf Remove ceph subcommand from storage 
Move everything from under "storage ceph" to "storage" to simplify the
CLI; additional subclasses can be re-added at a future time if and when
additional storage classes are supported.
 2020-01-11 16:03:40 -05:00
Joshua Boniface 83de40307c Fix missing console_log in CLI VM 2020-01-09 13:53:11 -05:00
Joshua Boniface b6474198a4 Implement cluster maintenance mode 
Implements a "maintenance mode" for PVC clusters. For now, the only
thing this mode does is disable node fencing while the state is true.
This allows the administrator to tell PVC that network connectivity,
etc. might be interrupted and to avoid fencing nodes.

Closes #70
 2020-01-09 10:53:27 -05:00
Joshua Boniface 4cda5ebb52 Correct syntax error 2020-01-09 09:46:58 -05:00
Joshua Boniface 6cd3d5c888 Support no-start/no-define in CLI client 2020-01-08 20:13:26 -05:00
Joshua Boniface 9342bc4959 Add extra newline after debug output 2020-01-08 19:41:01 -05:00
Joshua Boniface 4e5bce4975 Update copyright header year to 2020 2020-01-08 19:38:02 -05:00
Joshua Boniface 6e5065511c Move API call to function and implement keys 
Move all API calls to a new common function called call_api to
facilitate easier future changes. Use this function to implement API key
handling via request header value as well as integrate the request URI
generation and debug output handling.

Closes #65
 2020-01-08 19:34:24 -05:00
Joshua Boniface 10d892c698 Support adding API keys to client configs 
Sets the groundwork for the remainder of #65
 2020-01-08 18:43:02 -05:00
Joshua Boniface c515d63340 Add provision state for VMs 2020-01-08 17:40:02 -05:00
Joshua Boniface 9cab1c6eb5 Don't go unhealthy for migrate/unmigrate VMs 2020-01-08 17:37:07 -05:00
Joshua Boniface 23b1f6cb00 Improve handling of progressbar on success 2020-01-08 17:07:57 -05:00
Joshua Boniface a197d9da80 Support wait watching provisioning task 
Addresses #67
 2020-01-08 14:41:46 -05:00
Joshua Boniface c183bc7fef Use click.confirm for all confirmations 
Addresses #67
 2020-01-08 10:34:27 -05:00
Joshua Boniface 9e7d86d5cf Use click.echo_via_pager for VM log output 
Addresses #67
 2020-01-08 10:06:34 -05:00
Joshua Boniface edbe29d06a Use click.edit for modification tasks 
Closes #63
 2020-01-08 09:33:01 -05:00
Joshua Boniface 2005342a2d Correct verb on VM modify 2020-01-08 09:24:17 -05:00
Joshua Boniface d6182d92fc Strip incoming userdata and script documents 2020-01-06 23:52:29 -05:00
Joshua Boniface f4206c2b44 Use correct method for userdata update 2020-01-06 23:39:47 -05:00
Joshua Boniface 62798f9ee8 Use the passed name to modify commands 2020-01-06 23:37:28 -05:00
Joshua Boniface 2aaccfffa7 Use POST body for large data objects 2020-01-06 23:30:34 -05:00
Joshua Boniface a0542d61d0 Fix bad variable name 2020-01-06 16:30:59 -05:00
Joshua Boniface 456a50a73c Send proper delete for OSD deletion 2020-01-06 12:59:00 -05:00
Joshua Boniface 47c49f7672 Fix click import (again) 2020-01-06 12:55:44 -05:00
Joshua Boniface 27279c9686 Properly import click echo 2020-01-06 12:38:20 -05:00
Joshua Boniface 9ea86cc570 Better handle key values 2020-01-06 11:56:34 -05:00
Joshua Boniface 1113234270 Send DELETE instead of POST 2020-01-06 10:47:01 -05:00
Joshua Boniface 66863a0183 Improve output of cluster 2020-01-06 09:23:35 -05:00
Joshua Boniface 007ee56a46 Fix some bugs and where no-cluster message happens 2020-01-06 09:18:22 -05:00
Joshua Boniface be5c67f696 Add messages when adding/removing clusters to CLI 2020-01-06 09:11:00 -05:00
Joshua Boniface 8364d67c51 Correct bad name 2020-01-05 19:19:19 -05:00
Joshua Boniface a9b8403812 Support viewing source volumes 2020-01-05 19:11:52 -05:00
Joshua Boniface 33709828b1 Handle bad response to logs 2020-01-05 17:07:39 -05:00
Joshua Boniface c39c6e8d86 Mention what to do if no cluster found 2020-01-05 15:00:08 -05:00
Joshua Boniface ef293b7778 Fix syntax error 2020-01-05 13:19:21 -05:00
Joshua Boniface b9fdce6728 Use consistent function and stderr for debug 2020-01-05 12:51:06 -05:00
Joshua Boniface baae9b4982 Print initial cluster message to stderr 
Allows it to be easily filtered out
 2020-01-05 12:38:29 -05:00
Joshua Boniface 08570bd7b9 Don't click.echo except in main program 
Return all the strings we printed directly, so that the caller can print
them normally.
 2020-01-05 12:35:00 -05:00
Joshua Boniface 46f3915ab4 Put list handling in the right place 2020-01-05 11:51:03 -05:00
Joshua Boniface 44753c0609 Always handle single-instance results 2020-01-05 02:55:28 -05:00
Joshua Boniface 4fb0d66f6a Use proper variable name 2020-01-05 00:49:50 -05:00
Joshua Boniface 94c60f9daa Mention aborting on no cluster 2020-01-04 15:00:24 -05:00
Joshua Boniface 089b3d7a43 Better output of job status 2020-01-04 14:45:40 -05:00
Joshua Boniface aa2bb7c94c Add create and status functions 2020-01-04 14:31:22 -05:00
Joshua Boniface b88bdc29af Add profile handling 2020-01-04 14:06:36 -05:00
Joshua Boniface a4ad58e64c Add userdata and script options 2020-01-04 13:04:01 -05:00
Joshua Boniface e5d38ec6bd Implement template CLI functions 2020-01-04 11:58:30 -05:00
Joshua Boniface f80b432d3b Handle the empty list better 2020-01-02 12:18:41 -05:00
Joshua Boniface f624cf9e24 Handle no-good-cluster scenarios 2020-01-02 11:19:11 -05:00
Joshua Boniface cb3cc2782a Add initial bits of provisioner CLI 2020-01-02 11:19:08 -05:00
Joshua Boniface d6bd616b31 Fix some bad output 2019-12-30 14:35:44 -05:00
Joshua Boniface 0ced9b2777 Fix incorrect bolding on output 2019-12-30 14:19:17 -05:00
Joshua Boniface 217b163d4b Remove indented empty lines 2019-12-30 13:29:07 -05:00
Joshua Boniface 18e6192178 Add support for multiple clusters in CLI 2019-12-30 13:27:40 -05:00
Joshua Boniface 18c14cbf77 Handle there being no good nodes/VMs 2019-12-30 09:39:04 -05:00
Joshua Boniface 2325f87389 Correct typo in header 2019-12-30 09:07:41 -05:00
Joshua Boniface d2af6f4508 Complete conversion to API client 2019-12-29 20:33:51 -05:00
Joshua Boniface d2f27cc8fe Add network functions to API client 2019-12-29 16:34:32 -05:00
Joshua Boniface d0b6bb4cc3 Convert VM functions to API client 2019-12-29 16:33:50 -05:00
Joshua Boniface a85d8e5c4d Add some QOL improvements 2019-12-29 16:33:50 -05:00
Joshua Boniface e0c7397703 Complete node API client functions 2019-12-29 16:33:50 -05:00
Joshua Boniface 28a0ce772b Node info moved to API 2019-12-29 16:33:50 -05:00
Joshua Boniface 3d178304f1 Convert some node functions to API 2019-12-29 16:33:50 -05:00
Joshua Boniface a9aeb2e836 Implement log viewing/following with API 2019-12-29 16:33:50 -05:00
Joshua Boniface b7eddc6ae9 Add CLI lib 2019-12-29 16:33:50 -05:00
Joshua Boniface 1f6f486b3b Fix broken unwanted change to pvc.py 2019-12-25 13:35:31 -05:00
Joshua Boniface ebcf0cee43 Add doc endpoint 2019-12-23 23:11:55 -05:00
Joshua Boniface 7901ffb5e3 Handle the list of node_limits even better 2019-12-19 13:22:38 -05:00
Joshua Boniface 1924154dce Improve help text 2019-12-08 23:59:17 -05:00
Joshua Boniface 1fb560e996 Add DNS nameservers to networks 2019-12-08 23:55:45 -05:00
Joshua Boniface 32b4ed66e8 Add VM disable command 
Closes #53
 2019-10-23 23:44:47 -04:00
Joshua Boniface 4bff533571 Add cluster status command 2019-10-22 11:23:12 -04:00
Joshua Boniface 03447d3374 Update copyright string year to include 2019 2019-10-13 12:09:51 -04:00
Joshua Boniface 01c8986381 Update help messages in meta command 2019-10-12 02:08:52 -04:00
Joshua Boniface e5393082b6 Use build-in selector and respect limits in client 
Use the new built-in selector option, as well as respecting node limits,
when performing migrate or move actions on a VM via the clients.
 2019-10-12 01:45:44 -04:00
Joshua Boniface 8dc0c8f0ac Fix minor bugs 2019-10-12 01:36:50 -04:00
Joshua Boniface 5995353597 Implement VM metadata and use it 
Implements the storing of three VM metadata attributes:
1. Node limits - allows specifying a list of hosts on which the VM must
run. This limit influences the migration behaviour of VMs.
2. Per-VM node selectors - allows each VM to have its migration
autoselection method specified, to automatically allow different methods
per VM based on the administrator's preferences.
3. VM autorestart - allows a VM to be automatically restarted from a
stopped state, presumably due to a failure to find a target node (either
due to limits or otherwise) during a flush/fence recovery, on the next
node unflush/ready state of its home hypervisor. Useful mostly in
conjunction with limits to ensure that VMs which were shut down due to
there being no valid migration targets are started back up when their
node becomes ready again.

Includes the full client interaction with these metadata options,
including printing, as well as defining a new function to modify this
metadata. For the CLI it is set/modified either on `vm define` or via the
`vm meta` command. For the API it is set/modified either on a POST to
the `/vm` endpoint (during VM definition) or on POST to the `/vm/<vm>`
endpoint. For the API this replaces the previous reserved word for VM
creation from scratch as this will no longer be implemented in-daemon
(see #22).

Closes #52
 2019-10-12 01:17:39 -04:00
Joshua Boniface 94501f400f Remove vm add command 
This is likely not going to be used with the planned implementation of
the automatic provisioning daemon, which will be either an API client or
direct Python binding client.
 2019-10-11 23:52:20 -04:00
Joshua Boniface 6c9b49d7cd Add clone volume command to CLI 2019-10-10 14:11:13 -04:00
Joshua Boniface f6b40911f8 Revert "Add base CLI for deploy section" 
This reverts commit ed43c70f61.
 2019-10-10 14:09:40 -04:00
Joshua Boniface 1c44a1a26e Revert "Add base CLI for deploy image section" 
This reverts commit f918058e6e.
 2019-10-10 14:09:33 -04:00
Joshua Boniface f918058e6e Add base CLI for deploy image section 2019-09-09 10:57:36 -04:00
Joshua Boniface ed43c70f61 Add base CLI for deploy section 2019-08-25 21:18:33 -04:00
Joshua Boniface 3b7c1adf71 Implement configurable replcfg (client-side) 
Implements administrator-selectable replication configurations for new
pools in PVC clusters, overriding the default of copies=3,mincopies=2.
 2019-08-23 14:12:15 -04:00
Joshua Boniface 5876f4589b Add little easter egg to the init command 
Because who doesn't like David Wilcox?
 2019-08-20 09:19:56 -04:00
Joshua Boniface 427ef9454a Use dash in flush-locks command 2019-08-07 17:50:25 -04:00
Joshua Boniface d63d9637a5 Add flush_locks command to CLI client 2019-08-07 13:42:01 -04:00
Joshua Boniface 2dbf276e51 Implement snapshot renaming 
[3/2] Implements #44
 2019-07-28 23:00:35 -04:00
Joshua Boniface 35363671a0 Implement Ceph volume resize and rename 
Includes a simple implementation of a zookeeper "rename" facility,
allowing a key and all data to be replaced by a new key with a different
name but containing all the same child elements and data.

[2/2] Implements #44
 2019-07-26 15:13:21 -04:00
Joshua Boniface d5f263bdd6 Implement Ceph volume resize and rename in clients 
[1/2] Implements #44
 2019-07-26 15:13:21 -04:00
Joshua Boniface 9ccf3390ab Add storage prefix to Ceph commands 
Add the "storage" prefix to all Ceph-based commands in both the CLI and
the API. This partially abstracts the storage subsystem from the Ceph
tool specifically, should future storage subsystems be added or changed.
The name "ceph" is still used due to the non-abstracted components of
the Ceph management, e.g. referencing Ceph-specific concepts like OSDs
or pools.
 2019-07-10 15:14:17 -04:00
Joshua Boniface 7a8aee9fe7 Remove flush locking functionality 
This just seemed like more trouble that it was worth. Flush locks were
originally intended as a way to counteract the weird issues around
flushing that were mostly fixed by the code refactoring, so this will
help test if those issues are truly gone. If not, will look into a
cleaner solution that doesn't result in unchangeable states.
 2019-07-09 23:59:17 -04:00
Joshua Boniface 31b4dd9aed Show radosdf output in clients 2019-07-08 10:59:49 -04:00
Joshua Boniface f75216d946 Make force_migrate not replace existing node 
Previous to this, if once force-migrated a VM, the previous_node value
would be updated to the current node, which is likely never what an
administrator would want. Change this functionality so that the previous
node value is not changed, and update the documentation to reflect this.
 2019-07-07 15:10:48 -04:00
Joshua Boniface 804f72d905 Remove obsolete init attempt 2019-07-06 19:03:55 -04:00
Joshua Boniface 6ee3c91a63 Use None instead of all in ceph.py 
Make it like other optional args (like direction in net ACLs) and use
None instead of 'all' when specifying any option
 2019-07-05 13:59:29 -04:00
Joshua Boniface d755a80fe8 Fix bugs with refactoring 2019-07-05 00:44:40 -04:00
Joshua Boniface 84a905b7f0 Refactor ceph client for API 2019-07-05 00:30:03 -04:00
Joshua Boniface bcab11ee49 Refactor network.py for API usage 2019-07-04 23:01:22 -04:00
Joshua Boniface d28b1ba133 Support removing VMs (and disks) from PVC clients 
Adds full support for removing a VM entirely, including its RBD disks,
via the PVC client(s). Avoids needing an undefine + manual removal.
 2019-06-27 11:19:48 -04:00
Joshua Boniface d5e5ed4b69 Add hostname into confirmation 2019-06-21 15:54:54 -04:00
Joshua Boniface ef272b0b7d Add removal confirmations and zap disk before add 2019-06-21 15:52:28 -04:00
Joshua Boniface 26a4b3b757 Fix logic error in init 2019-06-21 15:38:32 -04:00
Joshua Boniface 519203d7fd Fix bad flag value 2019-06-21 15:09:15 -04:00
Joshua Boniface 93d2e0da6f Improve handling of init 2019-06-21 14:16:32 -04:00
Joshua Boniface 867ad1fc1b Support human-readable biconversion and in volumes 2019-06-21 09:23:52 -04:00
Joshua Boniface 33bf3ee3cf Add confirmation to cluster init 2019-06-21 09:02:58 -04:00
Joshua Boniface 3dd56c55c0 Fix a few more bugs 2019-06-19 15:32:32 -04:00
Joshua Boniface 822cf53a49 Nicer list handling for snapshots 2019-06-19 15:22:44 -04:00
Joshua Boniface 5d4ecafb03 Support listing of snapshots 2019-06-19 15:15:17 -04:00
Joshua Boniface 4ca0fe8f6f Handle per-pool lists better 2019-06-19 14:11:03 -04:00
Joshua Boniface a940d03959 Fix some bugs and add RBD volume stats 2019-06-19 10:25:22 -04:00
Joshua Boniface a4ab3075ab Correct some bugs around new code 2019-06-19 00:23:25 -04:00