parallelvirtualcluster
/
pvc
1
0
Fork 0
Code Issues 11 Pull Requests Projects Releases 109 Wiki Activity
2,206 Commits 1 Branch 111 Tags 27 MiB
Commit Graph

142 Commits

Author SHA1 Message Date
Joshua Boniface a4b80be5ed Add provisioned memory to node info 
Adds a separate field to the node memory, "provisioned", which totals
the amount of memory provisioned to all VMs on the node, regardless of
state, and in contrast to "allocated" which only counts running VMs.

Allows for the detection of potential overprovisioned states when
factoring in non-running VMs.

Includes the supporting code to get this data, since the original
implementation of VM memory selection was dependent on the VM being
running and getting this from libvirt. Now, if the VM is not active, it
gets this from the domain XML instead.
 2020-10-18 14:17:15 -04:00
Joshua Boniface 9d7067469a Correct proper type of uploads 2020-10-16 11:47:09 -04:00
Joshua Boniface 423da08f5f Add colour indication if alloc mem is above total 
Shows an "overprovisioned" state clearly without adding a hacky
additional domain state to the system.
 2020-08-28 00:33:50 -04:00
Joshua Boniface 45542bfd67 Avoid verifying SSL on local connections 
Since these will almost always connect to an IP rather than a "real"
hostname, don't verify the SSL cert (if applicable). Also allow the
overriding of SSL verification via an environment variable.

As a consequence, to reduce spam, SSL warnings are disabled for urllib3.
Instead, we warn in the "Using cluster" output whenever verification is
disabled.
 2020-08-27 23:54:18 -04:00
Joshua Boniface 6dc6dae26c Disable gtod_reduce for benchmarks 
This ended up disabling latency measurements entirely, so don't use this
option for benchmarks.
 2020-08-25 17:02:06 -04:00
Joshua Boniface 0089ec4e17 Multiple KiB values by 1024 in detail output 
Since these are KiB and not B. Also fix some other anomalies.
 2020-08-25 15:01:24 -04:00
Joshua Boniface 2731aa060c Finalize tests and output formatting 2020-08-25 12:16:23 -04:00
Joshua Boniface 18bcd39b46 Use nicer header format 2020-08-25 02:11:34 -04:00
Joshua Boniface d210eef200 Parse response message properly 2020-08-25 02:08:35 -04:00
Joshua Boniface 887e14a4e2 Add storage benchmarking to API 2020-08-25 01:57:21 -04:00
Joshua Boniface 15e78aa9f0 Add status information in cluster status 
Provide textual explanations for the degraded status, including
specific node/VM/OSD issues as well as detailed Ceph health. "Single
pane of glass" mentality.
 2020-08-17 12:25:23 -04:00
Joshua Boniface 2b4d980685 Display Ceph health in PVC status as well 
Makes this output a little more realistic and allows proper monitoring
of the Ceph cluster status (separate from the PVC status which is
tracking only OSD up/in state).
 2020-08-13 15:10:57 -04:00
Joshua Boniface 6fe74b34b2 Use .get for JSON message responses 2020-07-20 12:31:12 -04:00
Joshua Boniface a1ba9d2eeb Allow specifying arbitrary script_args on CLI 
Allow the specifying of arbitrary provisioner script install() args on
the provisioner create CLI, either overriding or adding additional
per-VM arguments to those found in the profile. Reference example is
setting a "vm_fqdn" on a per-run basis.

Closes #100
 2020-07-08 13:18:12 -04:00
Joshua Boniface 37a58d35e8 Implement limiting of node output 
Closes #98
 2020-06-25 11:51:53 -04:00
Joshua Boniface 2967c97f1a Format and display extra VM statistics 2020-06-07 03:04:36 -04:00
Joshua Boniface ce60836c34 Allow enforcement of live migration 
Provides a CLI and API argument to force live migration, which triggers
a new VM state "migrate-live". The node daemon VMInstance during migrate
will read this flag from the state and, if enforced, will not trigger a
shutdown migration.

Closes #95
 2020-06-06 12:00:44 -04:00
Joshua Boniface 9ee5ae4826 Volume and Snapshot are not sorted by ID 2020-05-29 13:43:44 -04:00
Joshua Boniface 48711000b0 Ensure stats sorting is by right key 2020-05-29 13:41:52 -04:00
Joshua Boniface 82c067b591 Sort list output in CLI client properly 2020-05-29 13:39:20 -04:00
Joshua Boniface e724e73140 Don't show built-in bridges as invalid 2020-05-12 10:46:10 -04:00
Joshua Boniface e451426c7c Fix minor bugs from change in VM info handling 2020-04-13 22:56:19 -04:00
Joshua Boniface 8faa3bb53d Handle info fuzzy matches better 
If we are calling info, we want one VM. Don't silently discard other
options or try (and fail later) to parse multiple, just say no VM found.
 2020-04-09 10:26:49 -04:00
Joshua Boniface b3a75d8069 Use post instead of get on initialize 2020-04-06 15:05:33 -04:00
Joshua Boniface 09aeb33d13 Don't convert non-integer bytes/ops 2020-03-30 19:09:16 -04:00
Joshua Boniface 36588a3a81 Work around bad RequestArgs handling 2020-03-03 16:48:20 -05:00
Joshua Boniface 11f045f100 Support showing individual userdata and script doc 
Closes #89
 2020-02-27 14:31:08 -05:00
Joshua Boniface 9d5f50f82a Implement progress bars for file uploads 
Provide pretty status bars to indicate upload progress for tasks that
perform large file uploads to the API ('provisioner ova upload' and
'storage volume upload') so the administrator can gauge progress and
estimated time to completion.
 2020-02-20 22:42:19 -05:00
Joshua Boniface 65932b20d2 Handle request failures more gracefully 2020-02-19 16:19:34 -05:00
Joshua Boniface 1b8b32b07c Don't return tuple value on error 2020-02-19 15:47:08 -05:00
Joshua Boniface 39ce704969 Implement wait for node primary/secondary in CLI 
Use a different wait method of querying the node status every
half-second during the transition, in order to wait on the transition to
complete if desired.

Closes #72
 2020-02-19 14:33:31 -05:00
Joshua Boniface 8678dedfea Revert "Implement wait for node coordinator transition" 
This reverts commit 0aefafa7f7.

This does not work since the API goes away during the transition.

References #72
 2020-02-19 10:50:21 -05:00
Joshua Boniface 0aefafa7f7 Implement wait for node coordinator transition 
References #72
 2020-02-19 10:50:04 -05:00
Joshua Boniface 78780039de Add wait support to VM CLI commands 
References #72
 2020-02-19 10:02:32 -05:00
Joshua Boniface ca68321be3 Allow modification of system templates 
Closes #82
 2020-02-18 16:18:27 -05:00
Joshua Boniface db558ec91f Complete implementation of OVA handling 
Add functions for uploading, listing, and removing OVA images to the API
and CLI interfaces. Includes improved parsing of the OVF and creation of
a system_template and profile for each OVA.

Also modifies some behaviour around profiles, making most components
option at creation to support both profile types (and incomplete
profiles generally).

Implementation part 2/3 - remaining: OVA VM creation

References #71
 2020-02-17 23:22:50 -05:00
Joshua Boniface 59ca296c58 Add basic OVA profile support 2020-02-17 12:00:51 -05:00
Joshua Boniface 1de57ab6f3 Add CLI client interface to image upload 
Closes #68
 2020-02-09 20:42:56 -05:00
Joshua Boniface 4505b239eb Rename API and common Debian packages 
Closes #79
 2020-02-08 18:50:38 -05:00
Joshua Boniface e7728b8375 Allow modification of a VM profile 
And fix some bugs around this.

Fixes #76
 2020-01-30 11:45:46 -05:00
Joshua Boniface 4868625809 Add try/except around new log lines 
Fixes #73
 2020-01-30 09:28:47 -05:00
Joshua Boniface ebfd9c1882 Gracefully handle API connect timeouts 
Prevents dumping a massive stacktrace if connection fails.
 2020-01-30 09:24:10 -05:00
Joshua Boniface 27e73fc6a9 Fix issues with wait on node flush/ready 
Ensure that the strings are converted to booleans on the API side, and
that we're sending a lowercase true/false (for consistency).
 2020-01-26 23:05:30 -05:00
Joshua Boniface 194a642c15 Report errors in log -f 2020-01-15 00:47:14 -05:00
Joshua Boniface c4ef73c953 Refactor provisioner data+list format to match 2020-01-12 14:23:11 -05:00
Joshua Boniface b9177c2f26 Add word Job to task ID output 2020-01-12 14:16:11 -05:00
Joshua Boniface 2057859b9f Add Celery task list output 2020-01-12 14:12:01 -05:00
Joshua Boniface 83de40307c Fix missing console_log in CLI VM 2020-01-09 13:53:11 -05:00
Joshua Boniface b6474198a4 Implement cluster maintenance mode 
Implements a "maintenance mode" for PVC clusters. For now, the only
thing this mode does is disable node fencing while the state is true.
This allows the administrator to tell PVC that network connectivity,
etc. might be interrupted and to avoid fencing nodes.

Closes #70
 2020-01-09 10:53:27 -05:00
Joshua Boniface 6cd3d5c888 Support no-start/no-define in CLI client 2020-01-08 20:13:26 -05:00
Joshua Boniface 9342bc4959 Add extra newline after debug output 2020-01-08 19:41:01 -05:00
Joshua Boniface 4e5bce4975 Update copyright header year to 2020 2020-01-08 19:38:02 -05:00
Joshua Boniface 6e5065511c Move API call to function and implement keys 
Move all API calls to a new common function called call_api to
facilitate easier future changes. Use this function to implement API key
handling via request header value as well as integrate the request URI
generation and debug output handling.

Closes #65
 2020-01-08 19:34:24 -05:00
Joshua Boniface c515d63340 Add provision state for VMs 2020-01-08 17:40:02 -05:00
Joshua Boniface 9cab1c6eb5 Don't go unhealthy for migrate/unmigrate VMs 2020-01-08 17:37:07 -05:00
Joshua Boniface a197d9da80 Support wait watching provisioning task 
Addresses #67
 2020-01-08 14:41:46 -05:00
Joshua Boniface 9e7d86d5cf Use click.echo_via_pager for VM log output 
Addresses #67
 2020-01-08 10:06:34 -05:00
Joshua Boniface 2005342a2d Correct verb on VM modify 2020-01-08 09:24:17 -05:00
Joshua Boniface f4206c2b44 Use correct method for userdata update 2020-01-06 23:39:47 -05:00
Joshua Boniface 62798f9ee8 Use the passed name to modify commands 2020-01-06 23:37:28 -05:00
Joshua Boniface 2aaccfffa7 Use POST body for large data objects 2020-01-06 23:30:34 -05:00
Joshua Boniface 456a50a73c Send proper delete for OSD deletion 2020-01-06 12:59:00 -05:00
Joshua Boniface 47c49f7672 Fix click import (again) 2020-01-06 12:55:44 -05:00
Joshua Boniface 27279c9686 Properly import click echo 2020-01-06 12:38:20 -05:00
Joshua Boniface 9ea86cc570 Better handle key values 2020-01-06 11:56:34 -05:00
Joshua Boniface 1113234270 Send DELETE instead of POST 2020-01-06 10:47:01 -05:00
Joshua Boniface 8364d67c51 Correct bad name 2020-01-05 19:19:19 -05:00
Joshua Boniface a9b8403812 Support viewing source volumes 2020-01-05 19:11:52 -05:00
Joshua Boniface 33709828b1 Handle bad response to logs 2020-01-05 17:07:39 -05:00
Joshua Boniface ef293b7778 Fix syntax error 2020-01-05 13:19:21 -05:00
Joshua Boniface b9fdce6728 Use consistent function and stderr for debug 2020-01-05 12:51:06 -05:00
Joshua Boniface 08570bd7b9 Don't click.echo except in main program 
Return all the strings we printed directly, so that the caller can print
them normally.
 2020-01-05 12:35:00 -05:00
Joshua Boniface 46f3915ab4 Put list handling in the right place 2020-01-05 11:51:03 -05:00
Joshua Boniface 44753c0609 Always handle single-instance results 2020-01-05 02:55:28 -05:00
Joshua Boniface 089b3d7a43 Better output of job status 2020-01-04 14:45:40 -05:00
Joshua Boniface aa2bb7c94c Add create and status functions 2020-01-04 14:31:22 -05:00
Joshua Boniface b88bdc29af Add profile handling 2020-01-04 14:06:36 -05:00
Joshua Boniface a4ad58e64c Add userdata and script options 2020-01-04 13:04:01 -05:00
Joshua Boniface e5d38ec6bd Implement template CLI functions 2020-01-04 11:58:30 -05:00
Joshua Boniface cb3cc2782a Add initial bits of provisioner CLI 2020-01-02 11:19:08 -05:00
Joshua Boniface d6bd616b31 Fix some bad output 2019-12-30 14:35:44 -05:00
Joshua Boniface 18c14cbf77 Handle there being no good nodes/VMs 2019-12-30 09:39:04 -05:00
Joshua Boniface 2325f87389 Correct typo in header 2019-12-30 09:07:41 -05:00
Joshua Boniface d2af6f4508 Complete conversion to API client 2019-12-29 20:33:51 -05:00
Joshua Boniface d2f27cc8fe Add network functions to API client 2019-12-29 16:34:32 -05:00
Joshua Boniface d0b6bb4cc3 Convert VM functions to API client 2019-12-29 16:33:50 -05:00
Joshua Boniface a85d8e5c4d Add some QOL improvements 2019-12-29 16:33:50 -05:00
Joshua Boniface e0c7397703 Complete node API client functions 2019-12-29 16:33:50 -05:00
Joshua Boniface 28a0ce772b Node info moved to API 2019-12-29 16:33:50 -05:00
Joshua Boniface 3d178304f1 Convert some node functions to API 2019-12-29 16:33:50 -05:00
Joshua Boniface a9aeb2e836 Implement log viewing/following with API 2019-12-29 16:33:50 -05:00
Joshua Boniface b7eddc6ae9 Add CLI lib 2019-12-29 16:33:50 -05:00