parallelvirtualcluster
/
pvc
1
0
Fork 0
Code Issues 11 Pull Requests Projects Releases 109 Wiki Activity
2,092 Commits 1 Branch 111 Tags 27 MiB
Commit Graph

171 Commits

Author SHA1 Message Date
Joshua Boniface 973c78b8e0 Use monkeypatch to allow multithreaded prod flask 
Without this tasks were blocking when other task were active (for
instance, any task with --wait). Using the moneypatch, these no longer
block.
 2020-08-28 02:09:31 -04:00
Joshua Boniface b51f0a339d Fix bug in SSL enabled WSGI server 2020-08-26 13:52:45 -04:00
Joshua Boniface 6dc6dae26c Disable gtod_reduce for benchmarks 
This ended up disabling latency measurements entirely, so don't use this
option for benchmarks.
 2020-08-25 17:02:06 -04:00
Joshua Boniface 486408753b Don't print results to output 2020-08-25 13:38:46 -04:00
Joshua Boniface 169e174d85 Fix size of test volume to 8GB 2020-08-25 13:29:22 -04:00
Joshua Boniface eb06c1494e Add API spec for benchmark results 2020-08-25 12:43:16 -04:00
Joshua Boniface bb7b1a2bd0 Remove aggrpct from results 
This value is useless to us since we're not running combined read/write
tests at all.
 2020-08-25 12:38:49 -04:00
Joshua Boniface 70b9caedc3 Correct typo 2020-08-25 12:23:12 -04:00
Joshua Boniface 2731aa060c Finalize tests and output formatting 2020-08-25 12:16:23 -04:00
Joshua Boniface 1dcc1f6d55 Rename sample database for API 
From pvcprov to pvcapi to facilitate the changing nature of this
database and its expansion to benchmark results.
 2020-08-25 01:59:35 -04:00
Joshua Boniface 887e14a4e2 Add storage benchmarking to API 2020-08-25 01:57:21 -04:00
Joshua Boniface 3dbdd12d8f Correct invalid comparison in template VNI add 2020-08-18 09:48:56 -04:00
Joshua Boniface 2b4d980685 Display Ceph health in PVC status as well 
Makes this output a little more realistic and allows proper monitoring
of the Ceph cluster status (separate from the PVC status which is
tracking only OSD up/in state).
 2020-08-13 15:10:57 -04:00
Joshua Boniface 0d470ae5f6 Work around formatting fail 2020-08-12 12:12:16 -04:00
Joshua Boniface 0dd719a682 Use single-quotes so Python isn't confused 2020-08-11 17:24:11 -04:00
Joshua Boniface 5526e13da9 Move all host provisioner steps to a try block 
Make the provisioner a bit more robust. This way, even if a provisioning
step fails, cleanup is still performed this preventing the system from
being left in an undefined state requiring manual correction.

Addresses #91
 2020-08-06 12:27:10 -04:00
Joshua Boniface bafdcf9f8c Use new_size to match new_name 2020-08-05 10:25:37 -04:00
Joshua Boniface 9f86f12f1a Only parse script_run_args if not None 2020-07-16 02:36:26 -04:00
Joshua Boniface be405caa11 Remove spurious print statement 2020-07-08 13:28:47 -04:00
Joshua Boniface a1ba9d2eeb Allow specifying arbitrary script_args on CLI 
Allow the specifying of arbitrary provisioner script install() args on
the provisioner create CLI, either overriding or adding additional
per-VM arguments to those found in the profile. Reference example is
setting a "vm_fqdn" on a per-run basis.

Closes #100
 2020-07-08 13:18:12 -04:00
Joshua Boniface 37a58d35e8 Implement limiting of node output 
Closes #98
 2020-06-25 11:51:53 -04:00
Joshua Boniface 4cdf1f7247 Add statistics values to the API 2020-06-07 02:15:33 -04:00
Joshua Boniface 654a3cb7fa Improve debug output and use ceph df util data 2020-06-06 22:52:49 -04:00
Joshua Boniface ce60836c34 Allow enforcement of live migration 
Provides a CLI and API argument to force live migration, which triggers
a new VM state "migrate-live". The node daemon VMInstance during migrate
will read this flag from the state and, if enforced, will not trigger a
shutdown migration.

Closes #95
 2020-06-06 12:00:44 -04:00
Joshua Boniface 73eb4fb457 Fix typo of macaddress in dhcp add 2020-05-09 00:15:25 -04:00
Joshua Boniface b91923735c Move some messages around 2020-05-05 16:19:18 -04:00
Joshua Boniface 34c4690d49 Don't convert bytes into KB in OVA import 
Doing so can create an image that is 1 sector (512 bytes) too large,
which will then break qemu-img because it's stupid (or, VMDK is stupid,
I haven't decided which is).. Current Ceph rbd commands seem to accept
--size in bytes so this is fine.
 2020-05-05 16:14:18 -04:00
Joshua Boniface 3e351bb84a Add additional error checking for profile creation 2020-05-05 15:28:39 -04:00
Joshua Boniface 4a52ff56b9 Catch failures in getPoolInformation 
Fixes #90
 2020-03-15 16:58:13 -04:00
Joshua Boniface ca5327b908 Make strtobool even more robust 
If strtobool fails, return False always.
 2020-03-09 09:30:16 -04:00
Joshua Boniface d36d8e0637 Use custom strtobool to handle weird edge cases 2020-03-06 09:40:13 -05:00
Joshua Boniface b8852e116e Improve handling of root disk in GRUB 
Since vdX names become sdX names inside VMs, use the same setup as the
fstab in order to map this onto a static SCSI ID.
 2020-03-02 10:02:39 -05:00
Joshua Boniface fd80eb9e22 Ensure profile creation works with empty lists 
If we get a 404 code back from the upper function, we should create an
empty list rather than trying to loop through the dictionary.
 2020-02-24 09:30:58 -05:00
Joshua Boniface 6ac82d6ce9 Ensure single-element templates are lists 
Ensures any list-assuming statements later on hold true even when there
is only a single template entry.
 2020-02-21 10:50:28 -05:00
Joshua Boniface b438b9b4c2 Import gevent for production listener 2020-02-21 09:39:07 -05:00
Joshua Boniface 56a9e48163 Normalize all return messages 
Ensure all API return messages are formated the same: no "error", a
final period except when displaying Exception text, and a regular spaced
out format.
 2020-02-20 22:42:19 -05:00
Joshua Boniface 31a117e21c Fix call to config dictionary from pvc_api 2020-02-20 15:11:20 -05:00
Joshua Boniface 8678dedfea Revert "Implement wait for node coordinator transition" 
This reverts commit 0aefafa7f7.

This does not work since the API goes away during the transition.

References #72
 2020-02-19 10:50:21 -05:00
Joshua Boniface 0aefafa7f7 Implement wait for node coordinator transition 
References #72
 2020-02-19 10:50:04 -05:00
Joshua Boniface 99f579e41a Add wait support to API commands 
References #72
 2020-02-19 09:51:42 -05:00
Joshua Boniface 097f0d9be4 Fix bug with script load from database 2020-02-18 20:39:36 -05:00
Joshua Boniface ca68321be3 Allow modification of system templates 
Closes #82
 2020-02-18 16:18:27 -05:00
Joshua Boniface b322841edf Complete integration of OVA provisioner 
Finishes a basic form of OVA provisioning within the existing create_vm
function. Future plans should include separating out the functions and
cleaning them up a bit more, but this is sufficient for basic operation.

Closes #71
 2020-02-18 14:42:45 -05:00
Joshua Boniface 4c58addead Fix typo'd storage_host entry 2020-02-18 14:42:32 -05:00
Joshua Boniface e811c5bbfb Fix renamed import for worker 2020-02-18 12:20:42 -05:00
Joshua Boniface 24c86f2c42 Remove obsolete print statement 2020-02-17 23:25:27 -05:00
Joshua Boniface db558ec91f Complete implementation of OVA handling 
Add functions for uploading, listing, and removing OVA images to the API
and CLI interfaces. Includes improved parsing of the OVF and creation of
a system_template and profile for each OVA.

Also modifies some behaviour around profiles, making most components
option at creation to support both profile types (and incomplete
profiles generally).

Implementation part 2/3 - remaining: OVA VM creation

References #71
 2020-02-17 23:22:50 -05:00
Joshua Boniface 59ca296c58 Add basic OVA profile support 2020-02-17 12:00:51 -05:00
Joshua Boniface c18c76f42c Add alembic script_location field 2020-02-17 11:36:33 -05:00
Joshua Boniface d975f90f29 Add database fields for OVA storage 2020-02-17 11:27:35 -05:00
Joshua Boniface 8125aea4f3 Clean up some database columns 2020-02-16 19:19:04 -05:00
Joshua Boniface f3de900bdb Import all database models 2020-02-16 19:15:21 -05:00
Joshua Boniface 760805fec1 Ensure database migrations are in source control 2020-02-15 23:16:40 -05:00
Joshua Boniface 158ed8d3f0 Remove obsolete schema definition 2020-02-15 23:04:21 -05:00
Joshua Boniface 574623f2a8 Remove obsolete script 2020-02-15 22:59:12 -05:00
Joshua Boniface 560cb609ba Add database management with SQLAlchemy 
Add management of the pvcprov database with SQLAlchemy, to allow
seamless management of the database. Add automatic tasks to the postinst
of the API to execute these migrations.
 2020-02-15 22:51:27 -05:00
Joshua Boniface bd8536d9d1 Add OVA upload to API (initial) 
Initial, very barebones OVA parsing and image creation.

References #71
 2020-02-15 02:10:14 -05:00
Joshua Boniface b29c69378d Just describe the body in words 2020-02-09 21:08:27 -05:00
Joshua Boniface ad60f4b1f1 Try again with just query 2020-02-09 21:06:33 -05:00
Joshua Boniface 68638d7760 Use in: body for body contents 2020-02-09 21:05:15 -05:00
Joshua Boniface 602c2f9d4a Use request instead of requestBody 2020-02-09 21:02:13 -05:00
Joshua Boniface c979fed10a Use proper requestBody description of file 2020-02-09 20:59:03 -05:00
Joshua Boniface 1231ba19b7 Ensure image_format is mandatory 
References #68
 2020-02-09 20:45:43 -05:00
Joshua Boniface e419855911 Support converting types during upload 
Allow the user to specify other, non-raw files and upload them,
performing a conversion with qemu-img convert and a temporary block
device as a shim (since qemu-img can't use FIFOs).

Also ensures that the target volume exists before proceeding.

Addresses #68
 2020-02-09 20:29:12 -05:00
Joshua Boniface 49e5ce1176 Support uploading disk images to volumes in API 
Addresses #68
 2020-02-09 13:45:04 -05:00
Joshua Boniface eeb8879f73 Move run_os_command to common functions 
References #80
 2020-02-08 23:33:15 -05:00
Joshua Boniface 26c2c2c295 Further split API so only Flask can be loaded 2020-02-08 20:36:53 -05:00
Joshua Boniface 4f25c55efc Fix startup of API daemon 
References #79
 2020-02-08 20:27:45 -05:00
Joshua Boniface 3532dcc11f Update startup of API and Swagger generator 
References #79
 2020-02-08 19:52:15 -05:00
Joshua Boniface ce985234c3 Use consistent naming of components 
Rename "pvcd" to "pvcnoded", and "pvc-api" to "pvcapid" so names for the
daemons are fully consistent. Update the names of the configuration
files as well to match this new formatting.

References #79
 2020-02-08 19:34:07 -05:00
Joshua Boniface 4505b239eb Rename API and common Debian packages 
Closes #79
 2020-02-08 18:50:38 -05:00