parallelvirtualcluster
/
pvc-ansible
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
364 Commits 1 Branch 0 Tags 2.4 MiB
Commit Graph

187 Commits

Author SHA1 Message Date
Joshua Boniface 4ccc23bd85 Add python3 version of psycopg2 explicitly 2021-06-22 03:15:23 -04:00
Joshua Boniface 238449904f Move some other tasks to bootstrap role 
Avoids an issue where the pvcnoded service is stopped on non-bootstrap
runs.
 2021-05-13 10:17:38 -04:00
Joshua Boniface 404751f695 Update relative path to bootstrap files 2021-04-08 14:04:56 -04:00
Joshua Boniface 622cef1586 Remove superfluous symlink 2021-04-08 13:50:47 -04:00
Joshua Boniface 82fa85834a Add libguestfs-tools to libvirt role deps 2021-03-15 13:39:37 -04:00
Joshua Boniface 1c05c8729f Fix incorrect systemd enabling in Patroni 2021-01-28 16:28:02 -05:00
Joshua Boniface 04ca8f73d2 Correct bugs during bootstrap 
1. Ensure Zookeeper restarts and checks out successfully before
proceeding with other steps.
2. Make sure PVC itself doesn't start prematurely.
 2021-01-28 13:32:36 -05:00
Joshua Boniface b7f251ea16 Retry pgsql bootstrap startup 6 times 
This will sometimes fail, so retry it several times
 2021-01-27 15:45:36 -05:00
Joshua Boniface 7b08610efa Retry msgr2 enabling 6 times 
This will sometimes fail, so retry it several times
 2021-01-27 14:13:09 -05:00
Joshua Boniface c4c285c7b3 Remove invalid timezone entries in postgres conf 2021-01-26 15:20:25 -05:00
Joshua Boniface 7585553225 Add default values 2020-12-21 00:20:45 -05:00
Joshua Boniface ac071f4bf0 Add configurable ZK memory limits 2020-12-21 00:20:45 -05:00
Joshua Boniface 98e3e39570 Remove libjemalloc package 2020-12-21 00:20:45 -05:00
Joshua Boniface 8e104113d7 Tune Zookeeper memory usage 
Use Xms and Xmx=128M to reduce overall Zookeeper memory usage.
 2020-12-21 00:20:45 -05:00
Joshua Boniface de04105a38 Add tuning for Ceph OSDs 2020-12-21 00:20:45 -05:00
Joshua Boniface 28c86d170f Don't use libjemalloc for Ceph daemons 
This was an artifact of a much, much older Ceph configuration I ran, and
is not relevant with newer Ceph versions like those used in PVC.
Performance testing with Nautilus and Bluestore reveals a minimal
performance hit, and using `jemalloc` prevents cache autotuning from
being effective, so remove it.
 2020-12-21 00:20:45 -05:00
Joshua Boniface cb96ef4e7a Use new init command location 
Command was renamed in the PVC CLI to facilitate other "task" actions
like backup/restore.
 2020-11-24 12:22:34 -05:00
Joshua Boniface 3c0c3e8e56 Add jute.maxbuffer to Zookeeper environment ops 
Adds this option based on the findings of
https://github.com/python-zk/kazoo/issues/630, whereby restores of >1MB
in size would fail. This is considered an unsafe option, but given our
usecase no actual znode should ever exceed this limit; this is purely
for the large transactions that come from a `pvc task restore` action to
an empty Zookeeper instance.
 2020-11-24 12:20:25 -05:00
Joshua Boniface 934f73af0f Support using existing SSL certs on system 
Add the additional pvc_api_ssl_cert_path and pvc_api_ssl_key_path
group_vars options, which can be used to set the SSL details to existing
files on the filesystem if desired. If these are empty (or nonexistent),
the original pvc_api_ssl_cert and pvc_api_ssl_key raw format options
will be used as they were.

Allows the administrator to use outside methods (such as Let's Encrypt)
to obtain the certs locally on the system, avoiding changes to the
group_vars and redeployment to manage SSL keys.
 2020-08-26 14:11:14 -04:00
Joshua Boniface 2f2123b70e Rename remaining "pvc_prov" items to pvc_api 2020-08-25 13:01:48 -04:00
Joshua Boniface d79c587384 Change name of default API database 
From pvcprov to pvcapi to reflect the changing use of this database.
 2020-08-25 02:00:29 -04:00
Joshua Boniface 663d525bb1 Add comments to defaults 2020-08-21 09:40:51 -04:00
Joshua Boniface e32dfe6200 Add additional configuration to group_vars 
Also include defaults and the new pvc_vm_shutdown_timeout option.
 2020-08-20 21:39:44 -04:00
Joshua Boniface 774595cdb7 Ensure ZK prioritizes IPv4 2020-08-19 13:10:03 -04:00
Joshua Boniface c9b487f5e6 Use FQDN for Zookeeper server entries 2020-08-19 12:47:06 -04:00
Joshua Boniface 6b8232d38e Use cluster_group variable for paths 
Instead of trying to automagic this group out of the Ansible hostvars,
just make it explicitly defined in the group_vars to avoid any
confusion.
 2020-08-06 13:20:14 -04:00
Joshua Boniface a488f62ef8 Ignore errors in bringing up bootstrap interfaces 2020-07-27 13:08:24 -04:00
Joshua Boniface 69b0590b54 Add storage components to default pvcnoded.yaml 2020-06-06 21:15:10 -04:00
Joshua Boniface b0186b85c2 Use correct syntax for init command 2020-04-06 15:19:49 -04:00
Joshua Boniface af1927e384 Use consistent naming in patroni.yml 2020-04-06 14:33:13 -04:00
Joshua Boniface f90f8f33da Use short names in PVC configs 2020-04-06 13:54:39 -04:00
Joshua Boniface f560f55010 Use shortname for Zookeeper 2020-04-06 13:45:29 -04:00
Joshua Boniface e37f2af6cd Use local CLI command instead of API to init 2020-04-06 13:36:38 -04:00
Joshua Boniface b9f6284e36 Use only short names in Ceph MON config 2020-04-06 13:36:38 -04:00
Joshua Boniface fe40811f2b Fix conditional checks with inventory_hostname 2020-04-06 13:36:38 -04:00
Joshua Boniface 2afccf44fb Handle bridge creation more sensibly 2020-04-06 13:36:38 -04:00
Joshua Boniface d60eabf63d Don't restart pvcd.service on bootstrap 2020-02-20 14:34:48 -05:00
Joshua Boniface a79aef90fa Allow deb migrations to be installed 2020-02-15 23:30:11 -05:00
Joshua Boniface eaf9467b75 Add symlink for pvc files dir 2020-02-15 23:02:33 -05:00
Joshua Boniface f5cd8a94c2 Handle creation and collection on bootstrap better 2020-02-15 23:01:32 -05:00
Joshua Boniface b922d47458 Use new in-built database migrations in API 2020-02-15 22:49:48 -05:00
Joshua Boniface 67d1f6761a Use new package and file names 
References parallelvirtualclient/pvc#79
 2020-02-08 19:47:47 -05:00
Joshua Boniface 94f2cd5c86 Don't mess with upstream at all during bootstrap 
This caused some major breakage and is not required.
 2020-01-13 15:12:54 -05:00
Joshua Boniface 7d6052f9cb Modify add_cluster_ips to support new bridges 2020-01-12 19:46:27 -05:00
Joshua Boniface 00315e01c3 Enable and start vhostmd service 2020-01-07 10:45:12 -05:00
Joshua Boniface d9b3f15381 Add source_volume column to storage table 2020-01-06 23:54:48 -05:00
Joshua Boniface 03779056c7 Add new empty script entry 2020-01-06 23:54:48 -05:00
Joshua Boniface cd7cdf2719 Add bridge_device entry to config 
Used to properly allow bridged networks to be formed.

Ref parallelvirtualcluster/pvc#64
 2020-01-06 14:35:25 -05:00
Joshua Boniface a1efa2f01a Fix additional reference to userdata_template 2020-01-04 13:41:03 -05:00
Joshua Boniface 761715d015 Adjust provisioner database schema 2020-01-04 12:13:11 -05:00
Joshua Boniface dcd3194432 Set msgr2 mode on Ceph monitors 2019-12-30 09:13:50 -05:00
Joshua Boniface a66d17252f Apply fix with some tweaks to other serial handlers 2019-12-25 13:45:29 -05:00
Joshua Boniface af606ac49c Change ordering of networks in file 2019-12-25 13:31:02 -05:00
Joshua Boniface a30edbfa54 Replace broken "serial" restarts with a new method 2019-12-25 13:30:37 -05:00
Joshua Boniface bf4de842d8 Correct bad address in pvcd.yaml 2019-12-25 12:57:51 -05:00
Joshua Boniface c5da6381c9 Set provisioner database in pvcd.yaml 2019-12-25 12:37:32 -05:00
Joshua Boniface 1dda60d301 Add and remove floating IP during cluster bootstrap 2019-12-25 12:12:53 -05:00
Joshua Boniface ee948cb91c Move netmask to separate config part 3 2019-12-24 14:27:31 -05:00
Joshua Boniface 79dd0cd4bc Ensure the Patroni ZK is clean for bootstrap 2019-12-24 14:17:41 -05:00
Joshua Boniface 06467b64ea Move netmask to separate config part 2 2019-12-24 14:16:20 -05:00
Joshua Boniface 22c6c13f0f Don't try to do crazy restart ordering, it fails 2019-12-24 14:15:52 -05:00
Joshua Boniface 73617fa1a6 Change Patroni scope to just pvc 2019-12-24 14:15:39 -05:00
Joshua Boniface a10fc7eb3f Move netmask to separate config 2019-12-24 14:15:14 -05:00
Joshua Boniface ebee10747c Use API endpoint to bootstrap PVC cluster 2019-12-24 09:08:21 -05:00
Joshua Boniface 93f44dd9dc Add additional API configuration 2019-12-23 23:25:27 -05:00
Joshua Boniface a37f511241 Install Provisioner schema to database 2019-12-23 23:19:09 -05:00
Joshua Boniface e08e19ee64 Add provisioner database schema 2019-12-23 12:58:03 -05:00
Joshua Boniface 4d2ef3b622 Remove empty newline 2019-12-23 12:57:46 -05:00
Joshua Boniface 64157e8c89 Remove invalid flag to ceph-authtool 2019-12-14 14:10:35 -05:00
Joshua Boniface 6bfc83e8f3 Don't become for uuidgen 2019-12-14 13:51:43 -05:00
Joshua Boniface 1c2f972e93 Move Ceph access to storage network 2019-12-14 13:14:21 -05:00
Joshua Boniface 80fdc88235 Improve script to run ZK cleanup on all hosts 2019-12-01 20:29:47 -05:00
Joshua Boniface d78d682fe5 Add jq dependency 2019-12-01 20:26:08 -05:00
Joshua Boniface 6a29400525 Make vacuum script more comprehensive 2019-12-01 20:24:18 -05:00
Joshua Boniface 273c048e6a Add check_mk check for PVC status 2019-10-24 09:46:10 -04:00
Joshua Boniface 0336cd998f Improve daily vacuum script 2019-10-24 09:43:24 -04:00
Joshua Boniface 7f5a7e48f8 Add daily Zookeeper data cleanup 2019-08-26 11:09:23 -04:00
Joshua Boniface ef9673de02 Add custom systemd unit for Zookeeper 
We're 100% systemd here, and the lack of control/information that the
old-school ZK initscript provides is frustrating. Replace it with our
own simple unit file.
 2019-08-26 11:06:30 -04:00
Joshua Boniface b75e84a124 Add logrotate configuration 2019-08-11 15:41:10 -04:00
Joshua Boniface 030a3ded99 Add daily Postgres vacuum script 2019-08-11 15:29:00 -04:00
Joshua Boniface 91509720ac Add Zookeeper autopurge @72h 2019-08-05 13:16:09 -04:00
Joshua Boniface 005ba71fc8 Update config template with recent changes 
1) Add debug flag
2) Move intervals config up one level
 2019-08-01 13:21:12 -04:00
Joshua Boniface ada3cb1d87 Set debug value in API config 2019-07-26 11:44:08 -04:00
Joshua Boniface dc27564157 Limit database tasks to coordinators only 
Non-coordinators don't need these configurations as they shouldn't run
there.
 2019-07-11 19:58:56 -04:00
Joshua Boniface db2c77d330 Support new log flags and update default log conf 
Tweak the defaults a bit; pvc-ansible assumes we're running under
systemd, so set a log format that's better for it (no colour or date).
 2019-07-10 21:49:38 -04:00
Joshua Boniface 0d562b829c Replace tabs with spaces 2019-07-08 19:24:59 -04:00
Joshua Boniface 6319241df9 Remove bad content from pvc-api.yml 2019-07-08 19:03:08 -04:00
Joshua Boniface e98649c417 Add quote around ZK nodes in Patroni 2019-07-08 16:59:12 -04:00
Joshua Boniface 7d3ba8bbc3 Bring up underlying interfaces 2019-07-08 14:40:02 -04:00
Joshua Boniface 75182b1570 Complete configuration of API via Ansible 2019-07-08 11:33:14 -04:00
Joshua Boniface d19e006f28 Enable client API in Daemon config 2019-07-06 02:50:05 -04:00
Joshua Boniface c438d2c9cd Add client API to configuration 2019-07-06 02:49:25 -04:00
Joshua Boniface 12d30acf90 Add only coordinators in Patroni config 2019-07-06 02:49:05 -04:00
Joshua Boniface 9b9234eeb1 Install ethtool 2019-06-25 22:13:16 -04:00
Joshua Boniface 140e044e96 Make libvirt listen only on cluster IP 2019-06-25 22:12:44 -04:00
Joshua Boniface 5dbb48abc4 Add debootstrap to package list 2019-06-24 12:22:24 -04:00
Joshua Boniface fc6475b872 Don't try to set pool limits on libvirt key 
I figured a * wildcard would work, but no it doesn't. Libvirt needs
the ability to talk to any pool arbitrarily since PVC can create and
remove them at will.
 2019-06-24 10:49:35 -04:00
Joshua Boniface 6031053716 Create mgr after starting monitors 2019-06-24 09:53:04 -04:00
Joshua Boniface 9711a31580 Set nicer default Patroni values re: replication 2019-06-23 21:03:24 -04:00
Joshua Boniface 86d4579c82 Split PVC bootstrap into separate task 2019-06-21 14:18:26 -04:00