Joshua Boniface
bcb5962353
Add jute.maxbuffer to Zookeeper environment ops
...
Adds this option based on the findings of
https://github.com/python-zk/kazoo/issues/630 , whereby restores of >1MB
in size would fail. This is considered an unsafe option, but given our
usecase no actual znode should ever exceed this limit; this is purely
for the large transactions that come from a `pvc task restore` action to
an empty Zookeeper instance.
2023-09-01 15:42:24 -04:00
Joshua Boniface
075ce8ea22
Add PVC status MOTD script
2023-09-01 15:42:24 -04:00
Joshua Boniface
68a475ccf9
Set proper mode on agent plugins
2023-09-01 15:42:24 -04:00
Joshua Boniface
f86ec62416
Add check-mk-agent plugin installs
...
These are used by various Ansible tasks, even if the administrator is
not using Check_MK for monitoring.
2023-09-01 15:42:24 -04:00
Joshua Boniface
62d53b0c9c
Add PCI and USB utils
2023-09-01 15:42:24 -04:00
Joshua Boniface
f79fb605de
Support using existing SSL certs on system
...
Add the additional pvc_api_ssl_cert_path and pvc_api_ssl_key_path
group_vars options, which can be used to set the SSL details to existing
files on the filesystem if desired. If these are empty (or nonexistent),
the original pvc_api_ssl_cert and pvc_api_ssl_key raw format options
will be used as they were.
Allows the administrator to use outside methods (such as Let's Encrypt)
to obtain the certs locally on the system, avoiding changes to the
group_vars and redeployment to manage SSL keys.
2023-09-01 15:42:24 -04:00
Joshua Boniface
a8419be587
Use generic Debian repos and PVC component
2023-09-01 15:42:24 -04:00
Joshua Boniface
2caed2ae12
Rename remaining "pvc_prov" items to pvc_api
2023-09-01 15:42:24 -04:00
Joshua Boniface
2a2d318dbc
Change name of default API database
...
From pvcprov to pvcapi to reflect the changing use of this database.
2023-09-01 15:42:24 -04:00
Joshua Boniface
833d99a360
Add comments to defaults
2023-09-01 15:42:24 -04:00
Joshua Boniface
8109f13386
Add additional configuration to group_vars
...
Also include defaults and the new pvc_vm_shutdown_timeout option.
2023-09-01 15:42:24 -04:00
Joshua Boniface
72df058684
Ensure ZK prioritizes IPv4
2023-09-01 15:42:24 -04:00
Joshua Boniface
457e18a850
Use FQDN for Zookeeper server entries
2023-09-01 15:42:24 -04:00
Joshua Boniface
777a4693a1
Improve SSH configuration for nodes
...
Ensure hostbased auth works with configs, remove erroneous old
conditional for authtypes, remove obsolete config option.
2023-09-01 15:42:24 -04:00
Joshua Boniface
88209a2b70
Use Google DNS instead of Cloudflare
...
For some reason Cloudflare works in fewer places than Google, so just
use it instead.
2023-09-01 15:42:24 -04:00
Joshua Boniface
fbbf5ffe09
Use cluster_group variable for paths
...
Instead of trying to automagic this group out of the Ansible hostvars,
just make it explicitly defined in the group_vars to avoid any
confusion.
2023-09-01 15:42:23 -04:00
Joshua Boniface
a925e4bd40
Ignore errors in bringing up bootstrap interfaces
2023-09-01 15:42:23 -04:00
Joshua Boniface
e3ad750412
Add storage components to default pvcnoded.yaml
2023-09-01 15:42:23 -04:00
Joshua Boniface
715fa103cd
Ensure uuid-runtime is installed
2023-09-01 15:42:23 -04:00
Joshua Boniface
12d50cfca6
Use correct syntax for init command
2023-09-01 15:42:23 -04:00
Joshua Boniface
92ccc0a737
Use consistent naming in patroni.yml
2023-09-01 15:42:23 -04:00
Joshua Boniface
0566aadfb0
Remove obsolete issue-gen script on install
2023-09-01 15:42:23 -04:00
Joshua Boniface
c35c58389d
Use short names in PVC configs
2023-09-01 15:42:23 -04:00
Joshua Boniface
157c56fd46
Use shortname for Zookeeper
2023-09-01 15:42:23 -04:00
Joshua Boniface
7e653d52c3
Include upstream and short names in hosts
2023-09-01 15:42:23 -04:00
Joshua Boniface
6a3c32f306
Use local CLI command instead of API to init
2023-09-01 15:42:23 -04:00
Joshua Boniface
c71415317a
Use only short names in Ceph MON config
2023-09-01 15:42:23 -04:00
Joshua Boniface
52862f9daf
Fix conditional checks with inventory_hostname
2023-09-01 15:42:23 -04:00
Joshua Boniface
91313e848e
Handle bridge creation more sensibly
2023-09-01 15:42:23 -04:00
Joshua Boniface
6d3999eaab
Don't restart pvcd.service on bootstrap
2023-09-01 15:42:23 -04:00
Joshua Boniface
0d9e209b45
Allow deb migrations to be installed
2023-09-01 15:42:23 -04:00
Joshua Boniface
4b89aff1d8
Add symlink for pvc files dir
2023-09-01 15:42:23 -04:00
Joshua Boniface
8c15edd75c
Handle creation and collection on bootstrap better
2023-09-01 15:42:23 -04:00
Joshua Boniface
b4079cae88
Use new in-built database migrations in API
2023-09-01 15:42:23 -04:00
Joshua Boniface
0e5cb688dc
Use new package and file names
...
References parallelvirtualclient/pvc#79
2023-09-01 15:42:23 -04:00
Joshua Boniface
999e50a68f
Don't mess with upstream at all during bootstrap
...
This caused some major breakage and is not required.
2023-09-01 15:42:23 -04:00
Joshua Boniface
e7e7f2cc96
Don't remove nano
2023-09-01 15:42:22 -04:00
Joshua Boniface
42d76618e3
Modify add_cluster_ips to support new bridges
2023-09-01 15:42:22 -04:00
Joshua Boniface
32b719cb4a
Enable and start vhostmd service
2023-09-01 15:42:22 -04:00
Joshua Boniface
b654be8825
Add source_volume column to storage table
2023-09-01 15:42:22 -04:00
Joshua Boniface
e3f83713a0
Add new empty script entry
2023-09-01 15:42:22 -04:00
Joshua Boniface
f68ba7a735
Add bridge_device entry to config
...
Used to properly allow bridged networks to be formed.
Ref parallelvirtualcluster/pvc#64
2023-09-01 15:42:22 -04:00
Joshua Boniface
9848eb10bb
Fix additional reference to userdata_template
2023-09-01 15:42:22 -04:00
Joshua Boniface
f3212d5e4f
Adjust provisioner database schema
2023-09-01 15:42:22 -04:00
Joshua Boniface
bc1d9cd33b
Set msgr2 mode on Ceph monitors
2023-09-01 15:42:22 -04:00
Joshua Boniface
372b949930
Apply fix with some tweaks to other serial handlers
2023-09-01 15:42:22 -04:00
Joshua Boniface
15768130e2
Change ordering of networks in file
2023-09-01 15:42:22 -04:00
Joshua Boniface
146e660a21
Replace broken "serial" restarts with a new method
2023-09-01 15:42:22 -04:00
Joshua Boniface
4a17a9918a
Correct bad address in pvcd.yaml
2023-09-01 15:42:22 -04:00
Joshua Boniface
744e3d0988
Set provisioner database in pvcd.yaml
2023-09-01 15:42:22 -04:00
Joshua Boniface
ba7270ab23
Add and remove floating IP during cluster bootstrap
2023-09-01 15:42:22 -04:00
Joshua Boniface
9546f34c34
Move netmask to separate config part 3
2023-09-01 15:42:22 -04:00
Joshua Boniface
211f83995b
Ensure the Patroni ZK is clean for bootstrap
2023-09-01 15:42:22 -04:00
Joshua Boniface
c27244f72d
Move netmask to separate config part 2
2023-09-01 15:42:22 -04:00
Joshua Boniface
a557b12e1c
Don't try to do crazy restart ordering, it fails
2023-09-01 15:42:22 -04:00
Joshua Boniface
45185c5a16
Change Patroni scope to just pvc
2023-09-01 15:42:22 -04:00
Joshua Boniface
21c9aa3e6a
Move netmask to separate config
2023-09-01 15:42:22 -04:00
Joshua Boniface
e97ba72a14
Set timezone to be a variable
2023-09-01 15:42:22 -04:00
Joshua Boniface
e76dc2b796
Use API endpoint to bootstrap PVC cluster
2023-09-01 15:42:22 -04:00
Joshua Boniface
f1e31ed169
Add additional API configuration
2023-09-01 15:42:22 -04:00
Joshua Boniface
da24aaf5ff
Install Provisioner schema to database
2023-09-01 15:42:22 -04:00
Joshua Boniface
484325698f
Add provisioner database schema
2023-09-01 15:42:22 -04:00
Joshua Boniface
cb3e4c2fa3
Remove empty newline
2023-09-01 15:42:22 -04:00
Joshua Boniface
f76802be6d
Remove invalid flag to ceph-authtool
2023-09-01 15:42:22 -04:00
Joshua Boniface
4b488a56ea
Don't become for uuidgen
2023-09-01 15:42:22 -04:00
Joshua Boniface
ff68f8a2a5
Move Ceph access to storage network
2023-09-01 15:42:22 -04:00
Joshua Boniface
6ce036f61e
Improve script to run ZK cleanup on all hosts
2023-09-01 15:42:22 -04:00
Joshua Boniface
9448cf3d90
Add jq dependency
2023-09-01 15:42:21 -04:00
Joshua Boniface
7689e659fe
Make vacuum script more comprehensive
2023-09-01 15:42:21 -04:00
Joshua Boniface
8163769183
Add check_mk check for PVC status
2023-09-01 15:42:21 -04:00
Joshua Boniface
2e0d5c9d44
Improve daily vacuum script
2023-09-01 15:42:21 -04:00
Joshua Boniface
94ef3490ab
Add daily Zookeeper data cleanup
2023-09-01 15:42:21 -04:00
Joshua Boniface
15a2bf1418
Add custom systemd unit for Zookeeper
...
We're 100% systemd here, and the lack of control/information that the
old-school ZK initscript provides is frustrating. Replace it with our
own simple unit file.
2023-09-01 15:42:21 -04:00
Joshua Boniface
7e74ce2ede
Add support for arbitrary /etc/hosts entries
2023-09-01 15:42:21 -04:00
Joshua Boniface
f98a2ee433
Add logrotate configuration
2023-09-01 15:42:21 -04:00
Joshua Boniface
c0acd3e994
Add daily Postgres vacuum script
2023-09-01 15:42:21 -04:00
Joshua Boniface
2cee9f08a0
Add Zookeeper autopurge @72h
2023-09-01 15:42:21 -04:00
Joshua Boniface
c83995ff1d
Update config template with recent changes
...
1) Add debug flag
2) Move intervals config up one level
2023-09-01 15:42:21 -04:00
Joshua Boniface
41c2dcd330
Set debug value in API config
2023-09-01 15:42:21 -04:00
Joshua Boniface
823310e8a3
Limit database tasks to coordinators only
...
Non-coordinators don't need these configurations as they shouldn't run
there.
2023-09-01 15:42:21 -04:00
Joshua Boniface
c85005f782
Always perform the apt-update
2023-09-01 15:42:21 -04:00
Joshua Boniface
1e562a58da
Support new log flags and update default log conf
...
Tweak the defaults a bit; pvc-ansible assumes we're running under
systemd, so set a log format that's better for it (no colour or date).
2023-09-01 15:42:21 -04:00
Joshua Boniface
675aa865de
Allow sysrc triggers in nodes
2023-09-01 15:42:21 -04:00
Joshua Boniface
035aef7c47
Replace tabs with spaces
2023-09-01 15:42:21 -04:00
Joshua Boniface
72d22a9c3c
Remove bad content from pvc-api.yml
2023-09-01 15:42:21 -04:00
Joshua Boniface
36657e28f9
Add quote around ZK nodes in Patroni
2023-09-01 15:42:21 -04:00
Joshua Boniface
db3198aadc
Bring up underlying interfaces
2023-09-01 15:42:21 -04:00
Joshua Boniface
5d3de3ece2
Complete configuration of API via Ansible
2023-09-01 15:42:21 -04:00
Joshua Boniface
32bcbbeca0
Enable client API in Daemon config
2023-09-01 15:42:21 -04:00
Joshua Boniface
d5516d891c
Add client API to configuration
2023-09-01 15:42:21 -04:00
Joshua Boniface
c5b07fcdad
Add only coordinators in Patroni config
2023-09-01 15:42:21 -04:00
Joshua Boniface
c9393ba957
Add templating of network interfaces
...
Closes #1
2023-09-01 15:42:21 -04:00
Joshua Boniface
cfbe724458
Install ethtool
2023-09-01 15:42:21 -04:00
Joshua Boniface
a42a349afc
Remove unattended upgrades
2023-09-01 15:42:20 -04:00
Joshua Boniface
c4382eb208
Make libvirt listen only on cluster IP
2023-09-01 15:42:20 -04:00
Joshua Boniface
1762d74aa8
Remove BLSE references
2023-09-01 15:42:20 -04:00
Joshua Boniface
f82bb6a414
Add debootstrap to package list
2023-09-01 15:42:20 -04:00
Joshua Boniface
d8e9b5353f
Don't try to set pool limits on libvirt key
...
I figured a * wildcard would work, but no it doesn't. Libvirt needs
the ability to talk to any pool arbitrarily since PVC can create and
remove them at will.
2023-09-01 15:42:20 -04:00
Joshua Boniface
0352dd7f8f
Create mgr after starting monitors
2023-09-01 15:42:20 -04:00
Joshua Boniface
e118267843
Set nicer default Patroni values re: replication
2023-09-01 15:42:20 -04:00
Joshua Boniface
dbf6e52f3c
Split PVC bootstrap into separate task
2023-09-01 15:42:20 -04:00
Joshua Boniface
935b4c48ae
Correct bug with libvirt permissions
2023-09-01 15:42:20 -04:00
Joshua Boniface
958d2525da
Handle restarting ceph-mon/mgr sequentially
2023-09-01 15:42:20 -04:00
Joshua Boniface
596ce789b1
Enable pool deletion in ceph.conf
2023-09-01 15:42:20 -04:00
Joshua Boniface
c4825e6de6
Support for detailed keepalive logging
2023-09-01 15:42:20 -04:00
Joshua Boniface
e9303c1ad1
Create manager auth keyring
2023-09-01 15:42:20 -04:00
Joshua Boniface
99bc566764
Make restart hander for pvcd sequential
2023-09-01 15:42:20 -04:00
Joshua Boniface
bcce7f5445
Remove per-host pvc.yml for good
2023-09-01 15:42:20 -04:00
Joshua Boniface
d277d18c35
Support configurable logging settings
2023-09-01 15:42:20 -04:00
Joshua Boniface
87f5fc527d
Enable storage support
2023-09-01 15:42:20 -04:00
Joshua Boniface
c19f653daf
Replace the square bracket with bracket
2023-09-01 15:42:20 -04:00
Joshua Boniface
aef72555c1
Consistent newhost format between roles
2023-09-01 15:42:20 -04:00
Joshua Boniface
f1cca0e22f
Add MTU configuration and update for new layout
2023-09-01 15:42:20 -04:00
Joshua Boniface
c46dae8852
Don't use (broken) ansible_lsb
2023-09-01 15:42:20 -04:00
Joshua Boniface
f5ea3aab24
Remove buster pre-release tweaks
...
Buster is now consistent in this regard
2023-09-01 15:42:20 -04:00
Joshua Boniface
9b457890d5
Use separate bootstrap files for base and pvc roles
2023-09-01 15:42:20 -04:00
Joshua Boniface
6dc57f374b
Revert "Keep zookeeper enabled"
...
This reverts commit 5554418210
.
This is not needed
2023-09-01 15:42:20 -04:00
Joshua Boniface
ebcd281490
Keep zookeeper enabled
...
Without this, the service seems to just loop failing to start
indefinitely even though PVC attempts to start the daemon itself.
Reenabling seems to work. Likely a bug due to Zookeeper not being
a proper systemd unit.
2023-09-01 15:42:20 -04:00
Joshua Boniface
90273f6599
Don't subset the re-setup
2023-09-01 15:42:20 -04:00
Joshua Boniface
a01720a09d
Ensure Ceph daemons are disabled (managed by PVC)
2023-09-01 15:42:20 -04:00
Joshua Boniface
a19d9c77ad
Clean up some tasks during bootstrap; parallel PVC
2023-09-01 15:42:20 -04:00
Joshua Boniface
218cec1126
Start Zookeeper during install
2023-09-01 15:42:20 -04:00
Joshua Boniface
e9fc24a8a8
Don't start pvc services on install
2023-09-01 15:42:20 -04:00
Joshua Boniface
f823d1b351
Touch the bootstrap ceph.conf
2023-09-01 15:42:20 -04:00
Joshua Boniface
aa72bb9bac
Move IP removal and restart after install
2023-09-01 15:42:20 -04:00
Joshua Boniface
09dc63a7d4
Gather all facts; flush handlers at end
2023-09-01 15:42:20 -04:00
Joshua Boniface
efd8dce53d
Simplify and combine
2023-09-01 15:42:20 -04:00
Joshua Boniface
e04e12f46f
Get NTP from real servers
2023-09-01 15:42:19 -04:00
Joshua Boniface
256a89d7cc
Reorganize some elements
2023-09-01 15:42:19 -04:00
Joshua Boniface
a922fe4df7
Reorganize and rejigger
2023-09-01 15:42:19 -04:00
Joshua Boniface
5cbce17bd1
Reorganize some bootstrap tasks
2023-09-01 15:42:19 -04:00
Joshua Boniface
0bf86fc5e8
Symlink ceph file dir back to group_vars
2023-09-01 15:42:19 -04:00
Joshua Boniface
fbd0ed9390
Improve handling around loop var and correct pvcd
2023-09-01 15:42:19 -04:00
Joshua Boniface
9bb513e695
Add hostsnames properly
2023-09-01 15:42:19 -04:00
Joshua Boniface
8fbbce3d16
Make things even nicer
2023-09-01 15:42:19 -04:00
Joshua Boniface
a22203c887
Define secret key properly
2023-09-01 15:42:19 -04:00
Joshua Boniface
e0fa826cfd
Flush handlers properly
2023-09-01 15:42:19 -04:00
Joshua Boniface
9d01d5d3cb
Improve libvirt key handling
2023-09-01 15:42:19 -04:00
Joshua Boniface
37506b981f
Flush handlers before starting PVC install
2023-09-01 15:42:19 -04:00
Joshua Boniface
ef0b1a8c5e
Add creation of libvirt client key
2023-09-01 15:42:19 -04:00
Joshua Boniface
feafe055f0
Add bootstrapping of Ceph
2023-09-01 15:42:19 -04:00
Joshua Boniface
54321b5fb4
Load jemalloc 2
2023-09-01 15:42:19 -04:00
Joshua Boniface
f126d3b071
Compatability with Ansible 2.2
2023-09-01 15:42:19 -04:00
Joshua Boniface
2650014ec1
Fix up some weird issues
2023-09-01 15:42:19 -04:00
Joshua Boniface
644097f290
Remove Ceph group creation
2023-09-01 15:42:19 -04:00
Joshua Boniface
6dfaf433dc
Initial commit of PVC Ansible role
2023-09-01 15:42:19 -04:00