Update README badge order

As PVC 0.9.101+ no longer support buster or bullseye, remove them as
installer options.

README.md

@@ -1,12 +1,38 @@
-# PVC Live Node Installer
+<p align="center">
+<img alt="Logo banner" src="https://docs.parallelvirtualcluster.org/en/latest/images/pvc_logo_black.png"/>
+<br/><br/>
+<a href="https://www.parallelvirtualcluster.org"><img alt="Website" src="https://img.shields.io/badge/visit-website-blue"/></a>
+<a href="https://github.com/parallelvirtualcluster/pvc/releases"><img alt="Latest Release" src="https://img.shields.io/github/release-pre/parallelvirtualcluster/pvc"/></a>
+<a href="https://docs.parallelvirtualcluster.org/en/latest/?badge=latest"><img alt="Documentation Status" src="https://readthedocs.org/projects/parallelvirtualcluster/badge/?version=latest"/></a>
+<a href="https://github.com/parallelvirtualcluster/pvc"><img alt="License" src="https://img.shields.io/github/license/parallelvirtualcluster/pvc"/></a>
+<a href="https://github.com/psf/black"><img alt="Code style: Black" src="https://img.shields.io/badge/code%20style-black-000000.svg"/></a>
+</p>
 
-**NOTICE FOR GITHUB**: This repository is a read-only mirror of the PVC repositories from my personal GitLab instance. Pull requests submitted here will not be merged. Issues submitted here will however be treated as authoritative.
+## What is PVC?
+
+PVC is a Linux KVM-based hyperconverged infrastructure (HCI) virtualization cluster solution that is fully Free Software, scalable, redundant, self-healing, self-managing, and designed for administrator simplicity. It is an alternative to other HCI solutions such as Ganeti, Harvester, Nutanix, and VMWare, as well as to other common virtualization stacks such as ProxMox and OpenStack.
+
+PVC is a complete HCI solution, built from well-known and well-trusted Free Software tools, to assist an administrator in creating and managing a cluster of servers to run virtual machines, as well as self-managing several important aspects including storage failover, node failure and recovery, virtual machine failure and recovery, and network plumbing. It is designed to act consistently, reliably, and unobtrusively, letting the administrator concentrate on more important things.
+
+PVC is highly scalable. From a minimum (production) node count of 3, up to 12 or more, and supporting many dozens of VMs, PVC scales along with your workload and requirements. Deploy a cluster once and grow it as your needs expand.
+
+As a consequence of its features, PVC makes administrating very high-uptime VMs extremely easy, featuring VM live migration, built-in always-enabled shared storage with transparent multi-node replication, and consistent network plumbing throughout the cluster. Nodes can also be seamlessly removed from or added to service, with zero VM downtime, to facilitate maintenance, upgrades, or other work.
+
+PVC also features an optional, fully customizable VM provisioning framework, designed to automate and simplify VM deployments using custom provisioning profiles, scripts, and CloudInit userdata API support.
+
+Installation of PVC is accomplished by two main components: a [Node installer ISO](https://github.com/parallelvirtualcluster/pvc-installer) which creates on-demand installer ISOs, and an [Ansible role framework](https://github.com/parallelvirtualcluster/pvc-ansible) to configure, bootstrap, and administrate the nodes. Installation can also be fully automated with a companion [cluster bootstrapping system](https://github.com/parallelvirtualcluster/pvc-bootstrap). Once up, the cluster is managed via an HTTP REST API, accessible via a Python Click CLI client ~~or WebUI~~ (eventually).
+
+Just give it physical servers, and it will run your VMs without you having to think about it, all in just an hour or two of setup time.
+
+More information about PVC, its motivations, the hardware requirements, and setting up and managing a cluster [can be found over at our docs page](https://docs.parallelvirtualcluster.org).
+
+# PVC Live Node Installer
 
 This repository contains the generator and configurations for the PVC Live Node Installer ISO. This ISO provides a quick and convenient way to install a PVC base system to a physical server, ready to then be provisioned using the [PVC Ansible](https://github.com/parallelvirtualcluster/pvc-ansible) configuration framework. Part of the [Parallel Virtual Cluster system](https://github.com/parallelvirtualcluster/pvc).
 
-## Using the PVC Installer
+# Using the PVC Installer
 
-### Preparing
+## Preparing
 
 1. Run `./buildiso.sh` from the root of the repository. This will pull down the Debian LiveCD image, extract it, debootstrap a fresh install environment, copy in the configurations, generate a squashfs, then finally generate an ISO file.
 
@@ -16,28 +42,12 @@ Note that artifacts of the build (the LiveCD ISO, debootstrap directory, and squ
 
 3. Boot the server from the ISO, ideally in UEFI mode.
 
-### Booting
+## Booting
 
 The built ISO can be booted in either BIOS (traditional ISOLinux) or UEFI (Grub2) modes. It is strongly recommended to use the latter if the system supports it for maximum flexibility.
 
-### Installing
+## Installing
 
 The installer script will ask several questions to configure the bare minimum system needed for [`pvc-ansible`](https://github.com/parallelvirtualcluster/pvc-ansible) to configure the node.
 
 Follow the prompts carefully; if you make a mistake, you can ^C to cancel the installer, then re-run via `/install.sh` from the resulting root shell.
-
-## License
-
-Copyright (C) 2018-2021  Joshua M. Boniface <joshua@boniface.me>
-
-This repository, and all contained files, is free software: you can
-redistribute it and/or modify it under the terms of the GNU General
-Public License as published by the Free Software Foundation, version 3.
-
-This program is distributed in the hope that it will be useful,
-but WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program.  If not, see <https://www.gnu.org/licenses/>.

buildiso.sh

@@ -114,12 +114,14 @@ echo
 echo "Initializing config..."
 # Initialize the live-build config
 lb config \
-       --distribution bullseye \
+       --distribution bookworm \
-       --archive-areas "main contrib non-free" \
+       --archive-areas "main contrib non-free-firmware" \
        --mirror-bootstrap "${mirror_server}" \
        --mirror-chroot-security "http://security.debian.org/debian-security" \
        --debconf-frontend readline \
        --apt-recommends false \
+       --debian-installer netinst \
+       --debian-installer-gui true \
        ${arch_config_append} || fail "Failed to initialize live-build config"
 echo
 
@@ -187,10 +189,12 @@ mkdir -p config/includes.chroot/etc/initramfs-tools
 cp ../../templates/modules config/includes.chroot/etc/initramfs-tools/modules || fail "Failed to copy critical template file"
 echo "done."
 
-# Install install.sh script
+# Install install.sh and detect.py scripts
 echo -n "Copying PVC node installer script template... "
 cp ../../templates/install.sh config/includes.chroot/install.sh || fail "Failed to copy critical template file"
 chmod +x config/includes.chroot/install.sh
+cp ../../templates/detect.py config/includes.chroot/detect.py || fail "Failed to copy critical template file"
+chmod +x config/includes.chroot/detect.py
 echo "done."
 
 # Customize install.sh script

detect.py

@@ -0,0 +1 @@
+templates/detect.py

templates/detect.py

@@ -0,0 +1,234 @@
+#!/usr/bin/env python3
+
+import subprocess
+import sys
+from json import loads as loads
+from re import match as re_match
+from re import search as re_search
+from re import sub as re_sub
+from shlex import split as shlex_split
+
+#
+# Run a local OS command via shell
+#
+def run_os_command(command_string, background=False, environment=None, timeout=None):
+    if not isinstance(command_string, list):
+        command = shlex_split(command_string)
+    else:
+        command = command_string
+
+    if background:
+
+        def runcmd():
+            try:
+                subprocess.run(
+                    command,
+                    env=environment,
+                    timeout=timeout,
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.PIPE,
+                )
+            except subprocess.TimeoutExpired:
+                pass
+
+        thread = Thread(target=runcmd, args=())
+        thread.start()
+        return 0, None, None
+    else:
+        try:
+            command_output = subprocess.run(
+                command,
+                env=environment,
+                timeout=timeout,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+            )
+            retcode = command_output.returncode
+        except subprocess.TimeoutExpired:
+            retcode = 128
+        except Exception:
+            retcode = 255
+
+        try:
+            stdout = command_output.stdout.decode("ascii")
+        except Exception:
+            stdout = ""
+        try:
+            stderr = command_output.stderr.decode("ascii")
+        except Exception:
+            stderr = ""
+        return retcode, stdout, stderr
+
+def get_detect_device_lsscsi(detect_string):
+    """
+    Parses a "detect:" string into a normalized block device path using lsscsi.
+
+    A detect string is formatted "detect:<NAME>:<SIZE>:<ID>", where
+    NAME is some unique identifier in lsscsi, SIZE is a human-readable
+    size value to within +/- 3% of the real size of the device, and
+    ID is the Nth (0-indexed) matching entry of that NAME and SIZE.
+    """
+    _, name, size, idd = detect_string.split(":")
+    if _ != "detect":
+        return None
+
+    retcode, stdout, stderr = run_os_command("lsscsi -s")
+    if retcode:
+        print(f"Failed to run lsscsi: {stderr}")
+        return None
+
+    # Get valid lines
+    lsscsi_lines_raw = stdout.split("\n")
+    lsscsi_lines = list()
+    for line in lsscsi_lines_raw:
+        if not line:
+            continue
+        split_line = line.split()
+        if split_line[1] != "disk":
+            continue
+        lsscsi_lines.append(line)
+
+    # Handle size determination (+/- 3%)
+    lsscsi_sizes = set()
+    for line in lsscsi_lines:
+        lsscsi_sizes.add(split_line[-1])
+    for l_size in lsscsi_sizes:
+        b_size = float(re_sub(r"\D.", "", size))
+        t_size = float(re_sub(r"\D.", "", l_size))
+
+        plusthreepct = t_size * 1.03
+        minusthreepct = t_size * 0.97
+
+        if b_size > minusthreepct and b_size < plusthreepct:
+            size = l_size
+            break
+
+    blockdev = None
+    matches = list()
+    for idx, line in enumerate(lsscsi_lines):
+        # Skip non-disk entries
+        if line.split()[1] != "disk":
+            continue
+        # Skip if name is not contained in the line (case-insensitive)
+        if name.lower() not in line.lower():
+            continue
+        # Skip if the size does not match
+        if size != line.split()[-1]:
+            continue
+        # Get our blockdev and append to the list
+        matches.append(line.split()[-2])
+
+    blockdev = None
+    # Find the blockdev at index {idd}
+    for idx, _blockdev in enumerate(matches):
+        if int(idx) == int(idd):
+            blockdev = _blockdev
+            break
+
+    return blockdev
+
+def get_detect_device_nvme(detect_string):
+    """
+    Parses a "detect:" string into a normalized block device path using nvme.
+
+    A detect string is formatted "detect:<NAME>:<SIZE>:<ID>", where
+    NAME is some unique identifier in lsscsi, SIZE is a human-readable
+    size value to within +/- 3% of the real size of the device, and
+    ID is the Nth (0-indexed) matching entry of that NAME and SIZE.
+    """
+
+    unit_map = {
+        'kB': 1000,
+        'MB': 1000*1000,
+        'GB': 1000*1000*1000,
+        'TB': 1000*1000*1000*1000,
+        'PB': 1000*1000*1000*1000*1000,
+    }
+
+    _, name, _size, idd = detect_string.split(":")
+    if _ != "detect":
+        return None
+
+    size_re = re_search(r'([\d.]+)([kKMGTP]B)', _size)
+    size_val = float(size_re.group(1))
+    size_unit = size_re.group(2)
+    size_bytes = int(size_val * unit_map[size_unit])
+
+    retcode, stdout, stderr = run_os_command("nvme list --output-format json")
+    if retcode:
+        print(f"Failed to run nvme: {stderr}")
+        return None
+
+    # Parse the output with json
+    nvme_data = loads(stdout).get('Devices', list())
+
+    # Handle size determination (+/- 3%)
+    size = None
+    nvme_sizes = set()
+    for entry in nvme_data:
+        nvme_sizes.add(entry['PhysicalSize'])
+    for l_size in nvme_sizes:
+        plusthreepct = size_bytes * 1.03
+        minusthreepct = size_bytes * 0.97
+
+        if l_size > minusthreepct and l_size < plusthreepct:
+            size = l_size
+            break
+    if size is None:
+        return None
+
+    blockdev = None
+    matches = list()
+    for entry in nvme_data:
+        # Skip if name is not contained in the line (case-insensitive)
+        if name.lower() not in entry['ModelNumber'].lower():
+            continue
+        # Skip if the size does not match
+        if size != entry['PhysicalSize']:
+            continue
+        # Get our blockdev and append to the list
+        matches.append(entry['DevicePath'])
+
+    blockdev = None
+    # Find the blockdev at index {idd}
+    for idx, _blockdev in enumerate(matches):
+        if int(idx) == int(idd):
+            blockdev = _blockdev
+            break
+
+    return blockdev
+
+
+def get_detect_device(detect_string):
+    """
+    Parses a "detect:" string into a normalized block device path.
+
+    First tries to parse using "lsscsi" (get_detect_device_lsscsi). If this returns an invalid
+    block device name, then try to parse using "nvme" (get_detect_device_nvme). This works around
+    issues with more recent devices (e.g. the Dell R6615 series) not properly reporting block
+    device paths for NVMe devices with "lsscsi".
+    """
+
+    device = get_detect_device_lsscsi(detect_string)
+    if device is None or not re_match(r'^/dev', device):
+        device = get_detect_device_nvme(detect_string)
+
+    if device is not None and re_match(r'^/dev', device):
+        return device
+    else:
+        return None
+
+
+try:
+    detect_string = sys.argv[1]
+except IndexError:
+    print("Please specify a detect: string")
+    exit(1)
+
+blockdev = get_detect_device(detect_string)
+if blockdev is not None:
+    print(blockdev)
+    exit(0)
+else:
+    exit(1)
+

templates/install.sh

@@ -72,7 +72,7 @@ target_deploy_user="XXDEPLOYUSERXX"
 supported_filesystems="ext4 xfs"
 default_filesystem="ext4"
 
-supported_debrelease="buster bullseye bookworm"
+supported_debrelease="bookworm"
 default_debrelease="bookworm"
 default_debmirror="http://ftp.debian.org/debian"
 
@@ -80,14 +80,20 @@ default_debmirror="http://ftp.debian.org/debian"
 basepkglist="lvm2,parted,gdisk,sudo,vim,gpg,gpg-agent,openssh-server,vlan,ifenslave,python3,ca-certificates,curl"
 case $( uname -m ) in
     x86_64)
-        basepkglist="${basepkglist},grub-pc,grub-efi-amd64,linux-image-amd64"
+        # If we're in EFI mode, install grub-efi, otherwise install grub-pc
+        if grep -q efivarfs /proc/mounts &>/dev/null; then
+            grub_pkg="grub-efi"
+        else
+            grub_pkg="grub-pc"
+        fi
+        basepkglist="${basepkglist},${grub_pkg},linux-image-amd64"
     ;;
     aarch64)
         basepkglist="${basepkglist},grub-efi-arm64,linux-image-arm64"
     ;;
 esac
 # Supplemental packages (installed in chroot after debootstrap)
-suppkglist="firmware-linux,firmware-linux-nonfree,firmware-bnx2,firmware-bnx2x,ntp,ipmitool,acpid,acpi-support-base,lsscsi"
+suppkglist="firmware-linux,firmware-linux-nonfree,firmware-bnx2,firmware-bnx2x,ntp,ipmitool,acpid,acpi-support-base,lsscsi,lsb-release"
 
 # Modules to blacklist (known-faulty)
 target_module_blacklist=( "hpwdt" )
@@ -227,56 +233,11 @@ seed_config() {
             target_disk="$( realpath ${target_disk} )"
         ;;
         detect:*)
-            # Read the detect string into separate variables
+            # Use the detect.py parser to get the target disk from the detect string
-            # A detect string is formated thusly:
+            target_disk="$( /detect.py ${target_disk} )"
-            #   detect:<Controller-or-Model-Name>:<Capacity-in-human-units><0-indexed-ID>
+        ;;
-            # For example:
+        *)
-            #   detect:INTEL:800GB:1
+            target_disk=""
-            #   detect:DELLBOSS:240GB:0
-            #   detect:PERC H330 Mini:200GB:0
-            echo "Attempting to find disk for detect string '${o_target_disk}'"
-            IFS=: read detect b_name b_size b_id <<<"${target_disk}"
-            # Get the lsscsi output (exclude NVMe)
-            lsscsi_data_all="$( lsscsi -s -N )"
-            # Get the available sizes, and match to within +/- 3%
-            lsscsi_sizes=( $( awk '{ print $NF }' <<<"${lsscsi_data_all}" | sort | uniq ) )
-            # For each size...
-            for size in ${lsscsi_sizes[@]}; do
-                # Get whether we match +3% and -3% sizes to handle human -> real deltas
-                # The break below is pretty safe. I can think of no two classes of disks
-                # where the difference is within 3% of each other. Even the common
-                # 120GB -> 128GB and 240GB -> 256GB size deltas are well outside of 3%,
-                # so this should be safe in all cases.
-                # We use Python for this due to BASH's problematic handling of floating-
-                # point numbers.
-                is_match="$(
-                    python <<EOF
-from re import sub
-try:
-    b_size = float(sub(r'\D.','','${b_size}'))
-    t_size = float(sub(r'\D.','','${size}'))
-except ValueError:
-    exit(0)
-plusthreepct = t_size * 1.03
-minusthreepct = t_size * 0.97
-if b_size > minusthreepct and b_size < plusthreepct:
-    print("match")
-EOF
-                )"
-                # If we do, this size is our actual block size, not what was specified
-                if [[ -n ${is_match} ]]; then
-                    b_size=${size}
-                    break
-                fi
-            done
-            # Search for the b_name first
-            lsscsi_data_name="$( grep --color=none -Fiw "${b_name}" <<<"${lsscsi_data_all}" )"
-            # Search for the b_blocks second
-            lsscsi_data_name_size="$( grep --color=none -Fiw "${b_size}" <<<"${lsscsi_data_name}" )"
-            # Read the /dev/X results into an array
-            lsscsi_filtered=( $( awk '{ print $(NF-1) }' <<<"${lsscsi_data_name_size}" ) )
-            # Get the b_id-th entry
-            target_disk="${lsscsi_filtered[${b_id}]}"
         ;;
     esac
 
@@ -566,8 +527,8 @@ interactive_config() {
         'static')
             if [[ -n ${vlan_id} ]]; then
                 modprobe 8021q >&2
-                vconfig add ${target_interface} ${vlan_id} >&2
                 vlan_interface=${target_interface}.${vlan_id}
+                ip link add link ${target_interface} name ${vlan_interface} type vlan id ${vlan_id} >&2
                 ip link set ${target_interface} up >&2 || true
                 ip link set ${vlan_interface} up >&2 || true
                 ip address add ${target_ipaddr} dev ${vlan_interface} >&2 || true
@@ -749,6 +710,7 @@ case ${install_option} in
 esac
 
 echo -n "Determining partition sizes... "
+ts=$(date +%s)
 blockdev_size_bytes="$( blockdev --getsize64 ${target_disk} )"
 blockdev_size_gbytes="$(( ${blockdev_size_bytes} / 1024 / 1024 / 1024 - 1))"
 if [[ ${blockdev_size_gbytes} -ge 100 ]]; then
@@ -757,114 +719,180 @@ if [[ ${blockdev_size_gbytes} -ge 100 ]]; then
     size_ceph_lv="8"
     size_zookeeper_lv="32"
     size_swap_lv="16"
-    echo "found large disk (${blockdev_size_gbytes}GB >= 100GB), using optimal partition sizes."
+    te=$(date +%s)
+    echo "found large disk (${blockdev_size_gbytes}GB >= 100GB), using optimal partition sizes. [$((te-ts))s]"
 else
     # Minimum sized disk (>=30GB), use small partitions
     size_root_lv="8"
     size_ceph_lv="4"
     size_zookeeper_lv="8"
     size_swap_lv="8"
-    echo "found small disk (${blockdev_size_gbytes}GB < 100GB), using small partition sizes."
+    te=$(date +%s)
+    echo "found small disk (${blockdev_size_gbytes}GB < 100GB), using small partition sizes. [$((te-ts))s]"
 fi
 
 echo -n "Unmounting potential partitions on target device... "
+ts=$(date +%s)
 for mount in $( mount | grep "${target_disk}" | awk '{ print $3 }' | sort -r ); do
     umount -f ${mount} >&2 || true
 done
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
 
 echo -n "Unmounting potential LVM logical volumes on target device... "
+ts=$(date +%s)
 for vg in $( pvscan | grep "${target_disk}" | awk '{ print $4 }' ); do
     for mount in $( mount | grep "/${vg}" | awk '{ print $3 }' | sort -r ); do
         umount -f ${mount} >&2 || true
     done
 done
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
 
 echo -n "Disabing potential LVM volume groups on target device... "
+ts=$(date +%s)
 for vg in $( pvscan | grep "${target_disk}" | awk '{ print $4 }' ); do
     vgchange -an ${vg} >&2 || true
     sleep 1
     vgchange -an ${vg} >&2 || true
     yes | vgremove -f ${vg} >&2 || true
 done
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
 
 echo -n "Removing existing LVM physical volumes... "
+ts=$(date +%s)
 for pv in $( pvscan | grep "${target_disk}" | awk '{ print $2 }' ); do
     yes | pvremove -f ${pv} >&2 || true
 done
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Wiping partition signatures on '${target_disk}'... "
+ts=$(date +%s)
 wipefs -a ${target_disk} >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Preparing GPT partitions on '${target_disk}'... "
+ts=$(date +%s)
 # New GPT, part 1 32MB BIOS boot, part 2 64MB ESP, part 3 928MB BOOT, part 4 inf LVM PV
 echo -e "o\ny\nn\n1\n\n32M\nEF02\nn\n2\n\n64M\nEF00\nn\n3\n\n928M\n8300\nn\n4\n\n\n8E00\nw\ny\n" | gdisk ${target_disk} >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Rescanning disks... "
+ts=$(date +%s)
 partprobe >&2 || true
 sleep 5
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
+if grep --silent '/dev/nvme' <<<"${target_disk}"; then
+    part_lvm="${target_disk}p4"
+    part_boot="${target_disk}p3"
+    part_esp="${target_disk}p2"
+else
+    part_lvm="${target_disk}4"
+    part_boot="${target_disk}3"
+    part_esp="${target_disk}2"
+fi
+
+echo -n "Creating LVM PV on '${part_lvm}'... "
+ts=$(date +%s)
+yes | pvcreate -ffy ${part_lvm} >&2
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
 
-echo -n "Creating LVM PV on '${target_disk}4'... "
-yes | pvcreate -ffy ${target_disk}4 >&2
-echo "done."
 
 echo -n "Creating LVM VG 'vgx'... "
-yes | vgcreate -f vgx ${target_disk}4 >&2
+ts=$(date +%s)
-echo "done."
+yes | vgcreate -f vgx ${part_lvm} >&2
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Creating root logical volume (${size_root_lv}GB)... "
+ts=$(date +%s)
 yes | lvcreate -L ${size_root_lv}G -n root vgx >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 echo -n "Creating filesystem on root logical volume (${filesystem})... "
+ts=$(date +%s)
 yes | mkfs.${filesystem} /dev/vgx/root >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Creating ceph logical volume (${size_ceph_lv}GB)... "
+ts=$(date +%s)
 yes | lvcreate -L ${size_ceph_lv}G -n ceph vgx >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 echo -n "Creating filesystem on ceph logical volume (${filesystem})... "
+ts=$(date +%s)
 yes | mkfs.${filesystem} /dev/vgx/ceph >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Creating zookeeper logical volume (${size_zookeeper_lv}GB)... "
+ts=$(date +%s)
 yes | lvcreate -L ${size_zookeeper_lv}G -n zookeeper vgx >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 echo -n "Creating filesystem on zookeeper logical volume (${filesystem})... "
+ts=$(date +%s)
 yes | mkfs.${filesystem} /dev/vgx/zookeeper >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Creating swap logical volume (${size_swap_lv}GB)... "
+ts=$(date +%s)
 yes | lvcreate -L ${size_swap_lv}G -n swap vgx >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 echo -n "Creating swap space on swap logical volume... "
+ts=$(date +%s)
 yes | mkswap -f /dev/vgx/swap >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Creating filesystem on boot partition (ext2)... "
-yes | mkfs.ext2 ${target_disk}3 >&2
+ts=$(date +%s)
-echo "done."
+yes | mkfs.ext2 ${part_boot} >&2
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Creating filesystem on ESP partition (vfat)... "
-yes | mkdosfs -F32 ${target_disk}2 >&2
+ts=$(date +%s)
-echo "done."
+yes | mkdosfs -F32 ${part_esp} >&2
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
 
+
+echo -n "Mounting disks on temporary target '${target}'... "
+ts=$(date +%s)
 vgchange -ay >&2
 target="/tmp/target"
 mkdir -p ${target} >&2
-echo -n "Mounting disks on temporary target '${target}'... "
 mount /dev/vgx/root ${target} >&2
 mkdir -p ${target}/boot >&2
 chattr +i ${target}/boot >&2
-mount ${target_disk}3 ${target}/boot >&2
+mount ${part_boot} ${target}/boot >&2
 mkdir -p ${target}/boot/efi >&2
 chattr +i ${target}/boot/efi >&2
-mount ${target_disk}2 ${target}/boot/efi >&2
+mount ${part_esp} ${target}/boot/efi >&2
 mkdir -p ${target}/var/lib/ceph >&2
 chattr +i ${target}/var/lib/ceph >&2
 mount /dev/vgx/ceph ${target}/var/lib/ceph >&2
@@ -874,12 +902,17 @@ mount /dev/vgx/zookeeper ${target}/var/lib/zookeeper >&2
 mkdir -p ${target}/tmp >&2
 chattr +i ${target}/tmp >&2
 mount -t tmpfs tmpfs ${target}/tmp >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Running debootstrap install... "
+ts=$(date +%s)
 echo "Command: debootstrap --include=${basepkglist} ${debrelease} ${target}/ ${debmirror}" >&2
 debootstrap --include=${basepkglist} ${debrelease} ${target}/ ${debmirror} >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 case ${debrelease} in
     buster)
@@ -899,15 +932,21 @@ case ${debrelease} in
 esac
 
 echo -n "Adding ${non_free} APT component (firmware, etc.)... "
+ts=$(date +%s)
 mkdir -p ${target}/etc/apt/sources.list.d/ >&2
 echo "deb ${debmirror} ${debrelease} contrib ${non_free}" | tee -a ${target}/etc/apt/sources.list >&2
 chroot ${target} apt-get update >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 
 echo -n "Installing supplemental packages... "
+ts=$(date +%s)
 chroot ${target} apt-get install -y --no-install-recommends $( sed 's/,/ /g' <<<"${suppkglist}" ) >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 # Determine the bypath name of the specified system disk
 for disk in /dev/disk/by-path/*; do
@@ -918,14 +957,17 @@ for disk in /dev/disk/by-path/*; do
     fi
 done
 
-# Check if TRIM is supported on the root disk
+# Check if TRIM is supported on the root disk (NVMe always is)
-if hdparm -I ${target_disk} | grep --silent "TRIM supported"; then
+if grep --silent '/dev/nvme' <<<"${target_disk}"; then
+    extdiscard="discard,"
+elif hdparm -I ${target_disk} | grep --silent "TRIM supported"; then
     extdiscard="discard,"
 else
     extdiscard=""
 fi
 
 echo -n "Adding fstab entries... "
+ts=$(date +%s)
 echo "# fstab configuration for PVC hypervisor" | tee ${target}/etc/fstab >&2
 echo "/dev/mapper/vgx-root / ${filesystem} defaults,${extdiscard}errors=remount-ro 0 1" | tee -a ${target}/etc/fstab >&2
 echo "/dev/mapper/vgx-ceph /var/lib/ceph ${filesystem} defaults,${extdiscard}errors=remount-ro 0 2" | tee -a ${target}/etc/fstab >&2
@@ -934,7 +976,9 @@ echo "/dev/mapper/vgx-swap none swap sw 0 0" | tee -a ${target}/etc/fstab >&2
 echo "${bypath_disk}-part3 /boot ext2 defaults 0 2" | tee -a ${target}/etc/fstab >&2
 echo "${bypath_disk}-part2 /boot/efi vfat umask=0077 0 2" | tee -a ${target}/etc/fstab >&2
 echo "tmpfs /tmp tmpfs defaults 0 0" | tee -a ${target}/etc/fstab >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 seed_interfaces_segment() {
     # A seed install is always "dhcp-raw" since the provisioner is always a dedicated, access port
@@ -968,6 +1012,7 @@ interactive_interfaces_segment() {
 }
 
 echo -n "Creating bootstrap interface segment... "
+ts=$(date +%s)
 case ${install_option} in
     on)
         seed_interfaces_segment
@@ -976,24 +1021,36 @@ case ${install_option} in
         interactive_interfaces_segment
     ;;
 esac
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Adding bootstrap interface segment... "
+ts=$(date +%s)
 echo -e "${target_interfaces_block}" | tee ${target}/etc/network/interfaces.d/${target_interface} >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 case ${install_option} in
     on)
         echo -n "Creating bond interface segment... "
+        ts=$(date +%s)
         bond_interfaces="$( ip -br link show | grep -E -o '^e[a-z]+[0-9]+[a-z0-9]*' | grep -v "^${target_interface}$" | tr '\n' ' ' )"
         bond_interfaces_block="auto bond0\niface bond0 inet manual\n\tbond-mode 802.3ad\n\tbond-slaves ${bond_interfaces}\n\tpost-up ip link set mtu 9000 dev \$IFACE"
-        echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
         echo -n "Adding bond interface segment... "
+        ts=$(date +%s)
         echo -e "${bond_interfaces_block}" | tee ${target}/etc/network/interfaces.d/bond0 >&2
-        echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
         echo -n "Adding bootstrap interface post-up checkin script... "
+        ts=$(date +%s)
         cat <<EOF | tee ${target}/etc/network/pvcprovisionerd.checkin.sh >&2
 #!/usr/bin/env bash
 target_interface=\${1}
@@ -1025,7 +1082,9 @@ if [[ \${action} == "system-boot_configured" ]]; then
 fi
 EOF
         chmod +x ${target}/etc/network/pvcprovisionerd.checkin.sh
-        echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
     ;;
     *)
         # noop
@@ -1034,10 +1093,14 @@ EOF
 esac
 
 echo -n "Setting temporary 'root' password... "
+ts=$(date +%s)
 echo "root:${root_password}" | chroot ${target} chpasswd >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Adding deployment user... "
+ts=$(date +%s)
 mv ${target}/home ${target}/var/home >&2
 chroot ${target} useradd -u 200 -d /var/home/${target_deploy_user} -m -s /bin/bash -g operator -G sudo ${target_deploy_user} >&2
 chroot ${target} mkdir -p /var/home/${target_deploy_user}/.ssh
@@ -1055,7 +1118,9 @@ if [[ -n ${target_keys_path} ]]; then
 else
     echo "${target_deploy_user}:${target_password}" | chroot ${target} chpasswd >&2
 fi
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 if [[ -n ${failed_keys} ]]; then
     target_password="$( pwgen -s 8 1 )"
     echo "WARNING: Failed to fetch keys; target deploy user SSH keyauth will fail."
@@ -1064,10 +1129,14 @@ if [[ -n ${failed_keys} ]]; then
 fi
 
 echo -n "Setting NOPASSWD for sudo group... "
+ts=$(date +%s)
 sed -i 's/^%sudo\tALL=(ALL:ALL) ALL/%sudo\tALL=(ALL:ALL) NOPASSWD: ALL/' ${target}/etc/sudoers
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Setting /etc/issue generator... "
+ts=$(date +%s)
 mkdir -p ${target}/etc/network/if-up.d >&2
 echo -e "#!/bin/sh
 IP=\"\$( ip -4 addr show dev ${target_interface} | grep inet | awk '{ print \$2 }' | head -1 )\"
@@ -1078,23 +1147,35 @@ Bootstrap interface IP address: \$IP
 
 EOF" | tee ${target}/etc/network/if-up.d/issue-gen >&2
 chmod +x ${target}/etc/network/if-up.d/issue-gen 1>&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Generating host rsa and ed25519 keys... "
+ts=$(date +%s)
 rm ${target}/etc/ssh/ssh_host_*_key* >&2
 chroot ${target} ssh-keygen -t rsa -N "" -f /etc/ssh/ssh_host_rsa_key >&2
 chroot ${target} ssh-keygen -t ed25519 -N "" -f /etc/ssh/ssh_host_ed25519_key >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Setting hostname... "
+ts=$(date +%s)
 echo "${target_hostname}" | tee ${target}/etc/hostname >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Setting resolv.conf... "
+ts=$(date +%s)
 echo "nameserver 8.8.8.8" | tee ${target}/etc/resolv.conf >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Installing GRUB bootloader... "
+ts=$(date +%s)
 mount --bind /dev ${target}/dev >&2
 mount --bind /dev/pts ${target}/dev/pts >&2
 mount --bind /proc ${target}/proc >&2
@@ -1118,14 +1199,19 @@ mount --bind /sys/firmware/efi/efivars ${target}/sys/firmware/efi/efivars
 chroot ${target} grub-install --force --target=${bios_target} ${target_disk} >&2
 chroot ${target} grub-mkconfig -o /boot/grub/grub.cfg >&2
 umount ${target}/sys/firmware/efi/efivars
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 echo -n "Adding module blacklists... "
+ts=$(date +%s)
 for module in ${target_module_blacklist[@]}; do
     echo "blacklist ${module}" >> ${target}/etc/modprobe.d/blacklist.conf
 done
 chroot ${target} update-initramfs -u -k all >&2
-echo "done."
+te=$(date +%s)
+echo "done. [$((te-ts))s]"
+
 
 DONE="y"
 

templates/installer.list.chroot

@@ -1 +1 @@
-live-task-standard live-tools live-boot live-boot-initramfs-tools psmisc mdadm lvm2 parted gdisk dosfstools debootstrap sipcalc vim ca-certificates vlan tftp-hpa curl ipmitool lsscsi hdparm
+live-task-standard live-tools live-boot live-boot-initramfs-tools psmisc mdadm lvm2 parted gdisk dosfstools debootstrap sipcalc vim ca-certificates vlan tftp-hpa curl ipmitool nvme-cli jq lsscsi hdparm