---
description: >-
  Automating the continuous deployment of a virtual pentest machine using
  Proxmox, Packer, Terraform, Ansible and GitLab.
---

# Proxmox+Packer+Terraform+Ansible

## What on earth?

Setting up a new pentest VM for every project is tedious, time-consuming and error-prone. Thus, I've set out to build from scratch an automation that will:

* Provision a release of Kali as VM template (Packer - IaC)
* Provision a staging and production version (Terraform - IaC)
* Modify the VM via simple changes in GitLab (GitLab + Ansible - CI/CD Pipeline)
* Do all that on a Proxmox server

To demonstrate the minimum setup and combination of Packer, Terraform, Gitlab and Ansible, I've documented the initial setup. Please keep in mind, that some best-practices, such as using a Vault and secure credentials have been skipped for brevity. Should you choose to follow this guide, make sure to read official documentation for best practice recommendations!

{% hint style="info" %}
In this guide, we won't look at how to install Proxmox or GitLab.
{% endhint %}

## Overview

{% hint style="info" %}
Use this chapter as a reference to keep in mind what components interact with eachother.
{% endhint %}

<figure><img src="https://1971224599-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mhlz_oZ3oVPSWFmU_3o%2Fuploads%2FKMcMyKkRdZPhxhuw1lgC%2Fsetup.png?alt=media&#x26;token=ec9f57b4-6173-476d-bed6-6f3676bec34b" alt=""><figcaption></figcaption></figure>

## Set Up a Base VM

{% hint style="info" %}
The base VM we will call `gitlab-runner` as it will perform all actions based on the instructions of a Pipeline that we are going to setup later. It is the one system that we will have to maintain manually. We could automate that too but that's out of scope.
{% endhint %}

1.  In Proxmox, create a new VM

    I will be using a [Debian](https://www.debian.org/CD/netinst/#netinst-stable) here but you are free to use whatever you want for your base VM. I will name it `gitlab-runner` and give it an ID of  `420`.
2.  Once the VM is ready, we will need to install a few things

    ```bash
    # Install Packer and Terraform: https://developer.hashicorp.com/packer/install
    wget -O - https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
    echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
    sudo apt update && sudo apt install packer terraform

    # We will also need git for the integration as GitLab Runner
    sudo apt install git curl
    curl -L "https://packages.gitlab.com/install/repositories/runner/gitlab-runner/script.deb.sh" | sudo bash
    sudo apt install gitlab-runner

    # And Ansible
    sudo apt install pipx sshpass
    pipx ensurepath
    source ~/.bashrc
    pipx install --include-deps ansible

    # In case your environment requires it:
    # Install the root certificates for Proxmox and GitLab
    # You can achieve this by exporting the root CA via a browser (URL bar, lock icon)
    sudo mv custom-ca.crt /usr/local/share/ca-certificates/custom-ca.crt
    sudo update-ca-certificates
    ```

{% hint style="success" %}
That's it for the installation part. We will continue to make some changes and add things but I recommend you **create a snapshot** of that VM now, so you can rollback in case anything goes wrong.
{% endhint %}

## Provision Kali with Packer

Packer is a tool for creating virtual images. Here, we will use it to automate downloading a Kali release from the official website, installing it in a VM and then creating a Proxmox template from it.

{% hint style="warning" %}
At the moment of writing, neither Packer nor Proxmox seem incredibly stable. Here, I am working with Proxmox 8.4.1, Packer 1.12.0 and the Packer Proxmox plugin 1.2.2. While many things work just perfectly, keep in mind that small or minor changes in either of these tools can break everything and result in headache-inducing rabbit holes of troubleshooting. On that note, I reported [a bug in the Packer Proxmox plugin](https://github.com/hashicorp/packer-plugin-proxmox/issues/321) that you may encounter too.
{% endhint %}

1. Before we dive into Packer, we want to create an API user in Proxmox that Packer (and later also Terraform) can use to orchestrate any changes, like creating and deleting VMs
   1. Navigate to Datacenter > Permissions > Roles
   2. Create a new role and call it `APIProvision` for example (the name cannot start with `PVE`)
   3.  We want to assign permissions to this role, that the API only really requires:

       <pre class="language-bash"><code class="lang-bash"># Feel free to experiment
       # In case of missing permissions Proxmox will throw API errors
       # They may just not always hint at a permission problem, just keep that in mind
       Datastore.Allocate
       <strong>Datastore.AllocateTemplate
       </strong>Datastore.Audit
       Datastore.AllocateSpace
       Pool.Allocate
       SDN.Use
       Sys.Audit
       Sys.Console
       Sys.Modify
       Sys.PowerMgmt
       VM.Allocate
       VM.Audit
       VM.Clone
       VM.Config.CDROM
       VM.Config.CPU
       VM.Config.Cloudinit
       VM.Config.Disk
       VM.Config.HWType
       VM.Config.Memory
       VM.Config.Network
       VM.Config.Options
       VM.Console
       VM.Migrate
       VM.Monitor
       VM.PowerMgmt
       VM.Snapshot
       VM.Snapshot.Rollback
       </code></pre>
   4. Next, navigate to Datacenter > Permissions > API Tokens and select `Add`
   5. Select a user (optimally it is a single-purpose API user but a normal account works too) and make sure that `Privilege Separation` is checked. This is especially important when using a normal account (or even `root`). When the checkbox is ticked, the API token will not inherit permissions from the user
   6. Set any token ID and click `Add`
   7. You will now see the complete token ID (i.e. `<username>!<tokenname>`) and the API secret - store both of them securely in your KeePass or elsewhere
   8. Lastly, go to Datacenter > Permissions and select `Add`
   9.  Here, we merge the API token, the role, and a resource pool

       Select the created role, the created API token and the resource pool where you want to provision your systems - I will use a resource pool called `Infrastructure` that I had created under Permissions > Pools

{% hint style="info" %}
If you loose your API key, you will have to create a new API token. Delete the previous one and reuse the role.
{% endhint %}

2.  Now we create our Kali Linix packer script `kali.pkr.hcl`

    The first item in the file defines the plugin we require to talk to the Proxmox API ([documentation](https://developer.hashicorp.com/packer/integrations/hashicorp/proxmox/latest/components/builder/iso)).

    <pre class="language-hcl"><code class="lang-hcl"><strong>packer {
    </strong>    required_plugins {
            proxmox = {
                version = ">= 1.2.2"
                source = "github.com/hashicorp/proxmox"
            }
        }
    }
    </code></pre>

    Now we want to define some variables, so that we keep valuable secrets out of the code:

    ```hcl
    # Variables
    # Supplied by Gitlab pipeline later
    variable "proxmox_api_user" {
    	type = string
    	sensitive = false
    }

    variable "proxmox_api_token" {
    	type = string
    	sensitive = true
    }

    variable "kali_iso_url" {
    	type = string
    	sensitive = false
    }

    variable "kali_iso_sha256" {
    	type = string
    	sensitive = false
    }
    ```

    Then follows the description of exactly what we want to build with that plugin.\
    Consult the linked documentation for an explanation of each field.

    ```hcl
    # source <plugin> <name>
    source "proxmox-iso" "kali-template" {
        
        # Proxmox connection settings
        proxmox_url = "https://<proxmox>/api2/json"
        username = "${var.proxmox_api_user}" # "<username!tokenname>"
        token = "${var.proxmox_api_token}" # "<apitoken>"
        
        # Image metadata
        node = "pve"
        pool = "Infrastructure" # Packer-promox does not support nested pools in 1.2.2
        vm_id = "421"
        vm_name = "kali-template"
        tags = "infrastructure"
        template_description = "Template created with Packer"
        
        # ISO source
        boot_iso {
          # Instead of a fix version you could also target https://cdimage.kali.org/current/...
          iso_url = "${var.kali_iso_url}" # "https://cdimage.kali.org/kali-2025.1c/kali-linux-2025.1c-installer-netinst-amd64.iso"
          iso_checksum = "sha256:${var.kali_iso_sha256}" # "sha256:<checksum-from-the-website>"
          iso_storage_pool = "local-iso"
          iso_download_pve = true
          unmount = true
       }
       
       # VM settings
       ## Disk
       disks {
         type = "scsi"
         disk_size = "60G"
         storage_pool = "local-lvm"
       }
       ## CPU
       cores = "2"
       sockets = "2"
       ## Memory
       memory = 8192
       ## Network
       network_adapters {
         model = "virtio"
         bridge = "vmbr0" # must be reachable from gitlab-runner
         vlan_tag = "200" # optional
         firewall = true
       }
       
       # Enable Cloud Init
       cloud_init = true
       cloud_init_storage_pool = "local-lvm"
       qemu_agent = true
       
       # SSH
       ssh_timeout = "2h" # maximum time we allow for the boot+installation process
       ssh_username = "kali"
       ssh_password = "kali" # fine for testing, will be replaced with variables later
       
       # Preseed Kali via HTTP (explained after the code section)
       # See https://gitlab.com/kalilinux/recipes/kali-preseed-examples
       http_directory = "boot-cfg"
       
       boot_command = [
         # Switch to boot menu
         "<esc><wait>",
         # Utilize the preseed file
         "/install.amd/vmlinuz vga=788 auto=true priority=critical url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/kali-preseed.cfg initrd=/install.amd/initrd.gz --- quiet",
         "<enter>"
       ]
    }

    # Finally, the build section
    build {
        name = "proxmox" # you can choose one
        sources = ["sources.proxmox-iso.kali-template"] # put together from the source
        
        # Post installation steps
        provisioner "shell" {
            inline = [
                # Clean up the VM before turning it into a template
                "sudo truncate -s 0 /etc/machine-id",
                # Create swapfile
                "sudo dd if=/dev/zero of=/swapfile bs=1G count=4",
                "sudo chmod 600 /swapfile","sudo mkswap /swapfile",
                "echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab",
                # Setup and configure cloud-init
                "echo 'datasource_list: [ NoCloud, ConfigDrive ]' | sudo tee /etc/cloud/cloud.cfg.d/99_pve.cfg",
                # Revert changes to the sudoers file that made sudo in this SSH session possible
                "sudo sed -i '/(^Defaults.*!requiretty$|^kali.*ALL$)/d' /etc/sudoers",
            ]
        }

    }
    ```

{% hint style="info" %}
Theory for the upcoming section: **Preseeding** is one way (of many) for automating OS installations. It consists of a file that contains all the answers to the questions we would otherwise see in a live install. Debian has some good documentation [here](https://wiki.debian.org/DebianInstaller/Preseed).

Another method is `cloud-init` which provides a way of configuring basic system configuration such as credentials and IP configuration via a standard interface. Proxmox supports `cloud-init` and allows you to set these configurations easily via the GUI.

The default Kali ISO is perfect for customization with preseeding but does not come with `cloud-init` installed. For cloning and easy configuration later on, we do want `cloud-init` though. Thus, we install and activate it manually (alternatively, you may also experiment with the Kali Generic Cloud image).
{% endhint %}

3. Now, as mentioned previously, we create a folder called `boot-cfg` in the current directory
4.  In that directory, we create the `kali-preseed.cfg` file, this one is based on the [Kali example](https://gitlab.com/kalilinux/recipes/kali-preseed-examples)

    ```bash
    # Locale and keyboard
    d-i debian-installer/locale string en_US
    d-i debian-installer/language string en
    d-i debian-installer/country string US
    d-i debian-installer/locale string en_US.UTF-8
    d-i localechooser/shortlist select US
    d-i localechooser/preferred-locale select en_US.UTF-8
    d-i localechooser/languagelist select en
    d-i keyboard-configuration/xkb-keymap select us
    # Disable popularity-contest
    popularity-contest popularity-contest/participate boolean false
    # Network
    d-i netcfg/choose_interface select auto
    d-i netcfg/get_hostname string kali-template
    d-i netcfg/get_domain string unassigned-domain
    d-i netcfg/wireless_wep string
    # Mirrors
    d-i mirror/country string manual
    d-i mirror/http/hostname string http.kali.org
    d-i mirror/http/directory string /kali
    d-i mirror/http/proxy string
    # User
    d-i passwd/user-fullname string kali
    d-i passwd/username string kali
    d-i passwd/user-password password kali
    d-i passwd/user-password-again password kali
    # Date
    d-i clock-setup/utc boolean true
    d-i time/zone string US/Eastern
    d-i clock-setup/ntp boolean true
    # Partitioning
    # (create one big partition, disable swap)
    d-i partman-auto/method string regular
    d-i partman-lvm/device_remove_lvm boolean true
    d-i partman-md/device_remove_md boolean true
    d-i partman-lvm/confirm boolean true
    d-i partman-auto/expert_recipe string \
        single-part ::                          \
            5000 10000 -1 $default_filesystem   \
                $primary{ }                     \
                $bootable{ }                    \
                method{ format }                \
                format{ }                       \
                use_filesystem{ }               \
                $default_filesystem{ }          \
                mountpoint{ / } .
    d-i partman-auto/disk string /dev/sda
    d-i partman/confirm_write_new_label boolean true
    d-i partman/choose_partition select finish
    d-i partman/confirm boolean true
    d-i partman/confirm_nooverwrite boolean true
    d-i partman-partitioning/confirm_write_new_label boolean true
    d-i partman-basicfilesystems/no_swap boolean false
    #Packages
    tasksel tasksel/first multiselect standard,core,desktop-xfce,meta-default
    d-i pkgsel/include string coreutils qemu-guest-agent cloud-init
    # Grub
    d-i grub-installer/only_debian boolean true
    d-i grub-installer/with_other_os boolean false
    d-i grub-installer/bootdev string /dev/sda
    d-i finish-install/reboot_in_progress note
    # Post Install
    d-i preseed/late_command string \
    echo "kali    ALL=(ALL)    NOPASSWD: ALL" >> /target/etc/sudoers; \
    sed -i "s/env_reset/env_reset\nDefaults\t!requiretty/" /target/etc/sudoers; \
    in-target systemctl enable ssh
    ```
5.  With this config we can change the hostname, username, password, install packages such as `cloud-init`, disable the swap partition (Don't get confused by the `no_swap false` option, it actually allows us to use no swap partition, see [here](https://github.com/turnkeylinux/di-live/blob/ad6b8f1255a1b1a809b1956cdd269dff5c1eb374/d-i/source/partman-basicfilesystems/debian/partman-basicfilesystems.templates#L32).), etc.

    The `d-i preseed/late_command` is documented [here](https://www.debian.org/releases/stable/i386/apbs05.en.html) and allows us to enable SSH and prepare `sudo` so that Packer can login via SSH once the setup is finished to perform final clean up tasks. &#x20;
6.  With everything prepared, we must run the following command once

    ```bash
    packer init kali.pkr.hcl
    ```

    This downloads the Proxmox plugin we specified in the beginning.
7.  From here on, we can use

    ```bash
    packer validate kali.pkr.hcl # to check for syntax errors, and
    packer build kali.pkr.hcl # to provision the kali template
    # The build command will download (and cache) the specified ISO
    # provision a VM, install the OS, run the build steps, shut the VM down
    # and create a template from it.
    # If we want another release, we just update the URL.

    # If you hardcoded credentials, the above command will work. If you used variables
    # then we need to pass them on the commandline like this:
    packer build -var 'proxmox_api_user=<api@pve!api-user>' -var 'proxmox_api_token=<api-token>' -var 'kali_iso_url=<cdimage.kali.org/url>' -var 'kali_iso_sha256=<sha256sum>' kali.pkr.hcl
    ```

{% hint style="success" %}
That sums it up for Packer and Proxmox. Feel free to adapt any step to your needs - in the end you should see a success message like the following one and a new template in your Proxmox host.
{% endhint %}

<figure><img src="https://1971224599-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mhlz_oZ3oVPSWFmU_3o%2Fuploads%2FGp12t8mm4Cyuy4t1JBwF%2Fimage.png?alt=media&#x26;token=15198711-dd77-46b6-8620-3ba014cc9761" alt=""><figcaption><p>Successful creation of a base template with Packer on Proxmox</p></figcaption></figure>

Finally, you can verify that everything is working by cloning the template in the Proxmox Web interface. Before you start the new machine, switch to the `Cloud-Init` tab and set a username and password. Subsequently, press on `Regenerate Image`. Now you are ready to start the VM and login with the credentials you just have set. Note that `cloud-init` does alot of things for us as boot time, including changing the hostname to the name of the new virtual machine.

<figure><img src="https://1971224599-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mhlz_oZ3oVPSWFmU_3o%2Fuploads%2FrqKKxszUwVgICaHJriFJ%2Fimage.png?alt=media&#x26;token=e04d2c71-a685-4876-bf26-d9f7d0ea8ab4" alt=""><figcaption><p>For testing purposes, clone the template manually and configure a user via cloud-init, then start the VM</p></figcaption></figure>

You could configure the behaviour of `cloud-init` by modifying the file `/etc/cloud/cloud.dfg` but that is out of scope for this series. By default, it will add the specified user as the only user and grant it root permissions.

## Deploy Staging and Production VM with Terraform

Having a template available, we now want to clone it to create a staging and production machine for our CI/CD pipeline. The staging VM will be used for feature and integration tests while the production VM is going to be the one we can use as base image to deploy during engagements.

For this we are going to use Terraform. Terraform is tool that we can use to declare the systems that we want to have in our infrastructure. Based on that, it will use the Proxmox API to build the environment.

1. We start by creating a `terraform` directory on the `gitlab-runner` machine.
2.  Switch to the directory and create a `kali-provision.tf`

    The first part may look familiar. It works almost similar to Packer.

    <pre class="language-hcl"><code class="lang-hcl"><strong>terraform {
    </strong>    required_version = ">=1.11.4"
        
        required_providers {
            proxmox = {
                source = "telmate/proxmox"
                version = "3.0.1-rc8"
            }
        }
    }

    provider "proxmox" {
        pm_api_url = "https://&#x3C;proxmox>/api2/json"
        pm_api_token_id = "&#x3C;username!tokenname>" # paste the Proxmox API ID we created
        pm_api_token_secret = "&#x3C;apitoken>"       # paste the token
        # We use this variant for simplicty here. The better option is to use variables
        # in the exact same way as we did with packer above.
    }
    </code></pre>

    The next part describes the infrastructure we want to build, using the `cloud-init` we configured for the template. Refer to the plugins documentation [here](https://registry.terraform.io/providers/Telmate/proxmox/latest/docs/resources/vm_qemu) for details.

    ```hcl
    # These are going to be the VM names we want to setup, add any you like
    variable "targets" {
        type = list(string)
        default = [
            "staging",
            "prod"
        ]
    }

    resource "proxmox_vm_qemu" "kali-deployment" {
        count = length(var.targets)

        # General
        target_node = "pve"
        pool = "Infrastructure" # remember the nested pool issue from Packer? Same thing here
        tags = "infrastructure"
        agent = "1" # since we have qemu_guest_agent installed
        full_clone = true
        os_type = "cloud-init"
        
        # Cloud init
        ci_wait = "20" 
        ciuser = "kali"
        cipassword = "kali"
        
        # VM
        clone_id = "421" # ID of the template we created with Packer
        name = "kali-${var.targets[count.index]}" # generate all VM names
        vmid = "${422+count.index}" # leave at 0 if you want automatically assigned IDs
        desc = "Created from template ${timestamp()}"
        boot = "order=scsi0"
        ## Now, we already specified all this in the template
        ## but apparently, while in the GUI it was enough to click clone
        ## here we have to specify all settings again or they won't be added
        cpu {
            sockets = "2"
            cores = "2"
        }
        memory = "8192"
        ## Set a static IP address
        ipconfig0 = "gw=192.168.0.1,ip=192.168.0.${22+count.index}/24"
        
        network { # copy from template settings unless you want to change it
            id = 0
            bridge = "vmbr1"
            model = "virtio"
            tag = "200"
            firewall = true
        }
        
        disks { # copy from Hardware settings of the template
            scsi {
                scsi0 {
                    disk {
                        storage = "local-lvm"
                        size = "60G"
                    }
                }
            }
            ide {
                ide0 {
                    cloudinit {
                        storage = "local-lvm"
                    }
                }
            }
        }
        
        # This last part is optional (in case we need some post setup action)
        connection {
            type = "ssh"
            user = "kali"
            password = "kali"
            host = "192.168.0.${22+count.index}"
        }

        provisioner "remote-exec" {
            inline = [
                "sudo ip a"
            ]
        }
    }
    ```
3. Save the file and then execute `terraform init` once - similar to what we did for Packer, this will prepare the environment and download the specified plugin
4. Run `terraform validate` to check for any errors
5. Then run `terraform plan` to check what Terraform plans to do next
6. If all looks good (in our example, 2 resources should be created, none modified, none deleted), then we can run `terraform apply` (if you want to revert the VMs, use `terraform destroy`)
7. Confirm you actions with `Yes` and wait a few minutes while Terraform instantiates the VMs

<figure><img src="https://1971224599-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mhlz_oZ3oVPSWFmU_3o%2Fuploads%2FZSuYk1j2XozJUNnEMMfi%2Fimage.png?alt=media&#x26;token=e9591c56-f684-41e2-89ed-e2a373827650" alt=""><figcaption><p>After a short while we should see two new VMs</p></figcaption></figure>

{% hint style="success" %}
That's already it for the Terraform part. Go ahead, login and test whatever requirements you have for your staging and production VM. Here, I am fine with the base installation and cloud-init being active. Next, I want to make sure that I can run Ansible playbooks against the VMs.
{% endhint %}

## Configure the VMs via Ansible

{% hint style="info" %}
Now would be a great time to take a snapshot of the `gitlab-runner` VM.
{% endhint %}

So far we've automated the provision of the infrastructure. Next, we want to automate the customization of our Kali. This may include custom programs, files, UI changes, you name it.

For this, we are going to use Ansible as it lets us define the exact state that we want our Kali to have without having to worry about scripting _all_ of it with Bash and Python.

1. Start by creating an `ansible` directory and navigate to it
2. Create two directories: `group_vars` and `inventory`
3.  We want to create the following structure:

    ```
    + ~/ansible/
        +- ansible.cfg
        +- kali.yml
        +- group_vars/
        |    +-all.yml
        +- inventory/
             +- 01-staging.yml
             +- 02-prod.yml
    ```
4.  First, the `ansible.cfg`

    ```
    [defaults]
    host_key_checking = False
    interpreter_python = auto_silent

    [ssh_connection]
    pipelining = True
    ssh_args = -o ControlMaster=auto -o ServerAliveInterval=60
    ```
5.  Then the main playbook: `kali.yml`

    ```
    - name: Deployment
      hosts: kali
      tasks:
        - name: Ping
          ansible.builtin.ping:
        
        - name: Print
          ansible.builtin.debug:
            msg: We could do everything here
    ```
6.  Next we want to add the SSH username and password as variables to the `group_vars/all.yml` but before we do that, we encrypt the password with `ansible-vault encrypt_string 'kali' --name 'ansible_password'`

    Copy the output and open `group_vars/all.yml`

    ```
    ansible_user: kali
    ansible_password: !vault |
        ... <output of previous command> ...
    ```
7.  Finally, we define the inventory, where we specify the targets to run our playbook against

    {% code title="inventory/01-staging.yml" %}
    ```
    staging:
        hosts:
            kali:
                ansible_host: 192.168.0.22
    ```
    {% endcode %}

    and

    {% code title="inventory/02-prod.yml" %}
    ```
    prod:
        hosts:
            kali:
                ansible_host: 192.168.0.23
    ```
    {% endcode %}
8. With this, we have configured a very basic Ansible setup. I highly encourage you to take a look at the [documentation](https://docs.ansible.com/ansible/latest/getting_started/get_started_playbook.html) of ansible. While our playbok really only connects to the target and prints a message, we now have a base to include arbitrary Ansible roles easily. We also added some basic SSH optimizations and showcased the Ansible vault.
9.  Run `echo kali | ansible-playbook --vault-password-file /bin/cat -i inventory/01-staging.yml kali.yml`

    Using `/bin/cat` is a trick to accept the password from the command like. Alternatively, use a password file. Later, we will replace the password on the command line with a more secure option.

{% hint style="success" %}
You should now have a functioning Ansible setup. We can customize both the `staging` and `prod` machine via the respective `inventory` file.&#x20;
{% endhint %}

Finally, what's left is the automation and deployment via GitLab and CI/CD pipelines.

## Create a GitLab Pipeline

GitLab pipelines can be configured to run any job you want on specific conditions (like a merge request, a commit, or even manually). Here, I simply demonstrate how to utilize the pipeline feature to automate all our previous steps. Checkout the [documentation](https://docs.gitlab.com/ci/pipelines/) for more details to customize the pipeline for your needs.

Pipelines in GitLab are created by creating/editing the `.gitlab-ci.yml` file. You can do so by navigating to Build > Pipeline editor - on the left side of the GitLab menu.

A very straight forward pipeline may look like this:

```yaml
# Define all stages (stages run sequentially)
stages:
  - build
  - provision
  - configure

# Define single jobs (jobs of a same stage run in parallel, depending on how many runners you have)
build-job:
  # This job is called build-job and belongs to the stage "build", so it runs first
  stage: build
  tags:
    - debian # GitLab runners will use the tag to see if they can run this job
  script:
    # Here we specify the action
    # At the moment we simply do what we did manually before
    # But you could extend this arbitrarily to for example:
    #  - check for new Kali releases and update the packer script with a new ISO file
    #  - update Packer or the Proxmox plugin
    #  - provide credentials from GitLab secrets
    #  - only run this stage once every month and skip else
    - cd /home/gitlabrunner/packer
    - packer build kali.pkr.hcl
    # If you defined CI/CD variables with the API tokens and set variables in
    # the packer script earlier, you can use
    - packer build -var "proxmox_api_user=$GL_PROXMOX_USER" -var "proxmox_api_token=$FL_PROXMOX_TOKEN" -var 'kali_iso_url=<cdimage.kali.org/url>' -var 'kali_iso_sha256=<sha256sum>' kali.pkr.hcl

provision-job: 
  # This runs once the build stage is complete without errors
  stage: provision
  tags:
    - debian
  script:
    # Again, we could:
    #  - validate the script
    #  - validate the plan
    #  - configure credentials
    #  - take snapshots of the target machines after creating them
    - cd /home/gitlabrunner/terraform
    - terraform apply -auto-approve

configure-job:      
  stage: configure
  tags:
    - debian
  script:
    # This stage is very simplified and unsecure, as we specify the password in the script
    # Optimally, we would deploy SSH keys, but for now, we will have to remove any
    # previous host certificates when we redeploy our VMs, or else, ansible will prevent
    # access via SSH.
    - ssh-keygen -f "/home/gitlabrunner/.ssh/known_hosts" -R "192.168.0.22"
    - ssh-keygen -f "/home/gitlabrunner/.ssh/known_hosts" -R "192.168.0.23"
    - cd /home/gitlabrunner/ansible
    # Ideally, we would use the GitLab vault instead,
    # run the playbook against the staging VM and only if it succeeds deploy the changes
    # to the production VM as well.
    - echo kali | ansible-playbook --vault-password-file=/bin/cat -i inventory/01-staging.yml kali.yml
    
# Though out of scope here, you could also add a stage with a script
# to convert the final production image into different VM formats in the end
```

With this basic pipeline configuration in place, we can attempt to run our pipeline automatically via a GitLab runner.

Note that, since credentials are still hardcoded, we have not yet uploaded all these files to GitLab. That would be our next step after removing all sensitive information. We could then edit all the scripts in Git and trigger a pipeline with our changes. Here, we skip that and use the static code on `gitlab-runner`.

## Create a GitLab Runner

1. In GitLab, navigate to Settings > CI/CD > Runners and select `New project runner`
2. Select a tag if you like (here I've used `debian` to indicate a runner that runs on `debian`)
3. Set a description and "Lock to current projects"
4. Lastly, click on `Create runner`
5. On the next page, select the operating system of the runner (here Linux) and follow the steps 1 to 3 as they are described on the page
6.  Once the `gitlab-runner` is running, it is ready to accept jobs from the pipeline

    You may want to experiment with the option to start the runner as a service instead of enabling it manually every time.

And that concludes this minimalistic guide to Proxmox + Packer + Terraform + Ansible + GitLab. As I have announced at the top, do pay attention to all the secrets floating around as plaintext and move them to appropriate keystores (using GitLab secrets for example)

Enjoy building your own automation pipeline!

{% hint style="success" %}
To test everything, you can go to Build > Pipelines in GitLab and press `New pipeline` at the top right corner.
{% endhint %}

<figure><img src="https://1971224599-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-Mhlz_oZ3oVPSWFmU_3o%2Fuploads%2Fh3sbXIJVLCaviStS6Alx%2Fimage.png?alt=media&#x26;token=3b4caff7-954a-4326-a17d-ee2c7748d5a4" alt=""><figcaption><p>Successful pipeline</p></figcaption></figure>
