When spinning up compute instances in the cloud, a proper and fully automated configuration management is a key success factor. I usually use Ansible for the job as it’s open source and widely adopted. But when it comes to highly volatile cloud infrastructure with instances spinning up and terminating on demand, configuration management systems utilizing a control master (like Ansible, Puppet or SaltStack) can be cumbersome.

Hello cloud-init

This is where cloud-init (or cloud-config) takes the stage. I recently needed to provision AWS auto scaling groups made out of spot instances.

This is perfectly doable using Ansible Pull. But Ansile needs a piece of software running upon system boot to initiate the Ansible Pull mechanism. I used cloud-init for this and was (yet again) amazed by it’s simplicity. I had last used it years ago and went ahead provisioning the whole system using cloud-init. I discovered some major advantages:

1. it operates without a control master,
2. it runs upon system boot and is executed by the operating system,
3. is applied once and the system is immutable afterwards and
4. is supported by most operating systems and cloud providers.

The drawback is that it lacks an inventory. Thus inventory information has to be collected using the cloud providers’ api. Nevertheless, most standard configuration management tasks can be performed with cloud-init.

Immutable infrastructure

Terraform is currently the state-of-the-art solution to set up infrastructure in a multi-cloud workflow. It focuses on immutable infrastructure and will leave you with cattle not pets.

Fortunately Terraform has support for rendering Cloud-init manifests. The example below supplies the default.yaml cloud-init manifest as user-data to a Digital Ocean Droplet.

data "template_cloudinit_config" "this" {
  gzip = false
  base64_encode = false

  # users, groups, ssh keys
  part {
    content_type = "text/cloud-config"
    content      = file("manifests/default.yaml")
  }
}

# create droplet at digital ocean
resource "digitalocean_droplet" "this" {
  image    = data.digitalocean_image.centos.id
  name     = "sandbox"
  region   = "fra1"
  size     = "s-1vcpu-1gb"
  user_data = data.template_cloudinit_config.this.rendered
}

For reusable configuration management, you can merge multiple cloud-init parts. In the example below, I first apply basic settings (user, ssh-keys), then install an Nginx server and apply website configuration in the last part. Those parts align pretty much with the Ansible role concept and are reusable throughout other servers.

# template cloud-init
data "template_cloudinit_config" "this" {
  gzip = false
  base64_encode = false

  # users, groups, ssh keys
  part {
    content_type = "text/cloud-config"
    content      = file("manifests/default.yaml")
  }

  # nginx web server
  part {
    content_type = "text/cloud-config"
    content      = file("manifests/nginx.yaml")
  }

  # website configuration
  part {
    content_type = "text/cloud-config"
    content      = templatefile("manifests/redirects.yaml", {
      redirects    = var.redirects
      certificates = acme_certificate.this
    })
  }
}

Each cloud-init part needs a header describing how to merge the different parts. By default, sections of parts get replaced. The following merge_how key merges part sections properly.

#cloud-config
merge_how:
 - name: list
   settings: [append]
 - name: dict
   settings: [recurse_array]

users:
  - name: torsten
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash
    uid: 2005
    ssh_authorized_keys:
      - ssh-ed25519 AAAAC3NzaC1lZ..

You can find all cloud-init parts in my personal infrastructure project as well as the full Terraform setup.

Summary

In my opinion cloud-inits’ simplicity focuses on provisioning the system with just the right amount of information. Plus it does not break the Terraform workflow as you do not need another tool for the job. So maybe give cloud-init a chance!