Ubuntu Server 20.04 cloud-config adding user does nothing

I am trying to create a VirtualBox .box image of Ubuntu Server 20.04 with packer. The box is to be later used by Vagrant. I want to have a single user (vagrant) with root privileges who logs through SSH with public/private key pair.

The configuration files are as follows:

ubuntu2004.pkr.hcl:

source "virtualbox-iso" "autogenerated_1" {
  boot_command            = ["<enter><wait2><enter><wait><f6><esc><wait>", "autoinstall<wait2> ds=nocloud;", "<wait><enter>"]
  boot_wait               = "2s"
  cd_files                = ["./http/user-data", "./http/meta-data"]
  cd_label                = "cidata"
  disk_size               = 8192
  guest_additions_path    = "VBoxGuestAdditions_{{ .Version }}.iso"
  guest_os_type           = "Ubuntu_64"
  headless                = false
  http_directory          = "http"
  iso_checksum            = "sha256:f8e3086f3cea0fb3fefb29937ab5ed9d19e767079633960ccb50e76153effc98"
  iso_urls                = ["https://releases.ubuntu.com/focal/ubuntu-20.04.3-live-server-amd64.iso"]
  shutdown_command        = "echo 'ubuntu'|sudo -S shutdown -P now"
  ssh_handshake_attempts  = "200"
  ssh_password            = "ubuntu"
  ssh_port                = 22
  ssh_username            = "ubuntu"
  ssh_wait_timeout        = "10000s"
  vboxmanage              = [["modifyvm", "{{ .Name }}", "--memory", "1024"], ["modifyvm", "{{ .Name }}", "--cpus", "1"]]
  virtualbox_version_file = ".vbox_version"
  vm_name                 = "packer-ubuntu-20.04-amd64"
}

build {
  sources = ["source.virtualbox-iso.autogenerated_1"]

  provisioner "file" {
    destination = "/home/vagrant/authorized_keys"
    source      = "/home/user/.ssh/virtual_id_ed25519"
  }

  provisioner "file" {
    destination = "/home/vagrant/.ssh/authorized_keys"
    source      = "/home/user/.ssh/virtual_id_ed25519"
  }

  provisioner "shell" {
    scripts = ["scripts/init.sh", "scripts/cleanup.sh"]
  }

  post-processor "vagrant" {
    compression_level = "8"
    output            = "ubuntu-20.04-<no value>.box"
  }
}

The cloud-config yaml is as follows:

./http/user-data:

#cloud-config
autoinstall:
  version: 1
  locale: en_US
  keyboard:
    layout: en
    variant: us
  network:
    network:
      version: 2
      ethernets:
        enp0s3:
          dhcp4: true
  storage:
    layout:
      name: lvm
  identity:
    hostname: ubuntu-server
    username: ubuntu
    password: "$6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0"
  ssh:
    install-server: yes
  groups:
    - ubuntu: [root, sys]
    - cloud-users
  users:
    - default
    - name: vagrant
      ssh_authorized_keys:
        - ssh-ed25519 <<my-public-key>>
      sudo: ALL=(ALL) NOPASSWD:ALL
      groups: sudo, users, admin
      lock_passwd: true
      shell: /bin/bash
  user-data:
    disable_root: false
  packages:
    - openssh-server
    - build-essential
  late-commands:
    - echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/ubuntu

I am explicitly uploading the public key to the image with the two file provisioners. Is this necessary?

The configuration in user-data should follow the documentation.

When Vagrant starts the machine, it can't log in with SSH. If I manually log into the server directly (with the ubuntu user), I can see that there is no user vagrant, i.e. getent passwd | grep vagrant returns nothing.

So,

1. How should I set up the cloud-config so that after the box is created I can log in with the user vagrant and with SSH keys (no passwords)?
2. Do I need the user ubuntu? If not, how can I remove it?
3. What is the identity part in user-data doing? Can I remove the password from there, use SSH keys and not bother with creating explicitly a new user with SSH keys?

I hope that these things can be done inside the configuration files and not via shell scripts.

Solution 1:

If you have an identity section then the users section does not get used. It is not documented this way, so it is likely a bug.

Examples

An autoinstall configuration like this will only create the ruttiger user with a password.

#cloud-config
autoinstall:
  identity:
    hostname: focallive-template
    password: $6$.c38i4RIqZeF4RtR$hRu2RFep/.6DziHLnRqGOEImb15JT2i.K/F9ojBkK/79zqY30Ll2/xx6QClQfdelLe.ZjpeVYfE8xBBcyLspa/
    username: ruttiger
  user-data:
    users:
      - name: vagrant
        ssh_authorized_keys:
          - ssh-rsa REDACTED
        lock_passwd: true
        shell: /bin/bash
        groups: [adm,sudo]
        sudo: ALL=(ALL) NOPASSWD:ALL

An autoinstall configuration like this will create the vagrant user with SSH key authentication. (There will not be any ubuntu user created. It is not required.)

#cloud-config
autoinstall:
  user-data:
    users:
      - name: vagrant
        ssh_authorized_keys:
          - ssh-rsa REDACTED
        lock_passwd: true
        shell: /bin/bash
        groups: [adm,sudo]
        sudo: ALL=(ALL) NOPASSWD:ALL

How does it work

The installer creates the file /target/etc/cloud/cloud.cfg.d/99-installer.cfg. This file contains the user configuration (and some other). When the installed system boots for the first time, cloud-init will include the configuration from this file and create the user(s).

Notes

I tested using Ubuntu 20.04.3 (subiquity 21.08.2).

This line in the source appears to be where the users configuration gets replaced by the configuration provided in the identity section.