Fetch autoinstall based on MAC

So it seems I finally solved it. It's both easy and riddled with bugs and obstacles. Answer is yes - use early-commands. But truth is in the details, so detailed answer it is.

First of all, prepare the rest of your environment, you can see my other post about detailed steps I went through to do BIOS/UEFI PXE boot of 20.04 and 20.10: https://askubuntu.com/a/1292097/1080682

Now when you get your environment working correctly (good luck), let's do the custom autoinstall based on config changes served via HTTP mid install.

So if you follow the guide I posted on the link, I kept my Ubuntu user-data here :

/var/www/html/ubuntu-server-20.04.1/user-data

Modify the file with something like this (note I've shortened it for readability):

#cloud-config

autoinstall:
  version: 1
  refresh-installer:
    update: yes
  apt:
    <apt stuff>
  identity:
    hostname: pxe-client
    password: $6$zN/uHJD1rEXD/ETf$q8CoBt3xXmBT37RslyWcpLT1za4RJR3QEtosggRKN5aZAAf6/mYbFEQO66AIPm965glBXB1DGd0Sf.oKi.Rfx/
    realname: pxe
    username: pxe
  keyboard: {layout: hr, toggle: toggle, variant: ""}
  early-commands:
    - curl -G -o /autoinstall.yaml http://10.10.2.1/user-data -d "mac=$(ip a | grep ether | cut -d ' ' -f6)"
  locale: en_US
  network:
    network:
      version: 2
      ethernets:
        eth0:
          dhcp4: yes
          dhcp6: no

Now, this user-data can be real basic, all we need really is for it to have network enabled and that one-liner curl in early-commands. IP 10.10.2.1 is local IP of my HTTP server (also my PXE server, as I serve other config files and ISO images and all that through it, but doesn't matter).

Use whatever you want to modify and serve this file based on the request. Way it is being done above with curl you will actually request from server something like this:

GET /user-data?mac=fa:fa:fa:00:0e:07

The part fa:fa:fa:00:0e:07 is what server sends after querying it's own interfaces. If you have multiple interfaces you'll maybe need to tweak the script, or make sure only one interface is up during early install steps.

I'm planning to use that via PHP + MySQL, and after fetching it in PHP using $_GET["mac"] do something like SELECT * FROM autoinstall-configs WHERE mac = '$_GET["mac"]'; and from data in database table build the new autoinstall.yaml and serve it back to subiquity.

Anyway, your reply has to NOT HAVE the line autoinstall: !!

Here is minimal example of what HTTP/PHP will reply, I changed just hostname and username, and modified it so it passes subiquity syntax checks, oh and excluded early command not to get stuck in loop:

  version: 1
  refresh-installer:
    update: yes
  apt:
    <apt stuff>
  identity:
    hostname: php-client
    password: $6$zN/uHJD1rEXD/ETf$q8CoBt3xXmBT37RslyWcpLT1za4RJR3QEtosggRKN5aZAAf6/mYbFEQO66AIPm965glBXB1DGd0Sf.oKi.Rfx/
    realname: php
    username: php
  keyboard: {layout: hr, toggle: toggle, variant: ""}
  locale: en_US
  network:
    network:
      version: 2
      ethernets:
        eth0:
          dhcp4: yes
          dhcp6: no
  ssh:
    allow-pw: true
    install-server: true
  late-commands:
    - poweroff

To make it clear, here is the diff of the two files:

diff /var/www/html/ubuntu-server-20.04.1/user-data /var/www/html/user-data

1,3d0
< #cloud-config
<
< autoinstall:
16c13
<     hostname: pxe-client
---
>     hostname: php-client
18,19c15,16
<     realname: pxe
<     username: pxe
---
>     realname: php
>     username: php
21,22d17
<   early-commands:
<     - curl -G -o /autoinstall.yaml http://10.10.2.1/user-data -d "mac=$(ip a | grep ether | cut -d ' ' -f6)"

So it's almost the same, just not. These changes (removal of autoinstall: and early-commands: ) is required to pass on to rest of installation. You can test other tweaks on your own.

After that installation will continue with whatever new info was served via answer to request /user-data?mac=<installer-mac-address>.

This is now opening door to further possibilities of making your own web management for your VMs or server farms, or whatever. No longer do you need hand crafted user-data file for every server or group of servers. You can send them each their unique config, so including exact static IP address, partition sizes, different hostname, password, etc.

Canonical, if you pick an idea from this, put me in the credits at least :)

Case closed, cheers!