inconsistent `apt-get update` behaviour on official Ubuntu AWS AMI

If you are using cloud-init you can wait for it to complete.

while [ ! -f /var/lib/cloud/instance/boot-finished ]; do
   echo 'Waiting for cloud-init...'
   sleep 1
done

e.g. packer json:

{
  "type": "shell",
  "inline": [
    "while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init...'; sleep 1; done"
  ]
}

Reference:

• https://github.com/hashicorp/packer/issues/2639
• https://www.packer.io/docs/other/debugging.html

packer.json before provision:

"provisioners": [
    {
       "type": "shell",
       "inline": ["/usr/bin/cloud-init status --wait"]
},

I just ran into this issue myself and I believe it happens because cloud-init is still in the process of configuring the EC2 instance when apt-get runs. I solved it by inserting a 30 second delay in my script that runs immediately after the instance boots. I think a better way would be to ask cloud-init to run any scripts through User Data or even letting it handle package installation and updates for you [1]. For my use case, where I don't want to acknowledge cloud-init, adding the delay was an acceptable solution.

1. https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html