aws cli not available on github actions

I have been trying to use github actions to deploy a docker image to AWS ECR, but there is a step that is consistently failing.

Here is the portion that is failing:

- name: Pulling ECR for updates and instantiating new updated containers.
        uses: appleboy/ssh-action@master
        with:
          host: ${{secrets.STAGING_HOST}}
          username: ${{secrets.STAGING_USERNAME}}
          key: ${{secrets.STAGING_PEM}}
          port: ${{secrets.STAGING_PORT}}
          script: |
            cd staging 
            
            aws ecr get-login-password --region us-east-2 | docker login -u AWS -p-stdin ***.dkr.ecr.us-east-2.amazonaws.com
            docker pull ***.dkr.ecr.us-east-2.amazonaws.com/*container name*:latest
            docker-compose -f docker-compose.staging.yml up -d
            docker rmi $(docker images --filter dangling=true -q 2>/dev/null) 2>/dev/null
            docker exec -i *** python manage.py makemigrations *dir name*
            docker exec -i *** python manage.py makemigrations accountsettings
            docker exec -i *** python manage.py makemigrations payment
            docker exec -i *** python manage.py runapscheduler
            docker exec -i *** python manage.py migrate

Not sure why it is an issue as github action's virtual environments already has AWS CLI installed (https://github.com/actions/virtual-environments/blob/main/images/linux/Ubuntu2004-Readme.md), and also I am using the AWS CLI in other steps in my github actions and there is no issue, for example:

- name: Build, Tag and Push image to Amazon ECR.
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          ECR_REPOSITORY: *ecr name*
          IMAGE_TAG: latest
        run: |
          cd *dir name* 
          docker build -f Dockerfile.staging -t *container name* .
          aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin ***.dkr.ecr.us-east-2.amazonaws.com
          docker tag *container name*:latest ***.dkr.ecr.us-east-2.amazonaws.com/*container name*:latest
          docker push ***.dkr.ecr.us-east-2.amazonaws.com/*container name*:latest

and the image successfully gets pushed to my aws ECR.

I have tried to install the aws cli as suggested here: GitHub Action - AWS CLI, but still to no avail.

here is the code I used to install the aws cli:

 - name: Intalling aws cli via python pip
        run: |
            python -m pip install --upgrade pip
            pip install awscli 

Here is the full error I have been getting:

======END======
err: bash: line 2: aws: command not found
err: WARNING! Using -*** the CLI is insecure. Use --password-stdin.
err: Error response from daemon: login attempt to https://***.dkr.ecr.us-east-2.amazonaws.com/v2/ failed with status: 400 Bad Request
err: Error response from daemon: Head "https://***.dkr.ecr.us-east-2.amazonaws.com/v2/*ecr name*/manifests/latest": no basic auth credentials
err: Pulling web (***.dkr.ecr.us-east-2.amazonaws.com/*ecr-name*:latest)...
err: Head "https://***.dkr.ecr.us-east-2.amazonaws.com/v2/*ecr-name*/manifests/latest": no basic auth credentials
err: Error: No such container: ***
err: Error: No such container: ***
err: Error: No such container: ***
err: Error: No such container: ***
err: Error: No such container: ***
err: Error: No such container: ***
err: Error: No such container: ***
err: Error: No such container: ***
20***/01/19 04:59:42 Process exited with status 1

Solution 1:

Welcome to StackOverflow and the joys of programming and the cloud!

It seems that the AWS CLI is failing to configure the access key id and secret on the pipeline. In order to solve this and make it easier to manage in the long run, I would recommend using the pre-built actions from AWS to ease your pipeline's setup process.

The most common way of building a Github action pipeline for pushing images to AWS ECR is by using the following actions:

• aws-actions/configure-aws-credentials@v1
• aws-actions/amazon-ecr-login@v1

Using the combination of these actions together enables us to configure the pipeline's shell session to store temporary credentials for the AWS CLI and the ECR credentials for the docker login.

    steps:
    - name: Checkout
      uses: actions/checkout@v2

    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
        aws-region: ap-south-1

    - name: Login to Amazon ECR
      id: login-ecr
      uses: aws-actions/amazon-ecr-login@v1

    - name: Build, tag, and push the image to Amazon ECR
      id: build-image
      env:
        ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
        ECR_REPOSITORY: ${{ secrets.REPO_NAME }}
        IMAGE_TAG: 1.0
      run: |
        # Build a docker container and push it to ECR 
        docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
        echo "Pushing image to ECR..."
        docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
        echo "::set-output name=image::$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG"

If the guide above is not sufficient and you need help in configuring the access keys and secrets, I would recommend following the blog written here