Docker Containers can not be stopped or removed - permission denied Error

Issue: Can not stop docker containers, whenever I try to stop containers I get the following Error message,

ERROR: for yattyadocker_web_1  cannot stop container: 1f04148910c5bac38983e6beb3f6da4c8be3f46ceeccdc8d7de0da9d2d76edd8: Cannot kill container 1f04148910c5bac38983e6beb3f6da4c8be3f46ceeccdc8d7de0da9d2d76edd8: rpc error: code = PermissionDenied desc = permission denied

OS Version/build: Ubuntu 16.04 | Docker Version 17.09.0-ce, build afdb6d4 | Docker Compose version 1.17.1, build 6d101fb

Steps to reproduce:

• Created a rails project with Dockerfile and docker-compose.yml. docker-compose.yml is of version 3.
• Image is built successfully with either docker build -t <project name> . or docker-compose up --build
• Containers boots up and runs successfully.
• Try to stop docker compose with docker-compose down.

What I tried::

• I have to run sudo service docker restart and then the containers can be removed.
• Uninstalled docker, removed docker directory and then re installed everything. Still facing same issue.

Note: This configuration was working correctly earlier, but somehow file permissions might have changed and I am seeing this error. I have to run sudo service docker restart and then the containers can be removed. But this is highly inconvenient and I don't know how to troubleshoot this.

Reference Files:

# docker-compose.yml
version: '3'
volumes:
  db-data:
    driver: local
  redis-data:
    driver: local  
services:
  db:
    image: postgres:9.4.1
    volumes:
      - db-data:/var/lib/postgresql/data
    ports:
      - "5432:5432"
    env_file: local_envs.env
  web:
    image: yattya_docker:latest
    command: bundle exec puma -C config/puma.rb
    tty: true
    stdin_open: true
    ports:
      - "3000:3000"
    links:
      - db
      - redis
      - memcached
    depends_on:
      - db
      - redis
      - memcached
    env_file: local_envs.env
  redis:
    image: redis:3.2.4-alpine
    ports:
      # We'll bind our host's port 6379 to redis's port 6379, so we can use
      # Redis Desktop Manager (or other tools) with it:
      - 6379:6379
    volumes:
      # We'll mount the 'redis-data' volume into the location redis stores it's data:
      - redis-data:/var/lib/redis
    command: redis-server --appendonly yes
  memcached:
    image: memcached:1.5-alpine
    ports:
      - "11211:11211"
  clock:
    image: yattya_docker:latest
    command: bundle exec clockwork lib/clock.rb
    links:
      - db
    depends_on:
      - db
    env_file: local_envs.env
  worker:
    image: yattya_docker:latest
    command: bundle exec rake jobs:work
    links: 
      - db
    depends_on: 
      - db
    env_file: local_envs.env

And Dockerfile:

# Dockerfile
FROM ruby:2.4.1

RUN apt-get update && apt-get install -y nodejs --no-install-recommends && rm -rf /var/lib/apt/lists/*

ENV APP_HOME /app
RUN mkdir -p $APP_HOME
WORKDIR $APP_HOME

ADD Gemfile* $APP_HOME/
RUN bundle install

ADD . $APP_HOME

RUN mkdir -p ${APP_HOME}/log
RUN cat /dev/null > "$APP_HOME/log/development.log"

RUN mkdir -p ${APP_HOME}/tmp/cache \
    && mkdir -p ${APP_HOME}/tmp/pids \
    && mkdir -p ${APP_HOME}/tmp/sockets

EXPOSE 3000

I installed Docker from the snap package and after a while I decided to move to apt repository installation.

I was facing the same problem and using sudo aa-remove-unknown worked for me.

So no reinstallation of Apparmor was needed.

For anyone that does not wish to completely purge AppArmor.

Check status: sudo aa-status

Shutdown and prevent it from restarting: sudo systemctl disable apparmor.service --now

Unload AppArmor profiles: sudo service apparmor teardown

Check status: sudo aa-status

You should now be able to stop/kill containers.

I was able to fix the issue. Apparmor service in ubuntu was not working normally due to some unknown issue. The problem was similar to the issue reported in moby project https://github.com/moby/moby/issues/20554.

The /etc/apparmor.d/tunables folder was empty, and https://github.com/mlaventure suggested to purge/reinstall apparmor to get it to the initial state.

So I reinstalled apparmor, and after restarting the problem was solved.

Hope this helps.

A direct fix to the problem is executing bash in the container to be killed and directly calling kill there. An example:

host$ docker exec -it <container-name> sh
container$ ps
PID   USER     TIME  COMMAND
    1 root      0:00 {entrypoint.sh} /bin/sh /entrypoint.sh
   16 root      0:00 {entrypoint.sh} /bin/sh /entrypoint.sh
   24 root      0:00 sh
   31 root      0:00 ps
container$ kill 1

To check that the container was killed, run docker ps. This is a useful alternative to the solution reinstalling apparmor as this will also remove snapd.

In my case the issue was that I had conflicting docker installations: docker itself from the official docker-ce package , but docker-compose from the Ubuntu snap package.

Installing correctly docker-compose from the official github (instructions here) did the trick. I also followed the Linux post-install instructions and it may have helped as well (to run docker as a non-root user)

I just left AppArmor alone here - I did not touch it.