Can't adopt new device when running Unifi Controller in docker container

I'm trying to host the Unifi Controller in a docker container but I can't adopt a new device (access point) unless I set networking mode to "host". I have followed several guides and opened all ports listed here. It's possible to manage already connected devices and to find new devices but adopting them results in a never ending "adopting" status.

The first configuration is based on jacobalberty/unifi:

version: "3.7"

services:
  controller:
    image: jacobalberty/unifi:arm32v7
    restart: unless-stopped
    init: true
    volumes:
      - ./data:/unifi
    environment:
      TZ: "Europe/Stockholm"
    ports:
      - "3478:3478/udp"
      - "5514:5514/udp"
      - "8080:8080"
      - "8443:8443"
      - "8843:8843"
      - "8880:8880"
      - "6789:6789"
      - "27117:27117"
      - "5656-5699:5656-5699/udp"
      - "1900:1900"
      - "10001:10001/udp"
#    network_mode: "host"

The other is a direct copy of linuxserver.io example:

version: "2.1"
services:
  unifi-controller:
    image: linuxserver/unifi-controller
    container_name: unifi-controller
    environment:
      - PUID=1000
      - PGID=1000
      - MEM_LIMIT=1024M #optional
    volumes:
      - ./data:/config
    ports:
      - 3478:3478/udp
      - 10001:10001/udp
      - 8080:8080
      - 8443:8443
      - 1900:1900/udp #optional
      - 8843:8843 #optional
      - 8880:8880 #optional
      - 6789:6789 #optional
      - 5514:5514 #optional
    restart: unless-stopped

The docker container is running on a Raspberry Pi connected to wired network. The host is running other containers but there are no conflicting ports. If there were port conflicts it shouldn't be possible to start the container anyway.

The problem is that the container doesn't know the host's local IP address, so it can't tell the AP what IP address it should use to connect back.

To fix this, you have to manually set the controller IP address in Settings -> Controller -> Controller Hostname/IP and also don't forget to check Override inform host with controller hostname/IP

There is more information about this in the description of the jacobalberty/unifi image, under Adopting Access Points/Switches/Security Gateway.