mysql authentification plugin is not loaded
This issue is explained here here.
Run mysql_upgrade -u root -p to fix the issue.
OR
If you are creating the user and getting this issue, then use BY instead of WITH in your query.
CREATE USER 'myuser'@'localhost' IDENTIFIED BY 'xyz';
Turns out this is because I forgot to run mysql_upgrade on the database after rebuilding the docker container with the latest mysql version.
After running a mysql_upgrade everything works again as expected. More information can be found here. https://bugs.mysql.com/bug.php?id=79622