Terminal.app and tmux session, can't use "open" command (without tmux it works)

I use Terminal.app and a tmux session.

> open -a VLC file.mp4
The window server could not be contacted.  open must be run with a user logged in at the console, either as that user or as root.

Without tmux this command works.

I googled that and found Unable to use 'open' command in OSX tmux from 10 July 2015.

brew update
brew install reattach-to-user-namespace
brew upgrade reattach-to-user-namespace
echo "set -g default-command \"reattach-to-user-namespace -l ${SHELL}\"" >> ~/.tmux.conf
cat .tmux.conf 
set -g default-command "reattach-to-user-namespace -l /bin/bash"

I did that and have still the same error

The window server could not be contacted.  open must be run with a user logged in at the console, either as that user or as root.

what is reattach-to-user-namespace supposed to do?

The command is supposed to move the tmux process from a newly created bootstrap context under the root, to the user's login bootstrap context.

To explain that in slightly more common terms:

macOS (formerly OS X) is built with a Mach-kernel at the lowest layer. Programs executing at this layer are called tasks.

On top of the Mach-kernel is the BSD subsystem. Programs executing at this layer are called processes. The applications you run and know on OS X (such as for example tmux or Terminal.app) use the BSD subsystem, so when running they become processes.

When a running program (a process or task) needs to communicate with another process/task over a Mach port it will go through the so called bootstrap task. This very low-level task is responsible for looking up requests for Mach port, and essentially direct the information to the right place.

macOS / OS X is essentially a multi-user system. Multiple users can be logged in at the same time, and each can be running multiple processes. They are generally not allowed to interact with each other, except when explicitly allowed.

In addition to user processes, the system will at boot (and sometimes later) run background programs called daemons. They are not associated with a specific user login, and keep running when users log out.

The previously mentioned Mach bootstrap task groups all tasks into either the "startup context" for daemon programs, or the "user login context". There's one of the latter for each currently logged in user.

In essence when programs look up Mach ports to communicate over, they can only see ports in their current context. When you start a normal program, it is started in your user's login context - and thus you can communicate with other programs you have started.

The tmux process however uses the function call daemon() to "daemonize" - i.e. keep running in the background. This also has the effect of moving the tmux process from the user's login context to the startup context.

Now programs such as "open" will no longer work, because they function by looking up the WindowServer's Mach port for the currently logged in user, and sends it commands to do their job. The lookup will fail, as tmux is now in the wrong bootstrap context.

The reattach-to-user-namespace program fixes this problem by using an uofficial way of interacting with the launchd process (which is responsible for managing daemon processes) - to get launchd to tell it which context is the user's login context.

Then it uses the Mach system call task_set_bootstrap_port() so that further port looks from the tmux processes (and its child processes) will hit the user's login context bootstrap task.

As to why it is not working for you: Have you remembered to kill and restart your tmux server after installation? (do this by running "tmux kill-server)

Also you could try running the reattach-to-user-namespace command manually from Terminal.app to see if you really have got it installed and it runs properly.