Cannot access secondary GPU - error: Could not load GPU driver

I fixed it myself. I stumbled upon this thread since I also installed the latest drivers; Nvidia (Bumblebee) failed to load after install latest Intel driver

But I did the instructions:

sudo apt-get purge i915-3.15-3.13-dkms
sudo apt-get install --reinstall xserver-xorg-video-intel libgl1-mesa-glx libgl1-mesa-dri xserver-xorg-core
sudo dpkg-reconfigure xserver-xorg
sudo apt-get install --reinstall nvidia-340
sudo dpkg-reconfigure bbswitch-dkms
sudo reboot

But it didn't work for me, so I searched some more. I found this bug report and tried these instructions (https://bugs.launchpad.net/ubuntu/+source/ubuntu-drivers-common/+bug/1310023):

sudo ldconfig -n
sudo update-initramfs -u
sudo reboot

Eureka, it worked!