Restrict user access to GPU
Add a group cuda to your Ubuntu, then set 750 permissions on the */dev/nvidia** devices and change the group ownership to cuda.
Add those users that are granted to use the GPU to the group cuda.
There are also driver parameters that do this for you during loading the nvidia driver. ( NVreg_DeviceFileGID and NVreg_DeviceFileMode ).