tensorflow (not tensorflow-gpu): failed call to cuInit: UNKNOWN ERROR (303)
My Test
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
My Error
2019-12-27 10:51:17.887009: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/local/openmpi/lib:
2019-12-27 10:51:17.888489: E tensorflow/stream_executor/cuda/cuda_driver.cc:318] failed call to cuInit: UNKNOWN ERROR (303)
2019-12-27 10:51:17.888992: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (3e7d899714a9): /proc/driver/nvidia/version does not exist
2019-12-27 10:51:17.890608: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-12-27 10:51:17.915554: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz
2019-12-27 10:51:17.918061: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56101fca67e0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2019-12-27 10:51:17.918228: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
My Environment
Ubunutu 18
tensorflow 1.15.0/1.14.0
My Question
I have reviewed similar issues for example TensorFlow : failed call to cuInit: CUDA_ERROR_NO_DEVICE
The key difference is I do not have the tensorflow-gpu package installed. How can this error be raised by normal TensorFlow?
You can attempt to work around this problem on cpu-only machines by using the tensorflow-cpu package instead of tensorflow.
pip uninstall tensorflow
pip install tensorflow-cpu
Installing nvidia-modprobe can solve this issue.
sudo apt install nvidia-modprobe
Other solutions you can try are :
- Uninstall and install CUDA and cuDNN.
- Install tensorflow-gpu.
- Uninstall and install different Nvidia driver versions.
The problem also could be that only some /dev/nvidia* files are present before running Python with sudo, check using $ ls /dev/nvidia*, after running the Device Node verification script the /dev/nvidia-uvm file gets added.