nvidia-smi stopped working after installing cuda-toolkit
I purged all nvidia stuff and then tried sudo ubuntu-drivers autoinstall
followed by sudo reboot
after which nvidia-smi
works fine.
So I guess the solution was to re-install NVIDIA drivers.