Where do I set the Caps Lock key behaviour?

For 16.04 LTS and newer:

Unfortunately disabling CapsLock can no longer be done from the keyboard system settings as depicted below. We will have to install and run dconf-editor to add 'caps:none' to or.gnome.desktop.input- sources.xkb-options.

sudo apt install dconf-editor

This can also be done from a terminal with the following command:

dconf write /org/gnome/desktop/input-sources/xkb-options "['caps:none']"

For 14.04 and older:

Click on the Ubuntu button and search for "Keyboard Layout" and select "Options..." or from System Settings... open the "Keyboard Layout" and select "Options...".

Look for the Caps Lock key behaviour entry and unfold the list of options to give you this: