How to switch windows/tasks using arrow keys

One can switch applications using the Task Switcher which is brought up by Alt + Tab. Once brought up I can cycle through applications using the arrow keys. I want to bypass the Task Switcher all together and navigate between windows directly using the keyboard. Something like Super + Home goes to the previous window and Super + End goes to the next window. I have been lucky to find this post which gets close but is not quite there Use arrow keys when switching applications with super + tab. Is there any way to get this functionality in Gnome 3 just with changing settings and the command line? Is there an extension to do this? Or is this the problem that is solved by window managers, if so, can I expect this sort of functionality out of most/all window managers?

Edit (As of 09/22/2021)

It looks like the two switch-windows keybindings don't work the way they use to. Nowadays I use Material Shell.

Thank you to Nmath for the helpful answer. Right below Switch Windows Directly in the Navigation section is Switch Windows (unbound by default I think) which brings up a arrow key navigable popup much like the one for Task Switcher.

But what if I want to switch windows without the popup. I discovered Alt + Esc cycles through windows. This led me to the Switch Windows Directly setting under the Navigation section after doing Devices -> Keyboard in the Settings app. One can change the key binding for Switch Windows Directly, which acts as a "next window" command, but unfortunately the corresponding "previous window" command is then bound to Shift + (the key binding of Switch Windows Directly) and can't be rebound in the Settings app. However, there is a command line utility call gsettings that can do what we want. To see the key bindings it has, do

gsettings list-recursively | grep org.gnome.desktop.wm.keybindings

After studying that output and some other basic gsettings information, these are the commands I used to create keybindings that allow arrow key navigation of windows.

gsettings set org.gnome.desktop.wm.keybindings cycle-windows "['<Super>Right']"
gsettings set org.gnome.desktop.wm.keybindings cycle-windows-backward "['<Super>Left']"

And if you prefer to have the popup, I bet

gsettings set org.gnome.desktop.wm.keybindings switch-windows "['<Super>Right']"
gsettings set org.gnome.desktop.wm.keybindings switch-windows-backward "['<Super>Left']"

(changed cycle to switch) would work for that (though I haven't tested it myself). I don't know how this stuff interacts with the Settings app. To be safe I just removed any key bindings for a command before making new ones for the command. Hopefully this has helped someone.