How to make the workspace toggle indicator disappear in Ubuntu 18.04?

You may use a GNOME shell extension called Disable Workspace Switcher Popup. This extension

Disables the overlay displayed when switching between workspaces.

Check out this extension at GitHub.

Another similar extension is No Workspace Switcher Popup. Note that this one has not been updated for a long time, but it still works with recent GNOME shell versions (3.26/3.28/3.30).

Go to https://extensions.gnome.org/ to search for GNOME extensions.

Go to https://extensions.gnome.org/local/ to view your installed extensions.

Search for "workspace" at the first URL, above, and install any of the GNOME Workspace extensions, like this one...

https://extensions.gnome.org/extension/943/workspace-scroll/

And you'll be able to switch workspaces, without the ugly box, by scrolling whilst pointing the mouse in the top panel.