How to open files with name starting in "." in a program

In the "File - Open" box, hit Ctrl+H to toggle the display of hidden files. You can also right-click in the file list and select "Show hidden files" from the right-click menu. That at least answers the "How to open files with name starting in "." in a program".

The same shortcut key works in nautilus. In nautilus, the setting propagates to all open instances when you change it in one instance. In contrast, GTK3 file dialogs take their initial state from how it was set through nautilus, but can then be changed individually and are remembered for the application as long as it remains open. That, at least, is how it works for several applications, including Gedit and LibreOffice. VSCode, however, may be resetting the dialog settings each time it is closed. GTK2 file open dialogs, such as in Gimp, remember their settings within the application between restarts.

At least, that is how it works on my system: I cannot reproduce the issue you are describing, not on Ubuntu 21.04 with Gnome 40 from PPA, nor on a standard install of Ubuntu 20.04.