Newly installed programs always appear with external drives in Finder

When you download an application, it often comes inside a "Disk image" (a .dmg file). This is a virtual volume, which appears just like other disks on your Desktop, once you've launched it.

You have to copy the application from there into your /Applications folder. (Disk images often contain an alias to your Applications folder, so you can easily drop the app onto the folder.
Or, if the image contains an installer app or package, you have to run the installer app.

Once you have done this, you can eject the disk image and trash the .dmg file. If you haven't copied the app or installed it, then you need to make sure this is done first.

Just to further clarify, understand what is happening conceptually when you download and install a program in modern day macOS:

1. In the "old days", you would buy a program from a physical store which would consist of a box containing a DVD, CD, or floppy disk/diskette (in order from newest to oldest). We call these memory devices "physical storage media".
2. The storage media contained the "installation files" which were essentially a copy of the application, and scripts to copy the application files to your internal hard drive, and make any configuration changes as necessary. We'd specifically call these physical media containing the installation files, "[an] install disk[s]".
3. You would then physically insert the external, physical storage media - the install disk - into the computer.
4. You would then "install the application" from the install disk to your internal hard drive.
5. You would then, usually, eject the install disk, as, with the application now installed directly to your faster internal hard drive, you no longer had use for it.*

Nowadays, we are emulating several of those steps virtually:

1. Instead of buying a physical box containing a physical install disk from a physical store, you're downloading a virtual disk from a "virtual" "store" (the Internet).
2. On macOS, these virtual disks usually have the extension *.dmg.* "Inside" of the *.dmg file, similar to how a *.zip file can contain several other files, are all of the application files and installation scripts (if necessary), just like the physical install disks of old.
3. When you double-click a downloaded *.dmg file, macOS automatically recognizes it (you can read more about "file associations") as a virtual disk and automatically "mounts" it as an "external" drive. This is the same process that occurs when you insert a CD/DVD (if you still have a CD/DVD drive), or when connect an external USB drive (USB thumb drive, USB card reader, USB hard drive, etc.). The only difference here is that it is happening "virtually". You aren't actually inserting anything physically into the computer, but the computer is going through the same motions with the *.dmg - the virtual disk - as if you were. That's why when you double-click the downloaded *.dmg file, it appears in your Finder window sidebar with a little eject icon option, just like any other physical external drive you might connect. Even though the *.dmg file is already on your internal hard drive, macOS treats the mounted *.dmg file as if it were a separate disk/drive.
4. From here, the rest of the process is the same: your virtual disk is mounted, just as if you inserted a physical install disk. Now you have to install the application to your internal disk drive. On macOS, this is usually as simple as copying the application to your root /Applications folder. Some applications, though, have an install program (or install script) that do this for you automatically, and potentially handle some more advanced configuration.
5. Once the program is installed from the virtual disk, you can then "eject" ("unmount") the disk, just as you would eject a physical install disk in the old days. The downloaded *.dmg file will still exist in your /Downloads folder, it's just no longer "unzipped" and "mounted". Once your application is installed, you no longer need the *.dmg file either, except for archival purposes, or if you someday need to reinstall that same program (and same version of that program) again in the future. But, except for very rare circumstances involving paid or hard-to-find software, in this Internet era, you can almost always just downloaded the *.dmg again (or the latest version of the program).

* ^{In the "old days" of limited and expensive internal storage, it was sometimes necessary to keep the original, physical, external application media in the computer to run the application or game or video. "Installation" would only involve copying the core application files to the hard drive, but larger application resources would still be read from the external media. This was done because, while external media is/was much slower, it was better than the alternative of using up all of the precious and expensive internal storage. Nowadays, internal storage is so cheap and plentiful, that you almost never see this done anymore, although it is still somewhat common for video game consoles (which are just specialized, branded computers). Also, then, and now, some applications and games can be run without installation, directly from external media. We call these "portable" or "standalone" applications or versions of applications. Many handheld, cartridge-based video game systems still work like this. These exceptions are all tangential to the idea of understanding how most modern-day macOS applications are installed, but it might be useful to know that there are almost always exceptions to everything.}

* ^{There are other common virtual disk extensions and formats. *.dmg is the most common and standard Mac format. *.iso is probably more common, but is generally seen more in Windows or Linux. *.udf is rarer these days, and other specialized formats like *.vmdk are beyond the scope of this answer.}