Mysterious "EFI Boot" partition appearing at startup?

Your question has two parts. The first part is repeated below.

The question is, how can I make the EFI Boot disappear when I press the options key at startup?

This a duplicate of the question "Ghost partition, Windows BSOD". The rest of your question, which again is repeated below, does require further explanation.

It's not creating any problems with my Mac or Windows partitions, but I'm very curious about what it actually is, what does it have to do with my computer, and why does it decide to show up when I pressed the options key?

The "EFI Boot" appears because the Mac firmware finds the boot file BOOTx64.EFI in the folder \EFI\BOOT on the EFI partition of your internal drive. This is part of the UEFI specification. For example, if you go to the "uefi.org Specifications web page", you can download and view the "UEFI Specification Version 2.5". Sections 3.5.1.1 Removable Media Boot Behavior and 3.5.1.2 Non-removable Media Boot Behavior (on pages 88-90) describe the naming of the BOOTx64.EFI file and that it should appear in the \EFI\BOOT folder.

When holding down the Option ⌥ at startup, the Mac firmware will look for the \EFI\BOOT\BOOTx64.EFI file on each EFI partition of all drives accessible at startup. Each occurrence will generate an icon with the label "EFI Boot" on the Startup Manager menu.

When Windows installs, some of the startup files are stored in the folder \EFI\MICROSOFT. These files are also duplicated (with some renaming) in the \EFI\BOOT folder. Not all non-Apple operating systems do this when installing. For example, Ubuntu Linux does not. (Well, last time I installed in EFI mode it did not.) The Ubuntu installer does create a folder \EFI\UBUNTU with startup files (GRUB), which the Mac firmware by default ignores. To get Ubuntu to appear in the Startup Manager, you have to manually mount the EFI partition and copy the files in the \EFI\UBUNTU folder to the \EFI\BOOT folder. (Also, the copied grubx64.efi file needs to be renamed to BOOTX64.EFI.)

If you need to do this for two operating systems, then you will need a second EFI partition. This should not present a problem since an EFI partition is fairly small (<200 MB) and the limit on Macs by default is 128 partitions. However, to tell one operating system for the other, while viewing the choices presented by the Startup Manager, you may want to change the icons.

For example, I have rEFInd installed in a second 134 MB EFI partition on my iMac. (This happens to be the last of 9 partitions.) Additionally, I have Windows 10, Yosemite (on volume "Steelhead") and El Capitan (on volume "Steelhead2") installed. Below is a image of my Macs Startup Manager menu. (For a better view, click on the image or open in a new window.)

The icon can be changed by mounting the EFI partition and adding the icon file .VolumeIcon.icns to the root folder. A collection of these icons, for popular Linux and other operating systems, can be found here.