What are the binaries of a file?

"sources" means that you get the source code, which is usually a bunch o text files (like .c, .cpp, .h in case of C/C++). You must feed these to a compiler.
Compiling then generates a binary, which is usually an .exe or a .dll file.

So by downloading the binaries you are relieved of the task of having to compile the code by yourself.

A setup will usually also contain binaries but additionally installs these properly.
A zip file containing binaries does not install anything on your system but is usually suitable for using software on a usb-drive (often called "portable" version).

Example

Let's have a look at 7-Zip's download page:
You can download several variants of .exe, .msi and .zip files. These are the binary files intended for end-users.
The other files (.7z, .tar.bz2) contain the source code or files used for developing software that uses 7-Zip.

A setup program (or more properly, an installer) is just like a zip file full of binaries that can unzip itself and copy the binaries to the right places on your computer. The binaries are the programs, ready to be used. They're made from sources, using compilers.