Do we compile the source or the binaries?

Solution 1:

To address your question specifically, "compile" in computing terms takes the sources as its direct object.

We compile the source files.

"Compile" doesn't take the binaries as a direct object; they are more of an indirect object that we "compile into." You could say:

We compile to the binary files.

but it sounds a bit strange and isn't common.

What might possibly confuse you is that if what we are working on has an overall name, we can use that as a direct object:

I need to compile GStreamer.
He is compiling the server now.

Solution 2:

Compile

1. To gather into a single book.

2. To put together or compose from materials gathered from several sources: compile an encyclopedia.

3. Computer Science To translate (a program) into machine language.

Yes, we compile the source files into binary files

EDIT: Well, strictly speaking, the proper usage would be to compile binary files from sources but no one in IT uses compile in this way:

The closest would be compile a program, but since program is used for both sources and binaries, it's not clear which of them is meant.