Difference between programs compiled for different OS

On a compiled code standpoint what is the difference between a program compiled for one os vs another (Linux vs windows for instance). Doesn't the program run directly on the cpu? Or is it because the program needs to reference OS specific libraries?

Ordinary compiled programs do "run directly" on the CPU, but a program doesn't run in a vacuum:

1. Many programs rely on external, dynamically loaded libraries (DLLs or .so libraries). The way to link them up is up to the compiler/linker, and each OS has different standards. However, there are also "statically linked" programs that provide all their own code.

2. A modern OS doesn't give complete control of the computer to a running program. Programs rely on "system calls" for i/o, access to hardware, and things like signals and entering a sleep state. The available services and interface are defined by the OS. The OS also controls which parts of the system (memory, registers, interrupts) the program is allowed to use.

3. A GUI program must also work through the graphical user environment in order to draw itself on the screen. But you've probably thought about this already.

For these reasons, OS-independent applications must rely on a "virtual machine" of some sort, like the one provided by the java runtime. Crucially, a VM provides a standard interface to OS resources (i/o, signals, etc). Of course, java or python also interpret "bytecode" instead of dealing with the quirks of Intel's instruction set; but that's a different story.

Different OSes have different functionality as well. Windows has I/O completion ports, Linux does not. FreeBSD has kqueue, Linux does not. Linux has futexes, Windows does not. They also have different ways of doing the same thing -- what parameters do you pass to open a file? What order do they go in? How specifically do you invoke the operating system's "open a file" function?

In general, programs are not compatible due to the differences in their application binary interface (ABI).

Doesn't the program run directly on the CPU?

NO! That's the job of the operating system, to prevent applications from running "directly" on the CPU. Typically, at the lowest level (i.e. the one the OS API is built on), an application interfaces with the operating system's kernel.

Is it because the compiled program itself needs to reference OS specific libraries?

Yes. Many OS libraries are written to facilitate interfacing with the operating system itself, but there's just as many that are written to be cross-platform. These hide the low-level OS interfacing from the developer, and assumes the compiled version for that OS will be available at runtime (see below).

Although libraries can be written in a cross-platform manner, when compiled they can't be run cross-platform. They still need to be recompiled for the specific target operating system, again to utilize the particular underlying components of the operating system (kernel).

What is the difference between a compiled program for one OS vs another?

Finally, executable files themselves often contain very specific binary loading headers and so-forth (e.g. the PE Executable file format [.exe, .dll, etc...] for Windows, or ELF for Linux [none, .o, .so, etc...]). These can also include code to load the compiled OS-specific binaries for a particular software library.

Lastly, from a programmer's perspective: calling convention. Compiled code passes variables to functions in a given manner (i.e. through registers, or on the stack) in a very particular order. Even then, it also needs to be agreed upon who's responsible to "clean up" function calls (the caller or the callee?). Although there are several standard and widely used x86 calling conventions, some may not be supported by certain operating systems (this is part of the ABI).