How do I compile a .java with support for older versions of Java?

I want to compile my .java's (several) into one .jar that are compatible with at least Java 1.6 and newer, preferably Java 1.5 and newer versions of Java. (I have Java 1.7.0_5)

Yes, you can set the version of compiler at compile time. And compile your java code into old versions of java.

From Oracle article : http://docs.oracle.com/javase/1.5.0/docs/tooldocs/windows/javac.html

Cross-Compilation Example

Here we use javac to compile code that will run on a 1.4 VM.

% javac -target 1.4 -bootclasspath jdk1.4.2/lib/classes.zip \
             -extdirs "" OldCode.java

You might also need following parameter to set denote the version of your code.

-source release

-Specifies the version of source code accepted.

As of JDK 9, javac support a new option for cross-compiling

javac --release N ...

which is equivalent to

javac -source N -target N –bootclasspath rtN.jar

You can use javac -target 1.5 <source files>.

If you're using a build system, Eclipse or some other IDE to build jars, please specify which one.