How does the "Compatibility Mode" in Windows work?
Compatibility mode is achieved using so called shims. There is a good article on TechNet describing how these work.
Windows application files contain an import table which tells the application loader which DLLs the application needs and which functions it uses from them. A process might for example reference GetVersionEx in the kernel32.dll. When a program shall run in compatibility mode, then the shim is put between the application and the shim replaces the GetVersionEx function, so that the application does not call GetVersionEx from kernel32.dll but the GetVersionEx in the shim. The shimmed functions then implement the behaviour of previous Windows versions. GetVersionEx is an easy sample, each Windows version returns its own version numbers in GetVersionEx, so when faking an old Windows the GetVersionEx function now not returns the Windows 7 version numbers but for example the Windows XP version numbers. So the application will believe it is running on Windows XP.
There have also been some other changes from Windows version to Windows version. In older versions for example, if a program loaded a DLL, the search path for the DLL also included the current directory. This is a security issue, so newer versions of Windows by default don't search in the current directory. With the proper shim you can simulate the old behaviour.
Since shims are just a layer between the application and the Windows API a shim can just do what the application could do itself. The shim cannot be used for example to circumvent UAC or access protected files.
If you want to know more, here are some links you might find interesting:
- Using the CorrectFilePaths Shim to Redirect Files on Windows Vista
- Creating an Application Compatibility Shim with the Microsoft Application Compatibility Toolkit
- MSDN blog on internal basics of shims
Especially Microsoft Application Compatibility Toolkit is worth a look. This tool gives you an overview over the applications with known issues, all available compatibility fixes and modes and which fixes are applied to each application.
I think a lot of different things happen. A straightforward example is that a program might check your windows version, but get confused by the return value of a new operating system. So using compatibility mode would tell windows to report a wrong version. Raymond Chen mentions some more things: http://blogs.msdn.com/oldnewthing/archive/2003/12/23/45481.aspx#45590