How does Windows Actually Handle File Associations?

I have a problem with file associations. My C# application can set the file association for its own extension so that users double clicking the project file it uses will open my app and load the file.

This causes problems with two third party components. Each use licensing.

When using the double click, the license dialogs for both components appear. When running the app from the executable this does not happen. If I drag the appropriate file onto my app shortcut the app starts with the file and no license dialogs appear.

Therefore I have to conclude that the problem is in the way Windows executes the file association action.

The folks who create the license code have told me that the problem is likely to be that the executable name got changed and the license code rejects it. I can only assume that this relates to the argument containing the file name.

I have tried to catch the code execution via breakpoints in my code but I can't get it to happen even if the breakpoint is on the first line that executes. Whatever Windows is doing gets past that point.

How associations work

When you double-click on a file in Windows explorer, the Windows shell looks up the extension of the file in the registry to see if the extension is registered. If the extension is not registered, Windows displays the Open With dialog box, allowing the user to choose an application to associate with the file type. If the extension is registered, Windows calls the ShellExecute() function with a command of "open." It also passes the name of the file that was double-clicked as a command line parameter.

Associations go further than simply opening a file, though. If you right-click on a text file (.TXT) in Explorer you will see two items at the top of the context menu. The first is named Open. Choosing this menu item is the same as double-clicking the file in Explorer. When you choose Open, NOTEPAD.EXE will be started with the selected file loaded (assuming a default Windows installation). The second menu item is called Print. Clicking this menu item will cause the file to be printed without displaying Notepad at all.

Other file types display even more items on Explorer’s context menu. If you right-click on a Microsoft PowerPoint file, for example, you will see context menu items named Open, New, Print, and Show. The items shown on the context menu for a particular file type are obtained from the registry.

There are at least two ways to create a file association in Windows. One way is to right-click a file in Windows Explorer and choose Open with… from the context menu. When you do, Windows will display the Open With dialog. Naturally, this method requires user intervention. When you deploy your application you probably don’t want to force your users to set up a file association manually.

A better way to create an association is by making various registry entries from your application. A good installation program will make the registry entries for you, but there are times when you need more control over the process.

Registering an association

Registering a file association requires creating two separate registry keys. Both keys are created in the HKEY_CLASSES_ROOT section of the registry.

The file extension key

The first key is the name of the file extension, preceded by a dot.

HKEY_CLASSES_ROOT\.zzy

In a production application, you should check the registry to be sure a key does not exist before you attempt to create a new key. If the key already exists, your application will need to either prompt the user to replace the file association, or be prepared to use a different file extension altogether.

The value of this key is linked to the second key you will create. In fact, it is the name of the second key. For the example program, I gave this key a value of "Test App File." This value can be anything you choose, but, as with the first key, you must be sure the key does not already exist in the registry.

The application association key

The second key has the same name as the default value for the first key.

HKEY_CLASSES_ROOT\Test App File

This key must have at least one subkey. Windows uses this subkey when it executes the application. The entire key is structured as follows:

HKEY_CLASSES_ROOT
  Test App File
    shell
      open
        command

The string given to the command key is the full path and file name of the application followed by %1 . For example:

C:\MyApp\MyApp.exe %1

When Windows launches the application, it replaces the %1 symbol with the path and file name of the file that was double-clicked in Windows explorer. This value is passed to your application as a command line parameter.

Additional keys

There are other subkeys that you can create under the file association key. One such key is the DefaultIcon key. This key is used to specify the icon that the Windows shell will display next to files of the registered types. This key is not required if you only have one file type registered and if that file type should use the application icon. Here’s how the value of the DefaultIcon key looks for an association that specifies the default application icon:

C:\MyApp\MyApp.exe,0

This specifies that the first icon found in the application’s EXE file should be used as the file association’s display icon. If your application has more than one file type, you can specify other icons by changing the icon index that follows the comma. For example, C++Builder has icons for a project file, a form file, a source file, and so on. If you look in the registry under HKEY_CLASSES_ROOT\BCBProject\DefaultIcon you will see that the icon for a project file is icon index 4 (for C++Builder 4, at least).

If you want to allow users to print a document you can add a print subkey in addition to the open subkey. The value of the print subkey is similar to that of the open subkey, with one exception:

C:\MyApp\MyApp.exe /p %1

Notice that this value has a command line switch of /p inserted between the application name and the %1 symbol. Your application can watch for the /p switch and take appropriate action when the switch is detected.

You can add as many subkeys as you like for a particular file type. The name of each subkey will appear on the Explorer context menu. You only need to add a command line switch for each command type so that your application can identify the context menu item that was selected. If you provide a default value for the subkey, Windows will use that text for the context menu item text. If you do not supply a default value, Windows will use the key name itself for the menu item.

All information is taken form this article.