Registering COM EXE with WIX

I created a solution with two new projects: ATLProject1 wich is a COM dll and ATLProject2 which is a COM EXE. To both projects added same simple class with single method.

Added both to WIX setup project (this is an existing setup project that already installs other COM dlls. I'm just using it for this test).

Added following lines to product.wxs:

        <Component Id="ATLProject1.dll" Guid="{...}">
          <File Id="ATLProject1.dll" Name="ATLProject1.dll" KeyPath="yes" SelfRegCost="0" Source="$(var.TargetDir)\ATLProject1.dll" />
        </Component>
        <Component Id="ATLProject2.exe" Guid="{...}">
          <File Id="ATLProject2.exe" Name="ATLProject2.exe" KeyPath="yes" SelfRegCost="0" Source="$(var.TargetDir)\ATLProject2.exe" />
        </Component>

and also

  <ComponentRef Id="ATLProject1.dll" />
  <ComponentRef Id="ATLProject2.exe" />

The file also has these lines:

<EnsureTable Id="PublishComponent"/>
<EnsureTable Id="Condition"/>
<EnsureTable Id="TypeLib"/>
<EnsureTable Id="Class"/>
<EnsureTable Id="Extension"/>

When running the setup I get error: "Module ATLProject2.exe failed to register. HRESULT -2147024769" (hex 0x8007007f the specified procedure cannot be found).

If I remove ATLProject2 from setup, it succeeds and ATLProject1 is correctly registered in registry (this is without generating registry information e.g. using heat, it just works).

Should exe component be treated differently?

I found this 10 year old post suggesting heat.exe does not treat COM exe as COM. If this is the problem, Not sure if this is still the case?

The Windows Installer does not recommend using SelfReg to register at install time. Instead, adding the registration to your .wxs code or capturing the registration at build time is highly recommended.

To add the registration manually, you don't use EnsureTable, you use the COM related elements (like Class, ProgId, TypeLib). It can be tedious but will be far more robust than trying to selfreg during installation.

Unfortunately, the alternative to capture the registration during build using heat.exe (provided in the WiX toolset) does not support capturing from executables. If you are open to a commercial solution, we (FireGiant) developed an alternative to heat.exe that can capture executable registration (and much more). That advanced harvesting solution has more documentation on the FireGiant site.