How do I avoid common design flaws in my WiX / MSI deployment solution?

Solution 1:

WiX / MSI Deployment Anti-Patterns

There are several deployment anti-patterns often seen in WiX / MSI files. Below is a rough-draft of some of the most common ones.

Before going into the problems, on the other hand here is a quick reminder of the things that have made MSI an overall success! (despite its problems).

This answer is a work in progress

What do you know I hit the max size for the answer. I guess that's a hint it's enough already :-). Some sections need clarification and improvement though.

If you recognize some of these problems, you might want to read on - these are all well known developer hates and annoyances with Windows Installer / MSI:

You cannot reliably overwrite a lower version file with your newest setup.
You cannot reliably overwrite non-versioned files (for example for IIS).
Files are mysteriously missing after you try to install an MSI upgrade.
Data gets wiped out during (major) upgrade scenarios. Examples include:
- Your registry stored license key.
- Data in configuration files such as config.xml, settings.ini, etc...
- Your service credentials for the service you don't run as LocalSystem.
Data does not get updated:
- Settings files are not reliably updated during installation with new settings you want to enforce.
- You have problems updating settings in data files stored per user (or HKCU). You update for the installing user, how do you update for other users?
You see self-repair kick in unexpectedly for your package.
Your custom action makes the setup bomb out with mysterious errors.
And this is a big one: you unnecessarily use custom actions for things that is already fully supported by the Windows Installer itself. This is a huge deployment anti-pattern, and the leading cause of deployment failures.
- You install Windows Services via custom actions. This is much better done in the MSI itself using built-in constructs.
- You install .NET assemblies to the GAC via a custom action. This is fully supported by Windows Installer itself without a line of (risky) code.
- You run custom .NET assembly installer classes. These are to be used for development and testing only. They should never be run as part of deployment. Rather your MSI should use built-in constructs to deploy and register your assembly.
- You run prerequisite setups and runtime installers via a custom action in your own MSI. This should be done entirely differently. See section 6.
You have problems deploying shared runtime files.
Silent installation of your MSI appears to result in a different installation state than running the setup interactively.

The sections below are in no particular order at all - as of now.

The sections are continually sought to be improved. Please add comments on what isn't clear or helpful.

Pending addition:

Difficult multi-instance installations

Relatively common requirement, especially for service installations

Uninstall is not working for MSI application - Error 1722

Service control: Failing to stop services before uninstalling

Uninstall CAs: Trying to run batch files / scripts that are no longer on disk during uninstall

Custom Actions: Erroneous conditioning so custom action runs unexpectedly. Often on uninstall or during major upgrades.

1. Self-repair problems

A particularly annoying problem is related to constructs that frequently trigger unwanted self-repair for your installed application.

Due to the multi-faceted nature of this problem, I have created a separate answer to describe the design constructs to avoid in order to prevent self-repair from striking without warning and intent for your application: How do I avoid triggering MSI self-repair with my WiX / MSI package?.
Sometimes self-repair is used as a method to populate HKCU with application settings, or put files in each users's user profile. This generally works, but is in my opinion not the best practice for application design and deployment - see more details below in section 9.

2. Incorrect installation of shared, vendor or Microsoft runtime files

Although this is extensively explained in the link above (self-repair issues), it should be noted here as well that one of the most common errors in any setup is the inclusion of "local copies" of shared runtime files - sometimes also globally registered on the system if they are COM files. The installers for old VB6 applications sometimes did this for common controls they required, breaking the system for other applications.

If you need a particular version of a shared file for COM use, and there is no way you can update your application to use the properly installed, shared component, then you can use registration-less COM. Essentially installing local copies of the binaries you need and force their loading over shared files via manifest files provided for the binaries.
See the self-repair issues link in item 1 above for more details on this topic.

3. Incorrect handling of (your own) shared files and data

If you create a suite of MSI files to deploy different products, they might share certain files between them. If you target the same file location (absolute path) from several MSI files - each using a different component GUID, then each setup will treat the file as if it "owns it" - happily uninstalling it on uninstall, or putting it in place again via self-repair.

The proper solution for this is to realize that for every absolute path you target, there must be a single component GUID. Absolute paths are reference counted by a component GUID - and it must be shared between all your setups for this to work correctly.
To achieve using the same component GUID in all your setups you should either create a merge-module to include in each setup, or use advanced constructs in WiX such as "include files" - with hard coded GUIDs for the components contained in them.
If the file in question is a data file that should never be uninstalled or replaced once updated, you should also consider installing it as a "permanent component" so that it is not uninstalled during major upgrades or manually run uninstalls.

4. Component creation errors - not following best practice

Not following best practice for component creation. MSI components are the basic installation units for files and registry settings.

There are best practice rules for how to "componentize" your application files. Breaking these rules can cause problems for patching and upgrades with mysterious symptoms such as missing files and settings after upgrades, or patches that bomb out with nonsensical errors.
To counter this problem the oversimplification is that you should use one file per component unless the number of files in your setup is truly enormous. This avoids all kinds of problems (read that link for a more thorough explanation of component ref-counting).

5. Upgrade problems relating to user data being overwritten or reset

This is no less than extremely common. I have answered several stackoverflow questions on this topic, and it keeps coming up.

Please read the section called "Overuse of per-user file and registry deployment" for a description of how to minimize the reliance on Windows Installer for user-data deployment in general. If you ask me this is the real answer to these persisting "data reversion" problems.
Since upgrades are complex in MSI, many standardize on major upgrades (the simplest form of upgrade). A major upgrade is essentially an uninstall and a reinstall of the same product (in different versions).
There are several ways to configure such a major upgrade, but if you uninstall the previous version completely before installing the new version, you could uninstall user data files that have been modified since installed. MSI does not check if data files have been modified since installed and will happily uninstall them without hesitation, unless you have marked the hosting component as "permanent" (it will never be uninstalled) or set a blank component GUID for the hosting component (a special feature to install the file and then ignore it completely).
A special case to be aware of is that even if you properly share such a file by using a merge module or WiX include file (to keep the installing component GUID stable)- it will likely still be uninstalled and reinstalled by a major upgrade if there is just one product on the box that has it referenced it at the time (reference count is 1).
After the major upgrade has completed it looks as if the data files have been overwritten or reverted, but in actual fact the modified data files were simply uninstalled and then reinstalled in their "fresh versions" (will update with some potential fixes for this soon).
In my opinion you should only install data files that are used read-only after installation. If the files should be written to, they should be generated by the application itself in my opinion, and stored in the user-profile. This is an example of how application design can be changed to make deployment more reliable. The "real solution" in my opinion.
If you do install the read/write data file with a component, set it permanent (or use blank GUID). The file overwrite rules will ensure that the file on disk isn't overwritten during installation (unless you do something stupid such as setting REINSTALLMODE to amus to force overwrite all files - this should never be allowed. It can downgrade shared files installed by merge modules as well - old-style DLL Hell). If you do want to wipe the file out and overwrite it, that is also possible using various methods, the best of which is probably to use a companion file. (more details will be added later).
Wix: Windows Service sometimes uninstalled when upgrading

6. Erroneous or unnecessary use of custom actions

The (over)-use of custom actions for MSI files is a huge topic and this section got too large and was split into a separate answer: Why is it a good idea to limit the use of custom actions in my WiX / MSI setups?.

Essentially custom actions are often unnecessary due to built-in support in MSI to achieve the same effect, or the availability of ready-made solutions in free frameworks such as WiX or commercial tools such as Advanced Installer or Installshield.

And custom actions are by their very nature error prone and the leading cause of deployment failures and errors. Please read the above link for details. Thousands of people, tens of thousands of people, even millions of people have tested these built-in constructs. Why on earth do you do it on your own?

Some "besserwissing" (advice that I should follow myself): Focus on what sets your product apart - what is new about it, and eliminate all other sources of errors. Good deployment won't make your product, but bad deployment can break it.

7. Failure to properly merge INI files

It is possible to install an INI file via the File table - as you would any other file. This allows no merging if there is an existing INI file at the target location.

If you import the INI entries into the appropriate MSI tables, you can update an existing INI file using "merging" with existing values, and not just do a file overwrite "wiping out" the existing entries, or not updating the file at all.
The "INI merging" is "auto-magic" which allows proper rollback support and "pin-pointed" updates to the values in any existing INI file. If the installer is aborted, the INI file is properly reverted to its initial state.
This is an excellent feature that really works great for almost any INI file I have ever seen. However, I have indeed seen a few cases where INI files have non-standard formatting. Sometimes they have large comments sections that you want to install (developer tools) or weird formatting that can't be supported by MSI's merging (comma delimited triple files and stuff like that). In these cases you have to install it as a file instead of as a "change transaction" to preserve the uniquely formatted INI file.
If you are developing and using a non-standard INI file, consider giving the file a different extension than *.INI in order to indicate its uniqueness and need for special handling. It is effectively no longer an INI file (key-value format). The opposite is also true: you have a unique extension, and you can change it to INI to handle it as a proper INI file if the file content is key-value pairs.

8. Erroneously using self-registration for COM files

Or install their registration via the Registry table. Use the appropriate COM advertisement tables. There are many reasons, as explained here: Self-registration considered harmful.

I have seen cases where self-registration performs other actions than actual COM registration on the system in question. This is generally horrible design from the developer in question, but I know of cases where people have chosen to use self-register rather than to re-implement what is done during self-registration as a proper custom action.
To allow a personal opinion: when I see network settings being affected by self-registration, I immediately want the software rejected for use altogether. That is how serious it is to do something so "hacky" in an standardized operation such as self-registration. The sane question to ask is "what else are they up to given that dodgy COM registration". It is just not a confidence builder to rely on non-standard, hacky stuff.

9. Overuse of per-user file and registry deployment

UPGRADE: new answer relating to this topic: Create folder and file on Current user profile, from Admin Profile.

This section got too large and was split into a separate answer: Why is it a good idea to limit deployment of files to the user-profile or HKCU when using MSI?

Essentially user-profile deployment of files or settings in HKCU are tolerable, but it might not be the best design, and it can be cumbersome to ensure that all settings and files make it into every user-profile and user registry on the box. The deployment problems that result and some proposed solutions are discussed in the linked answer above.

Essentially user deployment can be supported using MSI self-repair, Microsoft Active Setup, or by logical design changes to the application or solution in question (the preferred option - see linked answer for details). In general deployment should not interfere with user data and settings since it is really user data and should not be deployed but generated at runtime by the application.

10. Silent installation fails to complete or is incomplete

A built-in feature of Windows Installer is that any MSI file can be installed in silent mode. This is a core feature of the technology intended to help corporate deployment - which is generally always run in silent mode. Making sure your MSI is capable of completing and successfully working after a silent install, is no less than exceptionally important. In my experience custom actions can often cause problems for silent install.

Never make changes to the computer from within the InstallUISequence (from your setup dialogs). This issue was described above. The custom actions used in the interactive GUI are immediate mode (without elevation for regular users) and should just gather and validate user input (read-only). All non-standard changes made to the computer should be done between InstallInitialize and InstallFinalize in the InstallExecuteSequence - the transacted, elevated operations where only deferred mode and elevated custom actions can run.
- All changes made in the InstallUISequence will also be skipped entirely when you run in silent mode, and the install will then likely be incomplete. Silent installation is extremely important for corporate deployment - the GUI is generally always ignored and changes are enforced by using transforms and/or setting properties from the command line.
- Here is a lengthy discussion of how silent and interactive installs and uninstalls can yield different results (and how it is a serious MSI design flaw): Uninstall from Control Panel is different from Remove from .msi
Never show dialogs from within your custom actions in InstallExecuteSequence. Doing so can cause silent install to fail completely since these dialogs won't automatically obey the UILevel setting of the running installation. When the setup is run in silent mode via deployment systems, a modal dialog can show up and block the completion of the setup, and there will be no user to dismiss the dialog of course. You can use the property UILevel to determine if the setup is run silently, and then suppress the display of your dialog - but showing a dialog like this is just wrong design.

11. You try to "force overwrite" files with your MSI installer

MSI features some pretty complex "file versioning rules" designed to minimize the impact of "DLL Hell". They typically cause files to not be overwritten as intended - a classic MSI issue. As a result, people feel they can't find a reliable way to always force overwrite files on disk during installation.

There are ways to force overwrite files, but not in the way that most people picture as logical. Frankly the file replacement design is often frowned upon even when understood.
Overwriting files works quite differently for versioned files and data files (text, images, anything without a version property). In essence higher versioned files overwrite lower version files when the files are versioned. Data files are not replaced if the create and modify dates are different for the file in question. It has then been modified since installed.
The file overwrite behavior can be slightly tweaked by custom settings for the the REINSTALLMODE property set at msiexec.exe command line level (overwrite older versions, overwrite equal versions, overwrite any version etc...). Setting the REINSTALLMODE property changes file replacement logic for all files in the whole setup - including files deployed with merge modules that might target files in shared locations. You could hence downgrade shared files and components - exactly what "DLL Hell" was about.
Nevertheless it is crucial to understand the "file overwrite rules" and how they can be affected by settings for but it is a setting that applies to all files in the whole install. There are also some "hacks" to overwrite specific files only.
Check this article for how you can force overwrite a file that won't upgrade.
This section is not finished yet.

12. You install services that run with user credentials

In my opinion this just isn't good practice, and typically people wipe out the credentials during major upgrade scenarios as well - and in some cases also settings files that the service uses.

To me this is a prime example of how application design changes are needed to make deployment reliable and sane.
In my experience people insist on using these solutions and end up with a lot of custom action hacking to get it sort of working.
Save yourself a lot of trouble and design your service to run as LocalSystem (or maybe better - another account that is intended for service use - do have a quick read of this linked content and talk to your development team about options. Here is another post that might be worth a skim: Is it safe to run a pool under NT AUTHORITY\NETWORK SERVICE?).
See next section on NT privileges for a common problem seen when using user credentials to run a service.
UPDATE: the newer concept of managed service accounts should also be mentioned. Step-by-step (also see section in this answer on managed and group service accounts).

13. Your application requires extensive, custom NT privileges

NT privileges are different from discretionary access control (the access control of file system and registry objects) and include things such as SeServiceLogonRight "logon as a service" (which must be set for any user account trying to run a service - a very common setup issue for setups trying to run services with user credentials).

In some cases a plethora of such privileges are required to run an application or more likely a service. A very strong "deployment smell" or actually a "solution smell" - an anti-pattern if ever there was one.

Just about all these privileges are dangerous to squander around.

I suppose SeSystemtimePrivilege - setting the system time isn't too critical - at least at face value, but I don't really see any totally harmless privileges, and apart from the above mentioned service logon right, few should ever be needed as well.
In my experience the privileges asked for tend to revolve around "Logon User Rights". SeNetworkLogonRight (access computer from the network), SeInteractiveLogonRight (log on locally), SeBatchLogonRight (log on as a batch job) and the big one: SeServiceLogonRight (log on as a service).
Certain NT privileges such as SeAssignPrimaryTokenPrivilege, SeBackupPrivilege, SeDebugPrivilege, SeIncreaseQuotaPrivilege, SeTchPrivilege (act as part of the operating system) and several others should never be applied by any sane package.
The LocalSystem account intended to run services has most privileges (including dangerous ones) and should be used to run your solution rather than creating a separate user account and assigning these privileges to it. Seriously.
Here is a nice "grouped list of NT privileges" which provides some more context for understanding what each privilege is for and how they are related.

14. You apply a lot of custom disk and registry permissioning

This is a definite "deployment smell" or deployment "anti-pattern". In almost all cases this can be avoided by redesigning the application in question.

Applying custom permissions has traditionally been done using various command line tools. There are also built-in features in MSI to do this, but they lacked flexibility.
With the advent of WiX, applying permissions is now relatively reliable because it is a properly tested solution made by developers who understand MSI. Commercial tools also support custom permissioning of course.
In my opinion custom permissions is still a sign that something is wrong with the software you are installing, but I have applied a lot of custom permissions myself as well.
I have often seen repetitive self-repair problems caused by faulty permissions applied to disk or registry: How do I avoid triggering MSI self-repair with my WiX / MSI package? (section 5).
I have also seen several cases where erroneous permissions applied creates a situation where uninstall becomes impossible without some serious tweaking to the failed ACL permissions. Very gnarly work, and very easy to make worse by trying to deploy and automated fix.
Another obvious problem is the security risk you introduce by opening up write access to per-machine locations on the machine.

15. Your license key in the registry is reset on upgrade

A very common design is to write a license key to the registry using an MSI component. This can be either HKCU or more often HKLM - in order to make it a shared license for all users on the same machine.

If you use an MSI public property to set this license key, you should read this value back on a fresh install to ensure that you don't overwrite existing data there with an empty string. MSI public properties are (amazingly) not persisted and automatically read back by your upgrade setup in major upgrade scenarios. Forgetting to do this is a very common cause of people seeing their license key wiped out during major upgrade.

I rarely, if ever, recommend read/write custom actions. They are error prone and can be complex to get right - and most people never implement proper rollback (if the setup crashes and needs to roll back). However, you also have more power to check the system's "current state" with a custom action, and you can condition your custom action so it always runs, even during the patching sequence, and you can have it do different things during different sequences if you need to. Most of the time it can actually be problem that custom actions run when not intended - for example during a patch installation. Few people remember to condition their custom action with NOT PATCH (to prevent running during patching).

Despite all this I just might use a custom action to write a license key to HKLM during installation if I am instructed to write the license during the setup. However, and this is important, I would much rather remove the whole licensing issue from the setup altogether, for a lot of reasons described here: Installer with Online Registration for Windows Application (recommended read - there are a lot of reasons to keep licensing out of your setup).

16. Undesirable Hard Coded GUIDs

Some GUIDs can be hard coded in your WiX source file (or other MSI creation tool). For example component GUIDs - they should remain stable for each component, unless you change the installation location. The rationale for this is attempted explained here: Change my component GUID in wix?

However, don't hard code the Package Code. An MSI's Package Code should always be auto-generated for every build. It is simply supposed to be unique. In more detail; the idea of a package GUID is that it should be unique for each compiled MSI file. It is simply there to uniquely identify a file. Two different MSI files with the same package GUID will be treated by Windows Installer as the same file by definition. All kinds of X-files problems result. Accordingly a package GUID should always be auto-generated since it is simply supposed to be unique.

Many also auto-generate the Product Code - because they only use major upgrades to upgrade their applications. For this use-case auto-generated Product Codes work just fine. However, if you need to support Windows Installer minor upgrades as well, you should hard code your Product Code and update it when appropriate. The Upgrade Code should generally be hard-coded and manually managed. See this answer.

17. Erronous Inclusion of Sensitive Data

There is now a separate Q/A on the topic of preventing sensitive data from ending up in your final installer: How do I avoid distributing sensitive information in my MSI by accident?

Essentially the advice is to give your files a once-over for hard coded dev-box sins. How to check? I don't get fancy about it, open the MSI with Orca - and just skim through the tables. Most vulnerable tables are probably: Registry, Property, IniFile, maybe Directory, and if you use the MSI GUI: all tables relating to GUI. Any scripts (CustomAction table or Binary table - the latter requiring you to stream out any scripts - or check them in their source locations).

Links:

WIX does not uninstall older version (common major upgrade problems).
How to avoid having two versions of a product installed with Windows Installer / MSI?