How to make better use of MSI files

As you might know, msiexec is a command line application that you can use to install an MSI file. As you might know, you can run it in silent or invisible mode.

If the installer requires the user to answer specific questions about what parts to install, is there some way that I can put in the msiexec command line a series of options to do this?

I figure there must be some sort of way of setting the MSI file's default settings to make this happen. How are MSI files made? Are they developed through tools from Microsoft? Can they be opened and edited?

Solution 1:

MSI files are designed specifically to support silent installation as a built-in feature - you can always skip the GUI. However, some MSI files have design flaws that render the install incomplete in silent mode - which is a serious design error. This problem is described here:

• : Uninstall from Control Panel is different from Remove from .msi (Silent vs Interactive Installation - technicalities and practicalities)

• There is also a section on it in this general description of common MSI problems seen in the real world (Common MSI Problems - general problems and anti-patterns)

Short Version: How to parameterize msi file from electron builder - using PUBLIC PROPERTIES and transforms to customize an installation of an MSI package.

• Here is sort of an escalation model of how I deal with bad or difficult deployment packages
• Itninja.com Software Packaging Tips Library
• What is the best tool kit to transform .msi into .exe?

Configuring MSI Installations

When it comes to installing an MSI silently, what you need to do is to configure the setup either from the msiexec.exe command line, or by applying what is referred to as a transform to the original MSI file. Both these options are described below in separate sections.

If the MSI file is well-designed you will be able to set PUBLIC PROPERTIES (they are always UPPERCASE) from the msiexec.exe command line or by using a transform file to modify the original MSI. These operations are described below. Public properties are easiest to find in the MSI file's "Property table". Use the MSI tool of your choice to open the *.msi file and navigate to the Property table. There are also some free MSI tools you can use to generate transforms and view (and edit) MSI files: How can I compare the content of two (or more) MSI files? (links towards bottom).

Well-designed MSI setups are fully configurable via these public properties. Badly designed MSI files are not. Badly designed MSI files are best to tweak using transform files (which can make substantial changes to the whole MSI file to apply at install time). Setting public properties can only change whatever is configurable by public properties - as designed by the setup creator. Transforms can change almost anything in the whole MSI file.

In general, all corporate, silent deployment is done using transforms to "beat MSI files into shape" for the corporate standard. It is a very effective tool for corporate deployment and is extensively used.

A couple of links for safe-keeping:

• Can the resulting command line be determined after running an .msi installer?
• How to build MSI package on a linux server?
• Microsoft Installer command line string parameter not working?
• Change Program Name and Shortcut Title during installation

MSI "Features"

MSI is often counterintuitive and somewhat complicated under the hood. However, to over-simplify an MSI file contains one or more "Features" - and these features collectively constitute the "bits of the application" as you put it. Features, in turn, consist of "Components" - which are the atomic units of installation for the whole software - but this is a very technical detail - this answer is about the user-exposed bits of MSI - the features.

Screen Shots: What Features look like in a real MSI package (screen shots).

You can generally find a list of these features by running the setup interactively, and navigate to the customize install dialog (not always present). Features that show up here are the "user configurable" parts of the application that can be chosen for exclusion or inclusion (some are mandatory). You can also find these features by opening an MSI with a capable tool as mentioned above (you can also see links in section 2 below).

Typical features are: Core or Program, Dictionaries, Samples, Plug-Ins, Spell Checker, SDK & Developer Tools (for dev tools), etc... Some features are mandatory (must be installed) - examples above would be Core and Program, others are optional and are not needed for the application to launch (like the dev tools features above). It is possible to make the application install features "on demand" - for example spell checkers when the user initiates a spell check.

In my experience most users want the whole application installed. Many users are very annoyed if Windows Installer pops up unexpectedly and starts installing spell checker components. Frankly very understandable. However, rarely used modular components interesting only to a few users could be made into optional components - especially if system administrators may not want the feature available on their network. This is certainly the case for developer tools - these should not be available to regular users. They tend to be all the rope people need to shoot themselves in the foot.

As mentioned above, there are generally two ways to customize an MSI installation: (1) using msiexec.exe custom command lines, or using (2) transform files.

1: msiexec.exe command line:

The simplest and light-weight way of controlling what features are installed during an installation, is to specify your feature selection using the msiexec.exe command line. There is a whole family of properties used for feature configuration. But, in most cases it is sufficient to specify ADDLOCAL:

msiexec.exe /i myinstaller.msi ADDLOCAL="Program,Dictionaries" /qn

The above command line specifies that the features "Program" and "Dictionaries" should be installed locally (feature names are cases-sensitive!). This is generally enough, but you can also specify any features that you want to remove using the REMOVE property in a similar fashion. A special switch is ADDLOCAL=ALL which will install all features in the MSI on the local disk (provided there is not additional logic in the MSI to override this). ADDLOCAL property on MSDN.

A very common thing to define by public properties is the license key for the application. The following command line specifies to install the features "Program" and "Dictionaries" and to apply the serial key "1234-1234":

msiexec.exe /i myinstaller.msi ADDLOCAL="Program,Dictionaries" SERIALKEY="1234-1234" /qn

As is implied in the description above, the list of customizable properties for each setup is always different. You can find most properties listed in the MSI file's Property table, but it is also possible that some properties can be set that are not defined in the Property table. In most cases this relates to properties being set only from the setup GUI (indicates a setup design error in most cases). All properties should be defined in the property table in a properly authored package.

Look for documentation on the vendor's download page and ask their support for any documents relating to silent installation or large scale deployment. It is quick to do, and answers can be quick if they have standard answer templates. Companies with control of their deployment will always be able to provide this. In my view the ideal way is a one-page PDF which describes different deployment settings. Frankly, give them some heat if they can't provide this ;-).

2: Transforms:

MSI files are essentially SQL-databases wrapped in COM structured storage files (file system within a file). Transform files are "partial databases" constructed via installation tools such as Orca (SDK link), Installshield or Wise, Advanced Installer, etc... (link to descriptions of the different tools). These transforms can customize or override almost all settings or database fields in an MSI - including what "parts of the application" (features) are installed. After creating a transform you specify its application to the MSI at the msiexec.exe command line:

msiexec.exe /i myinstaller.msi TRANSFORMS="mytransform.mst" /qn

Windows Installer will then merge the MSI and the transform before installation starts. This is the approach used by large organizations who want full control of how the MSI gets installed. TRANSFORMS property on MSDN.

As mentioned above this is the option that allows all settings in an MSI to be modified. Substantial fixes can be applied to badly designed MSI files to allow reliable deployment. This is done by "application packagers". Their job is to tune all setups to work within the corporate standard. They can be among the most knowledgeable MSI specialists around - they see a lot of weird stuff in MSI files.

Many tools can be used to create a transform, here is a description of such tools inside the more technical context of comparing MSI files. Just jump straight to the list of free tools at the bottom: How can I compare the content of two (or more) MSI files?

Anti-Patterns Vs The Corporate Benefits of Windows Installer:

Windows Installer has many design quirks and may be particularly annoying for developers. Admittedly there are some issues that border on anti-patterns.

Potential Anti-Patterns

• Difficult multi-instance installations
  • Relatively common requirement, especially for service installations
• counter-intuitive file overwrite rules (symantec)
  • strange rules, especially for non-versioned files
  • an insane feature to force overwrite all files (REINSTALLMODE = amus)
    • can downgrade shared files system-wide
    • can cause inconsistent version estate since an old package can be installed after a newer one and downgrade only some of the shared files
    • can downgrade or wipe-out settings in non-versioned files (and registry settings)
    • can cause a significant increase in the number of requested reboots due to attempts to needlessly replace in-use files of the same version.
    • there are several further issues that are quite specific. One day I will write them all up
• unexpected reset user data in the registry after upgrades
  • This is extremely problematic. If you experience this it is not you, it is the technology
  • Often seen with service credentials logins and serial keys
  • Some techniques to avoid this problem
    • avoid writing ANY HKCU registry keys from your setup, write them from your application instead. Your setup will now never interfere with them - it has no knowledge of the values at all.
    • putting registry data in a feature of its own (should prevent problems on self-repair)
    • install registry data via a component with empty component GUID (will then not ever be rewritten during repair or self-repair)
    • set component flag to never overwrite if key-path exists.
    • write HKLM data (such as license keys) to the registry using a custom action instead (this has other problems, but will give you complete control of when data is written - in what installation mode)
    • make darn sure you keep a stable registry key path. Set a flag value KeyPath = 1 and never change it - and crucially - don't change the component GUID either
    • never set REINSTALLMODE to "amus" - certainly not hard code that value in the property table.
    • there are further tricks and rules of thumb, if only I could remember them all off the top of my head :-).
• complex upgrade mechanism
  • minor upgrades has a lot of limitations and restrictions
  • major upgrades have other challenges (reset registry data, missing files after install, self-repair for COM files after install, etc...)
• lackluster GUI features
  • not rocket science, but somewhat complex
  • lacking events and features to implement a properly smooth GUI
• shockingly complicated patching
  • extremely difficult to use effectively
  • not recommended for use except as a "hotfix" - i.e updating a couple of files or fixing a specific MSI file error in the installed setup's uninstall sequence.
  • Some patching comments:
    • Windows Installer "Error 1308. Source file not found" when uninstalling patch in sequence scenario
    • MSP vs. MSI packaging. Which to use for patches?
• extremely complicated implementation of custom actions
  • complex sequencing
  • complex conditioning
  • complex impersonation / partial running with elevated rights
  • overall extremely error prone.
• the lackluster implementation of per-user setups
  • conceptually dubious (folder redirects, unpredictability, impossibility of making setups in the real world support both per-user and per-machine installs)
  • complex to upgrade, uninstall and patch. Allows products to be installed multiple times for different users AND also per-machine
  • I have to admit - on a subjective note - that I deem the current implementation of per-user setup a full on deployment anti-pattern. I never use it and insist not to unless forced to.
• unexpected self-repair
  • 1) Self-repair - explained.
  • 2) Self-repair - finding real-world solutions.
  • 3) Self-repair - how to avoid it in your own package.
• the lack of built-in features to write to XML files
• poor features for IIS installs
  • part of the issue is the file overwrite rules for unversioned files (unpredictable results possible).
  • IIS may need a brand new deployment technology altogether to be honest - a way to define handling of non-versioned files in a totally predictable fashion - with sensible, real-world options. Perhaps auto-backup of force-replaced non-versioned files, enforcing of groups of consistent text files ("assemblies") that have to be correct version all of them, etc...
  • also several other issues with the complex configuration of IIS and virtual folders and sites
• sloppy enabling of "check exit code" on custom actions can cause packages that are not possible to upgrade or uninstall (without serious tweaking)
  • major upgrades may fail and trigger rollback for something insignificant
  • a minor upgrade can be used to fix the uninstall sequence or the faulty conditioning
• there are a few more...
  • In fact I wrote up a sprawling summary of commonly seen anti-patterns often seen in real-world MSI packages themselves (erroneous use of the technology): How do I avoid common design flaws in my WiX / MSI deployment solution?
  • I can stand by all the content, but the format is not great - it is a messy brain dump, but sometimes that seems to be the only way to get things done. Take it for what it is.
  • Overuse of custom actions is another MSI problem. There is a core of complexity behind this, but overall the problem is that people don't use fully functional pre-existing solutions in MSI or via extensions such as WiX (or commercial tools such as Installshield or Advanced Installer). Here is a summary: Why is it a good idea to limit the use of custom actions in my WiX / MSI setups?

The issue of high complexity of implementing custom actions (custom installation logic), could be argued to be unavoidable and the act of writing a custom action should be powerful and capable once needed - and hence complicated. Rarely should custom actions be needed if the technology itself offers what is commonly used for deployment. In other words you should use built-in MSI features rather than custom actions if they are available, or WiX or third party deployment software extension when available.

The WiX framework (open source) and commercial tools alike (Installshield, Advanced Installer, etc...) have implemented features to extend Windows Installer to deal with missing features such as the lack of an upgrade mechanism for XML files, share creation and management, creation of users and groups, advanced IIS configuration, COM+ installations, changing ACL permissions, setting firewall rules, persisting installation properties, etc... There should be less and less need to implement your own custom actions. Always use the features that are already tested by thousands of other users if you can (millions of users even - and these extensions have been written by the best deployment experts available - do you think you can do it better on your own?).

The Corporate Benefits of Windows Installer (very significant)

It requires a specific mindset to approach Windows Installer. However, it provides a number of crucial corporate benefits that were almost entirely lacking in previous installation technologies. The corporate benefits of using MSI files is recommended reading. Particularly for those who think Windows Installer is more trouble than it is worth.

To summarize the linked article in brief, the core corporate benefits of MSI over previous deployment technologies are (in my opinion):

• the reliable silent running (with standardized, completely suppressible GUI)
• the implicitly available uninstall (a nightmare with older deployment tecnologies)
• the verbose logging (can be helpful, though really verbose indeed)
• the reliable remote management (in effect the overall benefit altogether - the combined effect of all other listed benefits of sorts)
• the elevated install rights (no messy temporary admin rights)
• the standardized command line (a hugely beneficial feature - no more chasing hidden command line options)
• the installer's semi-transparent nature (open format, except compiled CAs which are black box)
• the rollback support (computer state management, prevent partial deployments, fail and roll back changes)
• the admin install (essential for corporate repackaging, extracts all files in a standard way)
• the standard package customization approach (transforms) (basically allows complete customization for corporate deployment)

That's just to cherry pick the most important ones (after many years doing corporate deployment). In all honesty these features make all the difference in the world (for corporate deployment) and truly makes MSI great to use despite all its flaws.

The Twilight Years Of Windows Installer

As Windows Installer hits its twilight years, we can only hope that the deployment technologies of the future will preserve these great corporate deployment benefits and deal with the mentioned anti-patterns in a way that benefits everyone, and developers in particular.

Deployment is a crucial part of development. Failing to get your great software successfully installed for your potential end users may be the most expensive mistake to make in software development overall. How can you succeed if the user never sees your software fully functional?

Windows Installer's complexity must be handled better (reduced), and its crucial benefits must be preserved properly in whatever paradigm comes next.

A reasonably good: summary of Windows Installer.

Cloud Platforms

With all this said; as computing in general moves to cloud-platforms, the world of deployment is likely to change considerable in unpredictable ways. However, as the famous saying goes: the more things change, the more they stay the same. Deployment needs to deal with all legacy technology that will be in use in companies for decades to come. Here is a piece on why deployment seems to get more complicated and not less complicated - despite all the marketing: What is the benefit and real purpose of program installation?.

It will be interesting to see what the future of deployment will be - in the years to come. Perhaps we will see simplified deployment for home computers, and corporate deployment will become more complicated than ever? In the future most deployment will probably be a database deployment task more than a file and folder deployment task. Server deployment can be extremely complicated as of now with database scripts, user and group creation, share setup and ACL permissioning, performance counters, firewall rules updates, AD queries and updates, COM+ and message queue configuration, service installation, etc... - the whole nine yards.

Solution 2:

Think of the user interface with MSI as optional. This means no answers should be required as the developer has sensible defaults in place so that things don't break.

We distribute our software in MSI format to corporate customers, I also provide them with documentation on the basics of Orca (orca.msi is distributed with the Windows Installer SDK) and how to customize certain fields we have listed in the Property table for their installation. Such as serial number, registration details and a few other settings.

In response to the original question about msiexec command line options just run MSIEXEC /? to set properties on the command line you would use something like

MSIEXEC /I test.msi SOMEPROPERTY="Some value" PROP2="something else"

Solution 3:

How to configure silent MSI setup

An MSI installation can be configured on the command line by setting the properties that the installer uses. There are pre-defined Windows Installer properties such as the ALLUSERS property. This property defines whether an installation will be done in the context of the current user or the machine.

Information on the available properties can e.g. be obtained from an install log which can be created using msiexec's /l option

msiexec /I mysetup.msi /l*vx log.txt

How to create MSI files

There are many ways to create MSI files. An MSI file is basically a database consisting of various tables containing all necessary setup information and installation dialogs.

Microsoft offers a simple tool call Orca which enables you to edit existing MSI files and allows you to find out which properties can be set to configure an installation. Theoretically it is also possible to create new MSI files using this tool but it is a very cumbersome way to go.

If you are looking for a free and open source solution I would recommend you to have a look at the WiX toolset available on SourceForge or the Nullsoft . All setup information is done via XML files which are then converted into an MSI installer. WiX is stable (although still tagged beta) and can be used in production. Actually it will be integrated in the upcoming version of Visual Studio 2010.

Of course there are also commercial solutions available, InstallShield being the market leader (also being the price leader) and Visual Studio probably being the most wide-spread tool.