Do not override the existing file when we install the .exe file

How does your application work? Does it write settings to these XML files over time, and you hence want to preserve them? If so, where are they stored? In a per-machine or per-user location?

Let me try to describe some problems for you that you are likely to face – and you could not possibly foresee ahead of time (without testing).

Some details about file overwriting

1. File versioning rules: MSI has built-in support to not overwrite non-versioned files that have been modified since installation (the create and modify dates are different). These are “File Versioning Rules“. They are somewhat complicated and strange – you should definitely read the linked content – the real McCoy. So modified non-versioned files should not be replaced during updates – but there are many complications and bear-traps – and most people seem to fall into one of them.

2. Major upgrade file reversion: Perhaps the most common complication seen for “unexpected file overwrite” is not an overwrite at all. It relates to major upgrades – or installing your next product version if you like.

   • If you use a major upgrade to install your next version, and you schedule RemoveExistingProducts early in the InstallExecuteSequence, then all files are actually uninstalled and then reinstalled in their original version. They will appear to have been overwritten, but they have been uninstalled and then installed “fresh” instead.

   • This is all due to the very strange decision that MSI will happily uninstall modified files, even if they qualify to not be overwritten. What you need is a way to define the files to not be uninstalled at all. Which brings us to the next point:

3. Permanent and Never Overwrite Flags: Each component has two important settings relating to file overwrite and file preservation: the never overwrite flag and the permanent flag. This is indeed a good way to indicate that a file should not ever be replaced, nor uninstalled. However…

   • Setting only the never overwrite flag will not help against the major upgrade problem described above, the file is still uninstalled and then reinstalled so it appears overwritten (but is reverted).

   • If you set the permanent flag only, the file will be overwritten according to the File Version Rules – provided no modification is specified at the command line using features such as REINSTALLMODE (see section below). (I will need to verify this again to be 100% sure. There are many factors: component can be set permanent in older or newer setup only – or in both setups, major upgrade can uninstall early or late, REINSTALLMODE has many flavours, etc… Each variable can change the overwrite behavior)

   • In other words setting both the permanent and never overwrite flags is recommended to protect settings files.

4. REINSTALLMODE: REINSTALLMODE is a “modifier” or disabler of the File Overwrite / Versioning Rules (symantec). It is possible to set the REINSTALLMODE property during installation equal to amus – which will (among other things) force overwrite all files during installation regardless of the above mentioned file version rules.

   • Explanation of the amus REINSTALLMODE parameters:

     • a – Force all files to be reinstalled, regardless of version
     • m – Rewrite all registry keys that go to HKLM or HKCU
     • u – Rewrite all registry keys that go to HKCU or HKEY_USERS
     • s – Reinstall shortcuts and icons

   • One of these days I will write up issues with REINSTALLMODE and its implementation. It is an incredibly destructive concept in MSI, and it should never, ever be used. The problem is that this is a generic command line that can be passed to any MSI package and applies for the whole setup (why couldn’t it at least be per feature?). I want to make my packages abort if amus is specified. Let me jot down quickly a few of the issues:

     • Downgrade: Can downgrade shared files system-wide. The files can come from merge-modules so people might not realize they target system folders. Windows 10 protects its OS-files pretty well now, but there are other files that can still be downgraded without OS-protection.
     • Inconsistent Versions: Can cause inconsistent version estate since an old package can be installed after a newer one and downgrade only some of the shared files.
     • Settings Wipe-out: Can downgrade or wipe-out settings in non-versioned files (and registry settings).
     • Reboot Prompts: Can cause a significant increase in the number of requested reboots due to attempts to needlessly replace in-use files of the same version. There are features to avoid this, but they are often not used correctly. Restart Manager, more on restart manager.
     • And then there was more…: There are several further issues that are quite specific. One day I will write them all up. I think. Maybe not? :-).
       • Windows File Protection (WFP): Maybe I can add that certain legacy merge modules contain files that are now protected by the OS, and trying to replace them used to potentially trigger a runtime error. This is now handled by Windows 10 “silently”.
       • Registry: The m and u switches of amus will reset registry data written by components (and not custom action code) outright irrespective of the a switch (At least this was the case last time I tried). High on my MSI anti-pattern list (see section “Potential Anti-Patterns”).
       • Unknowns : GAC, WinSxS. I don’t even known how things work with the GAC? Side-by-side is un-downgradable by design? One would think. Unless you poke around inside the assembly folder directly – which hopefully will trigger a runtime error – which I have not tried. In fact I don’t think it will. See this issue with publisher configuration files (see all tweets in “thread”).

This is all slightly mad to be honest, but it is how the technology works. So, let us stop complaining and try to deal with the problem. What are the options to deal with this?

Some options to deal with settings file-preservation issues

In summary I suppose you have a couple of options:

1. Use minor upgrades to upgrade your application. Then your files are never prone to be uninstalled and reinstalled and hence be reverted like they are for major upgrades.

   • A minor upgrade does not uninstall the existing installation – it upgrades it directly. Regular file versioning rules apply (modified non-versioned files are not replaced) – unless REINSTALLMODE is used to modify this logic.

   • However, few people succeed with minor upgrades only – they are too restrictive and limited. Close to none I would say.

2. Use a major upgrade with RemoveExistingProducts early in the InstallExecuteSequence and permanent and never overwrite enabled for the components hosting your settings files. When using a major upgrade the main choice is between early and late uninstall of the old version, and this choice has huge implications.

   • This section needs some cleanup. Will happen soon…. December 2019: and we are back. Years later. Some minor cleanup happened today – not much else. I shall keep promising to clean this up more. Empty promises makes the world go round and round. If this all seems like complaining, please read the major benefits of MSI (quick list). MSI delivered crucially important improvements to previous deployment technologies, but has a few big challenges left.

   • Early uninstall means that any component referencing errors you have made shouldn’t cause any problems. They are more “forgiving” if you like. A lot of people end up using this approach specifically because is the most “forgiving” of component referencing errors and overall “robustness”.

   • Early uninstall also means that modified files can be uninstalled and reinstalled during the upgrade, causing them to appear overwritten, but they are reverted (described above).

   • A remedy for the above problem is to set the hosting components for files to be permanent and never overwrite.

   • Early uninstall means that the complex component creation rules can be broken and your upgrade will still work. It is very common to break the component creation rules. If you use this approach you should set the files you want to preserve during upgrades to permanent and throw in never overwrite too. This should prevent the problem where modified files get uninstalled and fresh files are installed – yielding the impression that the files have been overwritten.

3. Use a major upgrade with RemoveExistingProducts late in the InstallExecuteSequence.

   • Late uninstall major upgrade installations do not uninstall and reinstall files that exist in the same location in the old and the new setups – and this works even when the files in question are not set permanent and never overwrite (allowing proper uninstall of the files as well – without any added, custom logic).

   • This kind of upgrade basically install as a “patch” – leaving files that are unchanged between releases untouched, and then upgrading other files according to the File Versioning Rules described above.

   • Rather they will be overwritten based on the normal file versioning rules (provided you did not set the REINSTALLMODE equal to amus – in which case you downgrade everything and wipe out settings).

   • Previous answer: How to keep a config file when major upgrade in wix v3.8?.

4. Custom backup mechanisms for settings files: many end up implementing complex custom actions that back up settings files and put them back in place after the installation has run.

   • I don’t like this approach due to its potential for custom action failures, but many use this option. Often this is a feature added in version 2 of an application since the files in question were not set permanent and “never overwrite” in the setup for version 1.

   • Getting the sequencing, impersonation (user / system context you run in) and conditioning right for such custom actions is not at all trivial.

   • As a concrete example of the complexities involved, here is a previous question / answer where such a feature has been implemented and there are problems determining exactly what is going on: Wix Tools update uses old custom actions.

     • Complexity: Very complex conditioning is going on to make the backup and restore operations happen at the right time in different installation modes (fresh install, upgrade, repair, modify, etc…)

     • Condition Debugger Light (practical tip!): To work out the complicated conditions I like to go very concrete with real-world testing of the conditions instead of too much analysis (yes, you will fail anyway – somehow).

       • Here is the overall approach I use: VBScript condition tester. It is in essence a dialog box approach from a custom action that is conditioned – it shows in real life WHEN the custom action runs without any logging or complex debugging.

       • Just run the setup in different modes and see if you see the dialog: install, uninstall, modify,
         repair, patching, major upgrade, suspended-resume…

5. Eliminate settings files from deployment: this is my favorite option, although it may sound a bit strange. Anything that can make your setup “dumber” and less complex will make it more reliable in actual use.

   • Essentially files that are to be changed after installation should not be installed at all, but generated by the application or copied from template files to the location where they are to be changed.

     • In these scenarios no deployment issues will ever interfere with your precious settings files – unless you make a weird custom action to do so (which you shouldn’t).

     • Your setup has no knowledge that these settings files exist, and they will hence always be left alone. Which is very often what you want.

     • You can make your own cleanup feature to delete them on uninstall, although my advice would be to leave them alone: they are user data.

   • I write about the general problems related to per-user and settings file deployment in this answer: Create folder and file on Current user profile, from Admin Profile. Please have a quick skim at least, to see if you can improve your deployment by killing complexity rather than embracing it.

     • Essentially there are various ways to deploy settings and user files in a setup (Active Setup, self-repair, etc…).

     • Then there are template files that you can “instantiate” and keep out of your setups (no interference as described above).

     • You can use application internal defaults instead of settings files in some cases – eliminating all deployment considerations.

     • And finally you can retrieve settings from the network (database) instead of using old-style settings files.

Practical Advice: If you can, do use a database for your settings or some form of clouded approach to get settings files OUT of your setup. Deploy your
binaries only! And the templates needed to make settings files or registry settings on launch. That will help tremendously to simplify your setup.

---

Links:

• The other side of it: files not overwriting properly