“Short” Answer
Essentially: you could
1)run completely silently (suppresses the files-in-use dialog),2)shut down locking
applications gracefully (application update to allow graceful shutdown – with or without restart manager support),3)ensure proper service control (if dealing with services),4)force-kill
running processes (the “sledgehammer-approach”),5)abort setup if locks are detected,6)require logoff before deployment,7)install to a new folder for each version (side-by-side install), etc…
Below is a little drill-down of files-in-use issues and the Restart Manager – intended as a quick review for files-in-use and reboot issues.
In terms of your actual problem. I wouldn’t mess with the FileInUse dialog(s). It won’t really solve your problem. Maybe consider these pointers:
- Services: If you are installing services and they trigger files-in-use issues, please see section on services towards the bottom to determine if you can improve your setup’s logic.
- Silent Mode: Running your setup in silent mode would be an obvious way to suppress such files-in-use dialogs, but then you have to suppress automatic reboot, or the system will spontaneously reboot without warning. Details below.
- Policy: Please check if the DisableAutomaticApplicationShutdown policy is enabled on your box / standard PC configuration. See details below.
- Registry location is:
HKLM\Software\Policies\Microsoft\Windows\Installer. - I am not sure if enabling this policy will make the files-in-use dialogs disappear.
- Registry location is:
- Restart Manager Compliance: Maybe check if you should update your application to heed the design of the Restart Manager feature – to allow for auto-magic and problem free upgrades by applications shutting themselves down gracefully (provided you are dealing with binaries that you can actually change yourself – in other words: you have the source code). Lots of details below.
- “Setup Overkill”: If you deem it safe to kill your application without mercy during upgrades, see section on this below.
- Graceful Shutdown Custom Action: If you make your application capable of graceful shutdown (restart manager-style), then you can trigger such a shutdown yourself as well (easiest for user context processes) via an immediate mode custom action (in case Restart Manager is disabled by policy – look out for timing and timeout issues though – especially for silent running – “deadlock”).
- Side-By-Side Installation: some details below. Some companies decide to install applications truly side-by-side so there are no file overwrite issues with their new deployments (old versions uninstall may still trigger required reboots though).
I suppose you could also abort the install if locked files are detected, or you could require users to log off before the installation is run – if you have a distribution system.
Please at least skim the rest of the answer for more details and context.
Restart Manager
Your applications and services should be prepared to be shut down by the Restart Manager and save user data and state information that are needed for a clean restart. This requires updates and changes to the application / service to adhere to standards for shutdown and restart of the application.
The Restart Manager: is a new C-style API available beginning with Windows Vista and Windows Server 2008. Restart Manager consists of a single DLL that applications can load to access the Restart Manager API. The idea is that the Restart Manager will auto-magically shut down and restart your applications during installations / updates, by having the application / service follow a set of guidelines:
- Guidelines for Applications – (
the crucial tech-read) - Guidelines for Services
In essence:
The whole idea is basically to prefer restarting applications rather than restarting the OS - and also to avoid reboots in general.. To that end:1)
Your application callsRegisterApplicationRestart()with a command
line specified for its eventual restart – it “signs up” for restart
management.2)Your application watches forWM_QUERYENDSESSION
messages and shuts down gracefully saving data in an appropriate way
when told to do so.3)Then Restart Manager can restart the
application when finished installing (restart can be disabled).
More Technical stuff:
- Here is a nice Symantec article on “Files In Use & The Restart Manager”.
- How do I add support for Windows Restart Manager to my application? (Advanced Installer)
Restart Manager Configuration: There are a number of properties that will affect how the Restart Manager will operate with Windows Installer:
- MSIRESTARTMANAGERCONTROL – use the Restart Manager or FileInUse Dialog
- MSIDISABLERMRESTART – restart or don’t restart shut down applications
- MSIRMSHUTDOWN – shutdown, force shutdown or do not force shutdown
When Restart Manager is used, the MsiRMFilesInUse dialog is used instead of the FileInUse dialog to show a list of applications that have locked files.
N.B! The whole Restart Manager feature can also be disabled by policy:
- DisableAutomaticApplicationShutdown policy.
- Check in
HKLM\Software\Policies\Microsoft\Windows\Installer. - Restart Manager behavior with Windows installer
FileInUse
If you don’t have the time or resources to implement proper interoperability with the Restart Manager (which is frankly the only sane thing to spend your resources on at this point in Windows’s development), then there are a few things that might be good to know:
- Silent Install: The first obvious thing to point out is that there will be no
FileInUsedialog if you install the setup in silent mode. However, this could trigger a system reboot unless you specify theREBOOT=ReallySuppress property. - Services: Do you install services that you do not properly shut down during upgrade? There are
built-in MSI constructstoshut down servicesduring upgrades – the Service Control table.- Used properly, this ServiceControl feature means you no longer have any problems with service executables triggering a reboot to be replaced (barring shutdown problems in the service itself).
- This is a built-in MSI constructs and works well when used right. People should not resort to custom actions to install services.
- Application Support: Beyond interoperability with the Restart Manager, some application – that have files in use – can shut down gracefully when told to do so.
- Some applications shut down properly when sent a command line, for example
App.exe -shutdown, despite not having been written to be interoperable with the Restart Manager. Maybe system tray applications that save no data for the user? - This obviously has to be implemented specifically for the application in question – and if you do that, you should use the Restart Manager instead at this point (or additionally, you could have both call the same actual shutdown implementation).
- Some applications shut down properly when sent a command line, for example
- “Setup Overkill”: some setups are designed to just kill application processes that are open at the time of installation.
- Not ideal, but it can work for certain types of applications that run in the background (OK, it is nuts, but it is done regularly).
- Use a custom action or whatever built-in constructs are available in your deployment tool.
WiX: Kill windows service forcefully in WIX. WiX: CloseApplication.VBScript: Close an application using VBScript.Advanced Installer: How to detect or stop a process.Installshield: Kill process. Kill process documentation.
- REINSTALLMODE: Do you perhaps use
REINSTALLMODE="amus"to force overwrite files during installation?- This can dramatically increase the amount of files in-use and reboot prompts since all files are attempted replaced – and generally unnecessarily so – especially in
repairandmodifyscenarios. - This is particularly true for setups that install services that do not use the ServiceControl table properly to shut down the service before trying to overwrite its binary.
- This can dramatically increase the amount of files in-use and reboot prompts since all files are attempted replaced – and generally unnecessarily so – especially in
- Side-By-Side Installations (SO): adding this for reference, it is beyond the scope of what is “normally relevant”. This approach requires quite a bit of technical changes and proper distribution processes to be successful – it is primarily for in-house, core corporate applications (full app control possible) – in my opinion.
- New versions, targeting new installation folders (add version number to installation folder?), can generally install without any file overwrite issues (unless any system-shared files are updated – in which case you should split them to a separate pre-requisite MSI – with its own distribution logic – when required – which should be rarely).
- Older version uninstall can still trigger reboot requirements as the files can be in-use and not ready to be uninstalled. Obviously.
- You can use automatic GUIDs for the setup components – so MSI can keep them tracked separately in a correct fashion. You must generally eliminate all need to set static components (or they must be installed to shared locations and kept static – or updated via a separate pre-requisite MSI when required).
- The whole application must be “well behaved” for side-by-side use and installation. In other words not fight over file associations and load all resources properly and manage database connections that could be shared between instances, etc…
- You add the version number to the start menu shortcut? Somehow you must be able to differentiate installations and launch the desired version – obviously. The application should be aware of its doppelgängers?
- I might consider setting a new upgrade code for each release, in order to decouple the products from one another, and then use the distribution system to uninstall older, legacy versions (as a weekend or monthly batch job?). This is not 100% necessary, it all depends on your scenario. Lots of things can work when planned coherently – obviously.
- Applications not suitable for normal side-by-side operation may sometimes be virtualized and sandboxed using App-V (virtual packages) to allow different versions to co-exist on the same box. New challenges.
Some Further Links:
- FileInUse or FilesInUse (lots of inside information from a Symantec employee towards bottom)
- Terminating process in CustomAction
- Restart Manager behavior with Windows installer
- How can I stop an .exe on repair, update and delete in wix?
- RmGetList() API fails when file is locked using Ez file locker but works with another file locking utility
- Suppress an MSI error code
- Wix stop service on uninstall/upgrade: prevent “restart popup” (file-in-use situation)