Debugging InstallScript Custom Action in Basic MSI

[AndrewRich]

I have an InstallScript Custom Action in a Basic MSI setup. I know the CA is called when setup.exe is run (it's a very simple CA that just shows a MessageBox). I am attempting to debug the CA on a remote machine.

I have copied ISDbg.exe and SciLexer.dll to the remote machine, which is running Windows 7-64. I ran ISDbg /regserver. I verified that the InstallScript Debugger was correctly registered by remotely debugging a pure InstallScript setup.

When I try to debug the InstallScript CA in my Basic MSI setup, the InstallScript debugger never launches. The CA runs and then the setup continues. I suspect the problem lies in the command-line I am passing to setup.exe. Information from the InstallShield help ("Setup.exe and Update.exe Command-Line Parameters" topic) suggests there are two ways to debug InstallScript CAs:

/v"ISSCRIPTCMDLINE=\" \"" : pass arguments that should be passed to the script

Basic MSI projects that have InstallScript custom actions

This option specifies command-line parameters to be passed to the script. Any property supported by InstallScript MSI (where appropriate) can be specified. (The most common ones are /d and /z.)

For example, the following indicates that you want to debug the script, and that the CMDLINE variable should contain TEST.

Setup.exe /v"ISSCRIPTCMDLINE=\"-d -zTEST\""

Note that as shown above, when you want to specify that a double quote character is not a delimiter for the command line but a delimiter for the property, use \".

Note also that as with any public Windows Installer property, this property should be specified with all uppercase letters.

I tried:

"\\vmware-host\Shared Folders\Downloads\setup.exe" /v"ISSCRIPTCMDLINE=\"-d\\WEBOS-IS-DEV\ISProjects\PreReqs\Script Files\""

/d : debug InstallScript

Basic MSI projects with InstallScript custom actions,
InstallScript,
InstallScript MSI

For an InstallScript project, running the command Setup.exe /d runs the installation program with the InstallScript Debugger.


Debugging InstallScript code requires the debug-information file Setup.dbg to be available. To debug an InstallScript project on a system other than the development system:

Copy the InstallScript Debugger executable file ISDbg.exe (located in the System folder of your InstallShield distribution) to the test system, and register it by launching the executable with the /REGSERVER command-line option.
Copy Setup.dbg to the test system.
Run Setup.exe with the command Setup.exe /d"", where path is the directory containing Setup.dbg.
For a Basic MSI project, the following command runs your InstallScript custom actions in the InstallScript Debugger:

Setup.exe /v"ISSCRIPTDEBUG=1 ISSCRIPTDEBUGPATH=\"path-to-Setup.dbg\""

I copied the entire project folder to the remote machine and, from the DISK1 directory where setup.exe was built, tried:

setup.exe /v"ISSCRIPTDEBUG=1 ISSCRIPTDEBUGPATH=\"C:\Users\Andrew Rich\Downloads\PreReqs\Script Files\""

In both cases, setup.exe launched and the InstallScript CA executed but the debugger never fired.

What is the really correct way to debug InstallScript CAs in a Basic MSI setup?

[TsungH]

Try the following.

setup.exe /v"ISSCRIPTDEBUG=1 ISSCRIPTDEBUGPATH=\"C:\\Users\\Andrew Rich\\Downloads\\PreReqs\\Script Files\""

[AndrewRich]

TsungH wrote:
Try the following.

setup.exe /v"ISSCRIPTDEBUG=1 ISSCRIPTDEBUGPATH=\"C:\\Users\\Andrew Rich\\Downloads\\PreReqs\\Script Files\""

I tried doubling the backslashes (as you showed) but still don't get the debugger.

[TsungH]

Okay, I got time to try it out myself. Turned out double backslashes are not necessary, in my tests on Win7-32 (physical machine) and WinSrv2k8 R2 64-bit (VM).

C:\Windows\system32>C:\Users\testuser\Desktop\ISDbg.exe /REGSERVER
C:\Windows\system32>C:\Users\testuser\Desktop\DISK1\setup.exe /v"ISSCRIPTDEBUG=1 ISSCRIPTDEBUGPATH=C:\Users\testuser\Desktop\DISK1"

In my test using a Basic MSI project with InstallScript CA on Win7-32 (logged on locally), I was able to step into InstallScript CA code using ISDbg.exe without InstallShield 2011 installed. However, setup.dbg (and setup.rul too, or ISDbg.exe will prompt) needs to be in the folder specified by ISSCRIPTDEBUGPATH property. Otherwise, install will continue normally.

On WinSrv2K8 R2 64-bit (via Remote Desktop), without setup.dbg, ISDbg.exe will crash on me.

setup.dbg does not have to be in the same folder as setup.exe. They are in my case, because it is convenient to move them around.

TsungH wrote:
Try the following.

setup.exe /v"ISSCRIPTDEBUG=1 ISSCRIPTDEBUGPATH=\"C:\\Users\\Andrew Rich\\Downloads\\PreReqs\\Script Files\""

[manomatt]

thankyou for the great post. I wonder if this method of debugging will work on a CustomAction which are deferred? for immediateCustomActions it was working for me.

thanks in advance

[DebbieL]

For debugging deferred custom actions, see this KB article:
http://kb.flexerasoftware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=Q112187

[manomatt]

Hi Debbie,

Thank you for pulling that post. But i am not able to launch the IS debugger even after applying the registry settings. The Setup is developed in IS 2012 Spring. When i did the registry change in my development machine where InstallShield2012 Spring is installed i was able to launch the debugger successfully.

The test environment does not have IS installed. I am able get the IS debugger to launch for immediate CAs.