cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Mystique
Level 5

Debugging Managed Code Custom Action

Can we debug managed code custom action that is integrated into an InstallShield 2009 project? The project typoe that I am working on is InstallScript MSI project. There is no problem for me to do so with C++ custom dll, but I was unable to step through the code with managed dll. What is the difference between the two? Is there a knowledge base article which describes this?

Thanks for any help on this.
Labels (1)
0 Kudos
(11) Replies
MichaelU
Level 12 Flexeran
Level 12 Flexeran

You might be able to do this by adding a message box and attaching to the custom action process (or using the MsiBreak environment variable), being sure to attach with managed code debugging support, and then using Debugger.Break, or similar. But if that doesn't work, I don't have any better ideas. We don't yet have a good debugging story for managed code custom actions, sorry.
0 Kudos
Mystique
Level 5

Thanks for your quick response. There is no problem to debug managed code dll in a regular C# application. But to debug managed code custom dll that is launched by InstallShield 2009 is somehow different. I did put a MessageBox there and had execution stopped for me to attach the Visual Studio debugger to the process. The process that I was attaching to is msiexec which shows the MessageBox. Somehow, break points in my Visual Studio are grayed out and it looks like that debug symbol is not loaded. Therefore I was unable to step through the code.

If this does not work, I can only put in code such as log file in order to debug managed code custom dll.

Did I miss anything?

Any help on this is appreciated.
0 Kudos
MichaelU
Level 12 Flexeran
Level 12 Flexeran

Let me clarify my comment "being sure to attach with managed code debugging support":

When you attach to a process with Visual Studio 2008 (and probably other versions), the dialog where you select the process has an option Attach to: which probably will say Automatic: Native code for msiexec.exe. Click the Select... button, and make sure to select manual and check the box for Managed. If that doesn't do it, I think it will take code changes in our host dll for the managed code.
0 Kudos
Mystique
Level 5

Thanks for your reply. Yes, I did try both and it was not working.
0 Kudos
Christopher_Pai
Level 16

You'll be much better off using WiX's DTF. There is an MMsiBreak environment variable and you can attach a debugger using a message box or otherwise just assert an error and attach.
0 Kudos
Mystique
Level 5

I am still unable to attach a debugger to managed code custom action. After the code pause for a message box, the break points that I set are still grayed out.

Please advise. Thanks.
0 Kudos
Christopher_Pai
Level 16

Are you attaching to the rundll process?
0 Kudos
sjeslis
Level 5

Christopher Painter wrote:
You'll be much better off using WiX's DTF. There is an MMsiBreak environment variable and you can attach a debugger using a message box or otherwise just assert an error and attach.


Does the CA DLL included in Installshield need to be the debug version in order to breakpoint? Seemed I couldn't get MMsiBreak to work in VS 2008 with the "Release" version of my DLL.:confused:
0 Kudos
Christopher_Pai
Level 16

It depends on how you build. Personally I build my CA's in release without debug information and all available compiler optimizations but then I also build the debug configuration. If I ever need to debug a situation, I just stream the debug dll into the binary table and attach a debugger.

Here's a good thread to read if you want to ship PDB's with your CA.

http://n2.nabble.com/DTF-and-PDB-files-for-better-error-logging-td3227229.html
0 Kudos
Mystique
Level 5

I am still having the same problem. My install is running on a remote machine and I started a remote debugging monitor on the remote machine. I could step through the code if running a regular c# application. But if the dll is launched by InstallShield, I have trouble step through the code. It looks like I was not attaching to the right process. Can you give more details as to how to debugg managed code custom action dll?

Thanks.
0 Kudos
Christopher_Pai
Level 16

Which process are you trying to attach to? In this case msiexec is not correct. You want to look for the rundll process that shows as running managed and unmanaged code.
0 Kudos