cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
Videstra
Level 7

Managed Custom Action

I have created a small DLL that I am using to see if a certain port is open on the network. It simply checks if the port is open and returns either:

IDABORT {3}
or
IDSUCCESS {1}

When I put the DLL into my project it runs just fine, but it's not clear how I get the return code (IDABORT) to actually abort the install.

I have it set to "Synchronous (Check exit code)"
Immediate Execution
Always Execute

I have the return value going to a property I created called "PortIsOpen"

Is there a specific property I should select for InstallShield to abort the install if my DLL returns IDAbort?
Labels (1)
0 Kudos
(7) Replies
Ajay_Ladsaria
Level 7

I am guessing that you need to return ERROR_INSTALL_FAILURE instead of IDABORT to cause the install to abort. ERROR_INSTALL_FAILURE is used as the return code from MSI C++ DLLs. It is defined in msi.h to have a value of 1603.

Reference:
Custom Action Return Values
http://msdn.microsoft.com/en-us/library/aa368072(VS.85).aspx
0 Kudos
Videstra
Level 7

Ajay Ladsaria wrote:
I am guessing that you need to return ERROR_INSTALL_FAILURE instead of IDABORT to cause the install to abort. ERROR_INSTALL_FAILURE is used as the return code from MSI C++ DLLs. It is defined in msi.h to have a value of 1603.

Reference:
Custom Action Return Values
http://msdn.microsoft.com/en-us/library/aa368072(VS.85).aspx


Here's what I've discovered so far...
If I do NOT check "Use custom method signature" then I can return either IDABORT or ERROR_INSTALL_FAILURE and it will abort the install.

But ... I need to pass a parameter into my managed dll (I need to send in the value of [INSTALLDIR] so my dll can check a few things there). It appears the only way to pass parameters into the dll is to check "Use custom method signature.".

Sure - I can pass back ERROR_INSTALL_FAILURE, but I have to assign it to a return property. Just returning ERROR_INSTALL_FAILURE doesn't do anything at all. The documentation says that InstallShield discards the return value if no return property is specified.

Is there a property that I can set to the value of ERROR_INSTALL_FAILURE that will cause InstallShield to jump to the install failure dialog just like it does when I do not set the DLL to "Use custom method signature?"
0 Kudos
Ajay_Ladsaria
Level 7

Thank you for the follow-up post. Please take a look at the following help documentation:

Specifying the Signature for a Managed Method in an Assembly Custom Action
http://helpnet.flexerasoftware.com/robo/projects/installshield16helplib/ManagedCA-Arguments.htm

Notice the following bit:

"Note that if you use a custom signature, the custom action’s return value is not passed to Windows Installer. However, if the managed code throws an unhandled exception, the custom action returns ERROR_INSTALL_FAILURE to Windows Installer. Therefore, in order for the custom action to indicate failure and for Windows Installer abort the installation, the managed code must throw an unhandled exception."

Regards,
Ajay
0 Kudos
Videstra
Level 7

Thanks:
In the doc it says:

...if you use the default method signature, the custom action handles methods with up to one parameter.

But it's very unclear how you actually pass the parameter to the method. There is no place to input the parameter like there is when you use a custom method signature.

It says "if the method takes an argument it is passed the msiHandle" - I have absolutely no idea what the msiHandle is or how to get it or use it in my DLL. I guess it is something I have to include in my project...

My Dll has a function that looks like this in VB.NET


Function SetHost(ByVal InstallDirectory as String) as Integer
If my.computer.filesystem.folderexists(InstallDirectory) then
'some code here to update a configuration file here
Return IDSUCCESS
Else
Return IDABORT
End if
End Function


Would someone be so kind as to provide an example of how I would change this to access the value of [INSTALLDIR] in the function?
0 Kudos
Christopher_Pai
Level 16

Reason number 7,543 to use DTF. Visual Studio / .NET / C# is a strongly typed language, rich in metadata and a nice IDE to work in, right?

So why should you have to worry about the number associated with exit codes?

Just switch to DTF and you'll have the ActionResult enumeration available to you. It's really as simple as typing:

return space (intellisense ) enter . ( intellisense ) enter ;

to get

retrn ActionResult.Success; // ( Microsoft.Deployment.WindowsInstaller.ActionResult enum { Failure, NotExecuted, SkipRemainingActions,Success,UserExit } )
0 Kudos
MichaelU
Level 12 Flexeran
Level 12 Flexeran

If you use the default method signatures, it emulates a C++ custom action, passing the Installer's "session" handle (an integer); however it also handles methods with zero parameters by passing nothing. You can either take a single parameter and P/Invoke to the MSI API, or you can instead use a custom signature and specify [INSTALLDIR] for that parameter. If you do the latter, you will probably want your code to look more like:

' Note: assumes custom signature to support strings and ignore return values
' (Standard signatures pay attention to standard MSI return values)
Function SetHost(ByVal InstallDirectory as String) as Integer
If my.computer.filesystem.folderexists(InstallDirectory) then
'some code here to update a configuration file here
Return IDSUCCESS ' <-- value here doesn't matter to InstallShield unless you specify a return property
Else
Throw New IOException("INSTALLDIR doesn't exist")
End if
End Function

In this code, the thrown exception will cause the installation to abort. If you don't want to consider this case as an error, you could use your original code.
0 Kudos
arshia555
Level 3

i am working on installscripts and is there a problem with me,
i want to display a progress dialog that look likes the picture that i attached, but i only can show this (Disable (STATUSDLG); Enable (STATUSOLD);) dialog but its haven't something such az cancel button or window mode, i want to know how i can show that dialog in the picture.
is there something else, i don't know how add a dialog between 2 dialog or
disable some dialogs, because if i do this, backbutton of dialogs not working properly, please tell me how i can do and what i must write in scripts.

THANKS

(sorry for my bad eng)
0 Kudos