cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 
jscozz
Level 4

CUstom action dialog NOT WORKING!

I have to be honest... I get tired of spending over $1000 a year and getting new problems every time I upgrade.

A custom action that has been running for YEARS in previous IS versions, is now having an issue in IS 2011 Pro. NOTHING has changed. Upgraded the old project, built it, new problem.

The action runs, but it is supposed to display a dialog and get a YES/NO from the user then act on it. The dialog never displays and the custom action continues as if NO was chosen. It uses the DialogBox() call with a resource and dialog handled in the custom action dll.

Can anyone tell me what is going on here??????

I have a major release to get out tonight and now because of this IS issue it will be delayed.
Labels (1)
0 Kudos
(7) Replies
jscozz
Level 4

OK. So I know what is happening... just not WHY!

In my dll I call GetModuleHandle("mydll.dll"). In all versions of IS prior to 2011 this has worked fine! Now I get 0 returned. So I can't load my Dialog resource from the dll.

Anyone know why this is? Or any workaround?
0 Kudos
jscozz
Level 4

Also, it appears the DLLMain is never called by Installshield, so I cannot get the instance handle form there either.

This seems utterly ridiculous! What is the secret to using a dialog resource in my custom action dll now that Installshield has messed this up? All I need is the instance handle of the dll...
0 Kudos
Alpesh
Flexera Alumni

Hi,

What kind of custom action are you using? If you are using a Managed Code custom action, can you check if there are two Sql Connections in your project? If there are, just make sure that the Sql password property across the Sql connections is different.

Thanks!
0 Kudos
jscozz
Level 4

Just simply a C++ DLL.

1) IS has NEVER called the DLLMain function when the DLL is loaded, so I can't get the instance handle there.
2) Now IS 2011 has somehow broken things so I can't get the instance handle from a GetModuleHandle call with the dll file name.

I found a round about way to get the instance handle from the base load address of the module... BUT, this is a pretty poor way to have to do it. Especially when it has worked for over 10 years in all other IS versions. Why does IS break the documented ways of doing this? I have to believe that somewhere else in the world someone has displayed a dialog from within a custom action... ????
0 Kudos
rguggisberg
Level 13

Is this perhaps a permissions issue?
Have you moved to Windows 7 or 2008 Server (where you may have been on XP before) on either the IS Build or the target machine?
0 Kudos
jscozz
Level 4

Nope. Behaves the same on XP, Vista, 7, etc.
0 Kudos
Roman1
Level 9

Hello jscozz,

I feel with you.

But, please try to put your issue in order.
Write down the way what are you doing:

1. IS calls CA, where is this CA (ISScript or C++ dll)?
2. Where is the new DialogBox? Who calls it?
3. DllMain is calling by LoadLibrary. You do not mantion about it
4. Zip a test project.
0 Kudos