Registering 64-bit native DLL failed

[helenAtHoriba]

I use InstallShield 2016  Professional edition. The project is InstallScript

Operating system: Windows 7 64-bit.

The COM-based DLL file is native 64-bit. I could not register as self-register, as few components are added at a later stage. So at the very end of installation I would like to run a batch file.  I use the code below to do the job without any success. The DLL failed to register as the command prompt is not running with elevated priviledges. If I execute the batch file manually "Run As Administrator", everything works as a charm.

What am I missing? If the DLL is 32-bit and the OS is 32-bit, there is no problem

szLaunchCmd = TARGETDIR ^ "regCOM_64.bat";

LAAW_SHELLEXECUTEVERB = "runas";
 retVal = LaunchApplication( szLaunchCmd, "", "", LAAW_STARTUPINFO.wShowWindow,
LAAW_PARAMETERS.nTimeOut, LAAW_OPTION_WAIT | LAAW_OPTION_FIXUP_PROGRAM  | LAAW_OPTION_USE_SHELLEXECUTE);

[banna_k]

Hi @helenAtHoriba,

Can you see whether registration works after disabling the windows file redirection in 64 bit operating system. 

WOW64FSREDIRECTION—Disables 64-bit Windows file system redirection. You may need to do this before installing files to the WINSYSDIR64 destination, and enable it back after the batch file/cmd execuation.

https://helpnet.flexerasoftware.com/installshield25helplib/installshield25helplib.htm#Subsystems/LangRef/helplibrary/LangrefDisable00000677.htm#langref_appendixa_2230407831_1128402

https://helpnet.flexerasoftware.com/installshield25helplib/helplibrary/Targeting64Bit-IS.htm

Alternatively, can you see whether you are able to register the 64 bit DLL by calling the regsvr32 from the System32 folder after disabling the File redirection.

[helenAtHoriba]

Hi @banna_k,

I want all my files to be installed in native 64-bit directories, such as "C:\Program Files",  which they are right now.

My batch file consists of the following line:

c:\windows\system32\regsvr32.exe "C:\<TargetDir>\<FileName>.dll"

I have tried  to avoid calling the batch file and used the call to "regsvr32" directly, but no luck as well.

As from my knowledge, if I disable WOW64FDREDIRECTION, then my package would finish up in "C:\Program Files (x86)" directory.

Helen

[rguggisberg]

Most likely InstallShield is executing the 32 bit version of regsvr32.exe. Try this to get it to use the 64 bit version on a 64 bit machine:

c:\windows\Sysnative\regsvr32.exe "C:\<TargetDir>\<FileName>.dll"

Note that you can't use Sysnative in Windows explorer.

[helenAtHoriba]

Hi @rguggisberg 

My main concern is that the command prompt is not opening with elevated privileges,  I don't see "Administrator:" in the title of the window

I'll try to run using Sysnative

[rguggisberg]

In InstallShield IDE:

• Go to Releases tab
• Select your Release
•  Setup.exe tab
•  Setup Launcher
• Set 'Required Execution Level' to Administrator
• Rebuild

[helenAtHoriba]

Hi @rguggisberg 

The "Required execution level" is already set to "Administrator".

Changing the path to "Sysnative" made things worse - as there command prompt didn't recognized it as proper path

[rguggisberg]

Since CMD is involved... I am sure it is executing the 32 bit version of CMD even though you specified System32 (redirection taking place in a 32 bit app). Try replacing CMD.EXE in your script with this to execute 64 bit version of CMD. %windir%\Sysnative\cmd.exe Then you don't need SYSNATIVE anywhere else in your cmd line. Note that SYSNATIVE is not available on older OS's. What is your OS?

[helenAtHoriba]

Hi @rguggisberg
I'm testing on both - Windows 7 and Win 10 64-bit

[banna_k]

Hi @helenAtHoriba ,

As I stated in my first comment, can you check your batch file execution after disabling the WOW64REDIRECTION.  If you didn't disabled WOW64REDIRECTION, the registration will try to do with 32 bit version of the regsvr32 for the 64 bit COM DLL and ends up in error.   

See the modified script, this will work if you are able to register the 64 bit DLL by executing the batch file manually from the command prompt.

Disable(WOW64FSREDIRECTION);
szLaunchCmd = TARGETDIR ^ "regCOM_64.bat";

LAAW_SHELLEXECUTEVERB = "runas";
retVal = LaunchApplication( szLaunchCmd, "", "", LAAW_STARTUPINFO.wShowWindow,
LAAW_PARAMETERS.nTimeOut, LAAW_OPTION_WAIT | LAAW_OPTION_FIXUP_PROGRAM | LAAW_OPTION_USE_SHELLEXECUTE);

Enable(WOW64FSREDIRECTION);

[helenAtHoriba]

Hi @banna_k , @rguggisberg 

Thank you for all your suggestions, but so far - nothing..

I'm still at the crossroads.

[rguggisberg]

You might try logging the install and see if there is any useful info in the log file. "Filename.exe" /v"/l*v \"%CD%\log.txt\""

[helenAtHoriba]

Hi @rguggisberg 

I tried to log the InstallScript actions, but it's loggin only MSI part of the installation.

There is another log, but I didn't find anything suspicious in these logs.

I'm crushed for time, any other ideas????

[banna_k]

Hi @helenAtHoriba,

See whether you can do the registration by executing the 64 bit regsvr32 directly:

Disable(WOW64FSREDIRECTION);
LaunchApplication( WINSYSDIR^"regsvr32.exe", "/s "+szFile, ........);

Enable(WOW64FSREDIRECTION);

keep the above script in an event, and ensure the target file will be in place when the it get executed.

And ensure, all the dependencies of the file is exist in the location when the regsvr32 is executed.

For debugging, remove "/s " from the LaunchApplication, and see what is the cause for the failure by examining

1- What regsvr32 is getting executed 32 or 64 bit

2- required dependencies are in place when executing the regsvr32

3- While executing the regsvr32, the target file itself avialble in the target location.

[helenAtHoriba]

Hi @banna_k As I mentioned before, all the required files at the time of registration are installed and in the appropriate location. The "regsvr32" command is using 64-bit version of Systems directory. The only issue is that the command prompt at the time of registration is not opening with the elevated privileges. When the registration through InstallShield failed, before the Installation is finished, I could run the batch file either as "Run as Administrator" or through "cmd (Administrator)" and everything works as a charm.

[helenAtHoriba]

Hi @rguggisberg , @banna_k 

May be I should rephrase my question.

Why LaunchApplication function

  LaunchApplication( szLaunchCmd, ""...

is not running with the elevated privileges for the specified program (szLaunchCmd).

I have tried batch file, executable...

[rguggisberg]

Put this in your InstallScript BEFORE LaunchApplication.

LAAW_SHELLEXECUTEVERB = "runas";

Look in HELP under "InstallScript Language Referrence" and search for LAAW_SHELLEXECUTEVERB. There you will see...

LAAW_SHELLEXECUTEVERB

InstallShield 2018 » InstallScript Language Reference

The LAAW_SHELLEXECUTEVERB script variable is a string that indicates the verb used by LaunchApplication when calling ShellExecuteEx. The default value is open. The lpVerb member of LAAW_SHELLEXECUTEINFO points to this string by default.

Tip • If you are using LAAW_OPTION_USE_SHELLEXECUTE on systems running Windows Vista or later and you want to launch the application using the full administrator account (similar to right-clicking the executable file to be run and clicking Run as Administrator), set LAAW_SHELLEXECUTEVERB to runas before using LaunchApplication in your script:

LAAW_SHELLEXECUTEVERB = "runas";

This ensures that the application is always run with full administrator privileges regardless of whether the application to be launched has an application manifest with relevant settings. Note that this may trigger a User Account Control (UAC) prompt for consent or credentials.

On systems running operating systems earlier than Windows Vista, if runas is used, a Run As dialog box is displayed. The behavior is similar to right-clicking the executable file to be run and clicking Run As. This dialog box enables the end user to select the user account that should be used to run the application.

[helenAtHoriba]

Hi @rguggisberg I used your suggestion from the very beginning, would not even consider run with out "runas"