InstallShield 2019 R3 crashes on "Build Release"

[gcouch]

I am running InstallShield 2019 R3 (Premier Edition) in Windows 10 and it crashes when I perform a "Build Release". The last log I see is "Extracting COM data from 1 component(s)" and then it freezes up.

Does anyone know what could cause this issue and how I can prevent it?

[banna_k]

@gcouch :

Can you see, is there any dump (.dmp) file created in the Installshield system folder. If so, can you share the dump file.

And see, if you are able to build the project if you make the "COM Extract at Build" option to "No" in the components view.

[gcouch]

I do not see any dump file being created in InstallShield\2019\System.

When I turn off the "COM Extract at Build" option to "No" in the components view, InstallShield no longer crashes.

[gcouch]

I just discovered that the problem is related to IsRegSpy.exe. If I kill this process using the Windows Task Manager, the InstallShield recovers and continues on with the build.

I found the following, which seems to describe a similar problem but the link containing a possible fix does not work:

https://community.flexera.com/t5/InstallShield-Forum/IsRegSpy-exe-not-extracting-COM-information-correctly/td-p/22971

[banna_k]

@gcouch 

Ok, what kind of component the com extraction got hanged.

is it a x86 or x64 C++ or C# component.

[gcouch]

It is hanging on an x86 component but I am not sure if it is written C++ or C#. 

(The DLL is capicom.dll:  https://docs.microsoft.com/en-us/windows/win32/seccrypto/getting-ready-to-use-capicom)

Does it make a difference whether it was written in C++ or C#?

[banna_k]

@gcouch :

Thanks for sharing the information.

Can you check, all the dependencies of capicom.dll are in place during the COM extraction. According to the Microsoft link, this DLL is supported in the Windows Vista, XP and server 2008. See, in what operating system you are building, and all the dependencies for this DLL is present during the COM extraction. [CAPICOM is a 32-bit only component that is available for use in the following operating systems: Windows Server 2008, Windows Vista, and Windows XP.]  

Yea, one is manged and other one un-managed. But, Installshield supports extracting both of them.

capicom.dll component is not a C# component, looks like C++ or VB.

We can analyze the COM extraction behavior on capicom.dll, if you are providing this DLL. Not able to download from the Microsoft link.

[gcouch]

I found a link where the capicom DLL is available:

https://www.microsoft.com/en-us/download/details.aspx?id=25281

How do you analyze the COM extraction behavior for a DLL? Are there any log files associated with this COM extraction?

From what i have read, the capicom.dll should work fine in Windows 10, even though it is very old.

[banna_k]

@gcouch ,

Thanks for providing the link.

I could extract the COM information from this DLL without any issues, in a windows 10 machine.

InstallShield provides three different methods to extract the COM information, but at a time InstallShield extract the information based on the method configured in the registry. UseAPIRegistryHooks registry value located in HKEY_LOCAL_MACHINE\Software\InstallShield\RegSpy key (on 32-bit machines) or HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\InstallShield\RegSpy  key(on 64-bit machines) holds the method configured to extract the COM information.

Value	Description
0	COM information captured by hooking Windows Registry API, old method.
1	COM information captured through Registry redirection API
2	COM information captured through Kernel mode registry filtering; latest method introduced in InstallShield 2012.

Value "2" is the latest method, which is more efficient than all others. But, in some cases value can be changed to the suitable method if the default one(value "2") is not working.

InstallShield provides a utility tool named RegSpyUI to cross check the values extracted from COM component, extracted by the method specified in the registry. RegSpyUI tool is available in the InstallShield support folder(C:\Program Files (x86)\InstallShield\2019\Support), this tool can be used to verify the COM extraction done by InstallShield.

See, RegSpyUI  able to extract the COM information with the registry value 2 and then see with value "1". 

[gcouch]

It looks like I don't have the registry entries
HKEY_LOCAL_MACHINE\Software\InstallShield\RegSpy
and
HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\InstallShield\RegSpy

I can see "RegSpyProxy" but there does not appear to be "RegSpy"

I am using Version 25 of InstallShield 2019 R3. Has the registry path changed?

The RegSpyUI.exe tool crashes as well when I attempt to extract the COM data from the capicom.dll file:|

[banna_k]

@gcouch 

Can you create the key manually and set the values as mentioned in the previous comment.

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\InstallShield\RegSpy

Then see the behavior with different key values.

[gcouch]

When I create the registery entryHKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\InstallShield\RegSpy\UseAPIRegistryHooks

and set the value to 1, my problem no longer occurs. Why exactly is this change is necessary?  Is this an indication of a problem with the file capicom.dll?

Is there a way to configure UseAPIRegistryHooks as part of the installation of InstallShield? Manually adding a registry entry seems like an archaic way to configure InstallShield to work with capicom.dll.

[banna_k]

@gcouch ,

By default, installshied uses the latest methodology to extract COM information (ie Value "2"). In some cases, if the kernel driver methodology is not working, it can be switched back to the other methodology by changing the values in the registry.

[roman2]

RegKey absent.

But RegSpyUI.exe extracts COM information.

I use IS2019 R3 Prof.

[pvoruganti]

My Visual Studio build is getting stuck at below step of output

2>Extracting COM data from 2 component(s)

after killing this it is continuing. It is getting stuck from today onwards on the build server(I am using windows 2019).

Can you please let us know the issue?