cancel
Showing results for 
Search instead for 
Did you mean: 
joshstechnij
Pilgrim

Re: InstallShield installation problem

Pete,

Thanks for trying this test. It looks like the test didn't go as expected due to changes in the underlying implementations of some Win32 functions starting with Windows 7. Microsoft refactored/reorganized a lot of the API surface which made our test not quite functional (the tests in the EXE were originally written to another issue we are seeing on Windows Vista/2008 Server).

I've attached an updated test EXE which should show the errors we are expecting. Please try the exact same steps again but using the test EXE in the zip file attached to this post. We're hoping this is the last test we need to run to capture the failure.
0 Kudos
pmeklin
Pilgrim

Re: InstallShield installation problem

Hi Josh,

here is a link to dump file: https://www.dropbox.com/s/t2awq20kqfig0al/MarshalTest.DMP?dl=0

Regards,
Pete
0 Kudos
joshstechnij
Pilgrim

Re: InstallShield installation problem

Hello Pete, David,

We believe we have tracked down at least one potential cause of this issue to having any MSI package installed that includes the oleaut32.msm merge module. This merge module was originally available to provide OLE support for Windows 95 and NT. However, it is no longer necessary and shouldn't be used. It may be possible for either older MSI packages or more recent MSI packages that still include the merge module to be installed. When these MSI packages are installed per-user, the COM information they contain for PSDispatch (a system implemented proxy/stub class to support marshaling IDispatch interfaces) is registered in HKEY_CURRENT_USER\Software\Classes, overriding what is already on the machine in HKEY_LOCAL_MACHINE\Software\Classes. When we call into COM APIs to marshal interfaces that our setup.exe's use, the overridden PSDispatch is used instead of the system registered object. Along the way, MSI gets involved because it registered this COM class ID. For whatever reason, MSI returns the implementing DLL (oleaut32.dll) back to COM without a path. When COM then loads the DLL, the relative path is combined with specific flags to the LoadLibraryEx API. This call fails with ERROR_INVALID_PARAMETER which then fails the entire call back up to the CoMarshalInterface call.


Long story short:
- We can now reproduce this failure on Windows 7/2008 R2 machines by installing an MSI package in a per-user context that also includes oleaut32.msm (this is one possible cause of the failure, we are unsure if any causes are possible)
- We're attempting to get some guidance on this issue from Microsoft; the original code change we made that triggers this failure is related to attempting to lock down how DLLs can be loaded into an installer process which we don't want to completely remove
- In the meantime, you can see if these machines you can't run the IS 2016 installer on have PSDispatch registered per-user:

1. Open regedit.
2. Go to HKEY_CURRENT_USER\Software\Classes.
3. Look for CLSID\{00020420-0000-0000-C000-000000000046}, or Wow6432Node\CLSID\{00020420-0000-0000-C000-000000000046}
4. If either of these contain the {00020420-0000-0000-C000-000000000046} key, rename this key (export the key first as a backup) by adding underscore characters before or after the GUID. Renaming the GUID back to its original name will cause the failure to occur again.

After following the above steps, running the MarshalTest.exe test should no longer present an error after the output 'Starting CoMarshalInterface test...'. As long as this error doesn't appear with the test EXE, you should be able to launch the IS 2016 installer.

Thank you for your patience and assistance helping to work through this issue.
0 Kudos
pmeklin
Pilgrim

Re: InstallShield installation problem

Hi Josh,

unfortunately we didn't find those registry entries under HKEY_CURRENT_USER\Software\Classes.
Under HKLM were some, but not under HKCU.

Pete
0 Kudos
Highlighted
MarkusLatz
Flexera beginner

Re: InstallShield installation problem

Hi Josh,

just tried at one of my customers. And 1st result was:

SetDefaultDllDirectories present...
SetDefaultDllDirectories result: succeeded (0x00000000)
Retrieving LoadLibrary pointers from kernelbase...
Starting shell link test...
LoadLibraryExW: C:\Windows\SysWOW64\shell32.dll, 0x00000000, 0x00002008
LoadLibraryExW succeeded
LoadLibraryExA: ole32.dll, 0x00000000, 0x00000000
LoadLibraryExW: ole32.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExA succeeded
Successfully created shell link
Shell link test complete
Starting CoMarshalInterface test...
LoadLibraryExW: msiltcfg.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExW: user32.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExW: msi.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExW: msiltcfg.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExW: msi.dll, 0x00000000, 0x00000022
LoadLibraryExW succeeded
LoadLibraryExW: msi.dll, 0x00000000, 0x00000022
LoadLibraryExW succeeded
LoadLibraryExW: msi.dll, 0x00000000, 0x00000022
LoadLibraryExW succeeded
LoadLibraryExW: msi.dll, 0x00000000, 0x00000022
LoadLibraryExW succeeded
LoadLibraryExW: oleaut32.dll, 0x00000000, 0x00002008
LoadLibraryExW: error 0x00000057 <-- likely cause of failure
Please open Task Manager and create a process dump of MarshalTest.exe (process id 6208)
Press any key to continue with this test...

And yes after looking into registry this pc had the key:

HKEY_CURRENT_USER\Software\Classes\Wow6432Node\CLSID\{00020420-0000-0000-C000-000000000046}

And as suggested, after rename the key I got the following output from MarshalTest:

SetDefaultDllDirectories present...
SetDefaultDllDirectories result: succeeded (0x00000000)
Retrieving LoadLibrary pointers from kernelbase...
Starting shell link test...
LoadLibraryExW: C:\Windows\SysWOW64\shell32.dll, 0x00000000, 0x00002008
LoadLibraryExW succeeded
LoadLibraryExA: ole32.dll, 0x00000000, 0x00000000
LoadLibraryExW: ole32.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExA succeeded
Successfully created shell link
Shell link test complete
Starting CoMarshalInterface test...
LoadLibraryExW: C:\Windows\SysWOW64\oleaut32.dll, 0x00000000, 0x00002008
LoadLibraryExW succeeded
CoMarshalInterface succeeded
LoadLibraryExW: api-ms-win-appmodel-runtime-l1-1-1, 0x00000000, 0x00000800
LoadLibraryExW: error 0x0000007e
LoadLibraryExW: ext-ms-win-kernel32-package-current-l1-1-0, 0x00000000, 0x00000800
LoadLibraryExW: error 0x0000007e
LoadLibraryExA: ADVAPI32.dll, 0x00000000, 0x00000000
LoadLibraryExW: ADVAPI32.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExA succeeded
LoadLibraryExA: OLEAUT32.dll, 0x00000000, 0x00000000
LoadLibraryExW: OLEAUT32.dll, 0x00000000, 0x00000000
LoadLibraryExW succeeded
LoadLibraryExA succeeded
Test thread exit code: 0x00000000

I made a dump file from 1st failure, if needed let me know.

I tried this at several pc from the customer and got on every pc the same results. May be a "base package" creates the key. The customer has > 1000 pc's

regards

Markus
0 Kudos
joshstechnij
Pilgrim

Re: InstallShield installation problem

Hello Pete, David,

We believe we have a fix for this issue. Please let me know what edition of IS 2016 you are attempting to install (Express, Professional, Premier) so that I can provide a link to download an installer that you can try on Windows 7 machines that exhibit this problem.

Markus,
Is this an issue with the IS 2016 installer, or with an installer you are building? If it is your installer, please let me know what type of project is failing: Suite, pure InstallScript, or InstallScript MSI. Also, is this failure occurring on a version of Windows older than Windows 8? We should have a fix available that you can try in this case. If this is related to installing IS 2016 please let me know what edition you are trying to install.
0 Kudos
MarkusLatz
Flexera beginner

Re: InstallShield installation problem

None of them.

Next week I have a new project to build a suite installer for a customer. Last version I build with IS 2013 (works very well). Now I would like to use IS 2016 to build the new suite installer.

I read the thread and tried your test program on a few pc's of this customer to test if I run into trouble if I use IS 2016 to build the new suite.

The customer uses Windows 7 , Windows 8/8.1 and Windows 10 pc's, all with 64-Bit edition.

If necessary I can create a small "test suite" to check if the setup fails on the pc's where your "MarshalTest" program fails.

regards

Markus
0 Kudos
joshstechnij
Pilgrim

Re: InstallShield installation problem

Markus,

If you plan on building any suite installers with IS 2016, please make a backup of the following files and then replace the existing files with the files contained in the attached zip file:
\Program Files\InstallShield\2016\Redist\Language Independent\i386\SetupSuite.exe
\Program Files\InstallShield\2016\Redist\Language Independent\i386\SetupSuite64.exe
\Program Files\InstallShield\2016\Redist\Language Independent\i386\SuiteAppxHelper.exe

The issue we have seen in the field is related to a change we made to lock down where third party DLLs can be loaded from in all of our setup launchers. On Windows 7 and older machine (back to Vista) an issue occurs partly due to this change that causes a failure in the CoMarshalInterface API the launchers rely on to implement certain parts of the setup UI functionality. As a result, suite launchers will fail to launch and will log a line such as 'UI resource exception: 80070057' in a debug log.

This failure is a combination of the code change we made, something installed on the machines where the failure occurs (one trigger is MSI packages that install the PSDispatch proxy/stub class that has been implemented by Windows for well over a decade), and then what appears to be a bug in the implementation of LoadLibraryEx with MS KB2533623 (or subsequent update that supersedes KB2533623) installed. The failure only occurs on Windows Vista/2008 or Windows 7/2008R2 machines as the behavior of LoadLibraryEx appears to have been changed on Windows 8.1 and newer.

Note that this issue also affects pure InstallScript and InstallScript MSI setup projects in the same circumstances, but the issue manifests itself as a crash during setup initialization (also as a crash while creating shortcuts on Vista/2008 machines that are fully updated).
0 Kudos
MarkusLatz
Flexera beginner

Re: InstallShield installation problem

Hi Josh,

thank you very much for your fast reply with Hotfix. I will try next week and will let you know if everything works.

regards

Markus
0 Kudos
pmeklin
Pilgrim

Re: InstallShield installation problem

Hi Josh,

we tried to install Premier version of IS 2016.

Regards,
Pete
0 Kudos