UseDll fails after update to SP2

[olivercern]

Hi,
we have some older solution that uses InstallShield 2015. While trying to solve another problem, we got recommendation to update InstallShield to SP2.

now we have problem because script that worked before now fails. in this script we call UseDLL() on library that is in SUPPORTDIR. it returns -1.

we can revert to SP1, but I believe it would be better to keep InstallShield up to date.

[GeorgeK]

Try the following to identify an error number or other information used to investigate. This is intended to produce a test-case for you. What are the results?

1.  File > New > InstallScript project.
2. Go to the 'Behavior and Logic->Support Files/Billboards' view. 
3. Under 'Support Files->Language Independent' right click in the Files windows and select 'Insert Files'. 
4. Select the file(s) above and insert them into your project as support files. 
5. Go to the 'Behavior and Logic->InstallScript' view .
6. Select 'Before Move Data' and then 'OnBegin'. 
7. Add the following code to the OnBegin() event.

NUMBER	nResult;
...
ChangeDirectory(SUPPORTDIR);
nResult=UseDLL(SUPPORTDIR^"<YOURDLLNAME.dll>");
if (nResult<0) then
	MessageBox("Fail: "+SUPPORTDIR^"<YOURDLLNAME.dll>", 0);
	SprintfBox(INFORMATION, "Debug", "Err.Number= %d", Err.Number);
else
	MessageBox("Success [<YOURDLLNAME.dll>]",0);
	UnUseDLL("<YOURDLLNAME.dll>");
endif;

[Jenifer]

Hello,

Possible causes would be:

• Dll might not be present in the expected location
• Or dll file's dependent files might be missing
• There is a claim in previous InstallShield forum threads saying if it is 64-bit dll and platform is of 64-bit UseDLL()  might not work.(I am not sure on this point)

I could see this from InstallShield UseDLL reference on returning errorcode less than 0:

Return Values

UseDLL Return Values

Return Value	Description
0	Indicates that the function successfully loaded the .dll file into memory.
< 0	Indicates that the function was unable to load the .dll file into memory. If UseDLL fails, the most likely cause is that the .dll file was not found. If this happens, make sure the correct path is specified in the parameter szDLLName. Another common cause of failure associated with using .dll files is related to .dll file dependencies—.dll files accessed by the .dll file that you load. If the .dll files that your .dll file accesses are not loaded or found, your .dll file call may fail. If this occurs, make sure that the other .dll files are on the system and that they are accessible. If the script exits or terminates before properly unloading the .dll file with UnUseDLL, the .dll file will be locked in memory. If you attempt to access the .dll file again, your script may fail. You must remove the .dll file from memory by restarting Windows.

More over i could see relevant forum links which is similar to the one you face:

• https://community.flexera.com/t5/InstallShield-Forum/SUPPORTDIR-and-UseDLL/m-p/27071?collapse_discussion=true&filter=location&location=category:InstallShield&q=usedll&search_type=thread
• https://community.flexera.com/t5/InstallShield-Forum/Usedll-fails-to-load/m-p/63723?collapse_discussion=true&filter=location&location=category:InstallShield&q=usedll&search_type=thread

Hope it helps,

Thanks,

Jenifer

[banna_k]

Hi @olivercern   

Can you check the reported issue with InstallShield 2018, looks like this issue has been fixed in IS2018 (Refer IS 2018 release note @ https://helpnet.flexerasoftware.com/installshield24helplib/rn/ReleaseNotes.htm#isreleasenotes_2630790174_1049156). 

Hope you have mentioned about updating the project from  IS2015 to IS2015 SP2.  If you are updated to IS2015 SP2 and want to continue with IS2015 SP2 then you can call win32 API "SetDllDirectory" to add the SUPPORT DLL to your search path in your script. 

For more info on "SetDllDirectory", you can refer https://docs.microsoft.com/en-us/windows/desktop/api/winbase/nf-winbase-setdlldirectorya

Better solution will be migrating to InstallShield 2018 directly from IS 2015, then your script functions will work without any changes. 

Second solution, continue with IS2015 SP2 and modify the script function to call the win32 function "SettDllDirectory(SUPPORTDIR)" before calling the "UseDLL" functions, and safer side call "SetDllDirectory" with and empty string after the "UseDLL" call to avoid the DLL preloading attack as mentioned in the release note.