call Winapi OpenMutex from InstallScript problem

[Alibaba]

Hi guys,

has anybody an idea why my use of the OpenMutex function always returns 0:

	

prototype  NUMBER kernel32.OpenMutex (NUMBER, BOOL, STRING);

nRetUD   = UseDLL(sSystemFolder ^ "kernel32.dll");
		
	svName         = "{B45C9CA7-478C-455E-A84A-A516374CB2AC}";
	bInheritHandle = TRUE;
	nDesiredAccess = 1048576;
	
nHandle = OpenMutex(nDesiredAccess, bInheritHandle, svName);

If I call this DLL (OpenMutex) from InstallAware it is working and returning the handle for the mutex.

Or Anybody an other idea how to check if a program is running?

[joshstechnij · Accepted solution]

OpenMutex is one of many Win32 APIs that have ANSI and Unicode variants. OpenMutex, like other APIs, is typically a #define in the Windows headers to either OpenMutexA or OpenMutexW depending on preprocessor defines such as UNICODE/_UNICODE. The InstallScript engine, since InstallShield 2011, defaults to calling the W (Unicode) variants of these APIs when neither has been specified. When external functions accept string parameters in such cases, the string parameters should be defined as Unicode strings (WSTRING in InstallScript). If they are not, an ANSI string will be passed to a Unicode API which will, generally, always fail.

The prototype for OpenMutex should be one of the following:

// Explicit Unicode API
prototype NUMBER kernel32.OpenMutexW(NUMBER, BOOL, WSTRING);

// Explicit ANSI API
prototype NUMBER kernel32.OpenMutexA(NUMBER, BOOL, STRING);

// Implicit Unicode API as neither was specified
prototype NUMBER kernel32.OpenMutex(NUMBER, BOOL, WSTRING);

For more information on WSTRING, see the help topic 'Data Types and Predefined Structures' in the help library.

[Alibaba]

It is working!!! 🙂 Thank you very much! I already had a feeling that is has to do with UNICODE but I could not find anything in MSDN...

I am happy