DotNetCoCreateObject function getting padded string
I wrote a C# dll with a function that requires a string. I noticed that the returned value was not correct. So I changed my function to simply return the length of the string. It turns out the C# version of my function is getting the string padded with \0 up to a length of 1024.
I printed out the string and its length before passing it, and it's correct. So what the heck is happening?
Here's an old thread with a workaround, but why is it happening in the first place? https://community.flexera.com/t5/InstallShield-Forum/ComboBox-Values-Length-is-always-1024/td-p/7575...
prototype string ServerInstallHelper.sha256Hash(BYREF STRING);
set oInstallHelper = DotNetCoCreateObject( SUPPORTDIR ^ "ServerInstallHelper.dll","ServerInstallHelper.Helper", "");
szHash = oInstallHelper.sha256Hash(szHashMe);
We ran into this a while ago for what I'm guessing is a very similar example. Our solution was to add an extra line in the InstallScript before sending the string to the DLL function. We have this issue when the string comes from both CtrlGetText and a Silent Install response file.
//need to "fix" it here so a valid string will be sent to the decryption DLL StrSub( svEncryptedSQLPassword, svEncryptedSQLPassword, 0, StrLengthChars( svEncryptedSQLPassword ) );