Unicode wrapper requires codepages?

[JulienPicalausa]

Hello,

I am working with the multilingual MSI installer and have difficulties regarding the Unicode support of the InstallShield wrapper.
I am trying to add some Unicode only languages to this installer (namely hindi, tamil, telugu and georgian), but, unlike for the other languages that I am adding, this proves unsuccessful. I can successfully build a package with those languages but upon running it and choosing one of the aforementioned languages(let's say Tamil), I am getting an error 1208, telling me that the codepage for Tamil is not available on the system. The error message displays correctly, using Unicode.

I've asked someone to have a look at the generated package and they told me this:

<<
I reverse engineered the installer, and found this:
After choosing a language, the installer calls GetLocaleInfo with the respective locale ID (e.g. 0x449 for Tamil) to request the LOCALE_IDEFAULTANSICODEPAGE.
It then calls IsValidCodePage on the codepage that is returned, and if that fails, displays the error message above and quits.
Both the Unicode and the Ansi versions of the installer do this.

Problem is, the codepage returned by GetLocaleInfo on my system is "0".
This article confirms that indeed, LOCALE_IDEFAULTANSICODEPAGE for the languages in question is always 0:
>>

So, my questions are:
Did I miss an option somewhere to skip this check?
If not, why does the wrapper need to know the codepage in this case? If it uses Unicode anyway, what is the codepage good for?
If the codepage is necessary anyway, should it not assume one of the Unicode codepages (whichever encoding is most appropriate) when dealing with unicode only languages?

So, basically is there any way this can be fixed? 🙂

Thanks in advance,
Julien Picalausa

[MichaelU]

Ah, you're examining things built in the long history in our product. 😉 First let's verify that what you've built is actually usable. If, instead of running setup.exe, you run the .msi file directly, do you get your Tamil or other Unicode-only languages displayed correctly? Having not tested it, I'd fear our build might choose an invalid code page (probably 0) and invalidate your strings well before the step you've found.

If this does display correctly, skipping setup.exe may be the best option for a work-around, and we will have to re-examine our check for code pages (a comment suggests the purpose of the check you found is to ensure that the requested language can be displayed - for example the fonts for Japanese and other East-Asian languages are not available by default on an English Windows XP system), and find a check that verifies whether a Unicode-only language is supported on a machine.

[JulienPicalausa]

MichaelU wrote:
Ah, you're examining things built in the long history in our product. 😉 First let's verify that what you've built is actually usable. If, instead of running setup.exe, you run the .msi file directly, do you get your Tamil or other Unicode-only languages displayed correctly? Having not tested it, I'd fear our build might choose an invalid code page (probably 0) and invalidate your strings well before the step you've found.

You are right. Looking at the MSI, it does appear that the strings are invalidated. (Looking at it with Orca and an hex editor revealed that the characters were just replaced with 0x7f). I have read somewhere however that MSI should supports UTF-8 (using 65001 as codepage). I assume that this is something you may be able to get working on your side?

MichaelU wrote:
If this does display correctly, skipping setup.exe may be the best option for a work-around, and we will have to re-examine our check for code pages (a comment suggests the purpose of the check you found is to ensure that the requested language can be displayed - for example the fonts for Japanese and other East-Asian languages are not available by default on an English Windows XP system), and find a check that verifies whether a Unicode-only language is supported on a machine.

English Windows XP systems can support all languages but you have to install the supplemental language supports from Control Panel > Regional and Language options > Languages
The best way to check if a particular subset of Unicode is supported on a machine is to test the available fonts themselves. (with GetTextCharsetInfo, for exemple)

[MichaelU]

Sorry to hear the data is lost. Our next release should have a feature that you'll be quite interested in, then. 🙂 However I think it will still have the same problem in setup.exe - namely that we're still checking for a code page. After some quick research, it looks like IsValidLocale may be a much better check for us to use. We'll have to verify the behavior on various versions of Windows, though, and whatnot; perhaps we can coordinate to get a fix for this bad check earlier than the following release.

I've reported this as IOC-000081558. For my ease in fixing this, would it be easy for you to send me an exported string table from Tamil or any other Unicode-only language? Stripped down to a few strings is fine, and I don't care if they're meaningful; just that they're on an early dialog.

[JulienPicalausa]

I've got the authorization to send you the strings we are using. Can you provide me with your email address or another mean to make those files available to you directly?

[MichaelU]

Sure. It's the same as my community id @acresso.com, or feel free to send me a private message. I'll try to keep you posted with what I find out.