Preliminary Troubleshooting Steps for Digital Signing Issues prior to InstallShield 2023 R2

[#1]

Introduction

This article discusses preliminary troubleshooting steps for digital signing issues with InstallShield versions prior to InstallShield 2023 R2.

Troubleshooting Steps

1. Determine the InstallShield Edition (Express, Professional - renamed to InstallShield Edition, Premier) based on the Help > About InstallShield screen.
2. Determine the InstallShield project type: Basic MSI, InstallScript, InstallScript MSI.
3. Determine which Cloud HSM is storing the digital certificate: AWS Cloud HSM, Azure Key Vault, DigiCert KeyLocker, or a different Cloud HSM.
4. Determine whether you can manually digitally sign an arbitrary file with signtool.exe, azuresigntool.exe, if you're using Azure Key Vault, or smctl.exe, if you're using DigiCert KeyLocker, outside of and without using InstallShield, just as a test.
5. If step# 4 succeeds, the project type is Basic MSI, and the InstallShield Edition is Premier Edition, under the Releases > Release > Events, configure the following Windows Batch file for the Precompression Build Event to digitally sign the MSI:
   
   In the Precompression Build Event field, specify the following:
   
   C:\InstallShield 2022 Projects\BasicMSIDigitalSigningTest\signMSI.bat​
   
   In the signMSI.bat file, include the following:
   
   "C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x86\signtool.exe" sign /fd SHA256 /debug /f "C:\Users\Test\Desktop\MySelfSignedCertXYZ.pfx" /du https://www.revenera.com /t http://timestamp.digicert.com /p "<PFXPassword>" "C:\InstallShield 2022 Projects\BasicMSIDigitalSigningTest\Product Configuration 1\Release 1\DiskImages\DISK1\BasicMSIDigitalSigningTest.msi"​
6. If the project type is Basic MSI and the InstallShield Edition is Premier Edition, under the Releases > Release > Events, configure the following Windows Batch file for the Postbuild Build Event to digitally sign the setup.exe:
   
   In the Postbuild Build Event field, specify the following:
   
   C:\InstallShield 2022 Projects\BasicMSIDigitalSigningTest\signSetupEXE.bat​
   
   In the signSetupEXE.bat file, include the following:
   
   "C:\Program Files (x86)\Windows Kits\10\bin\10.0.19041.0\x86\signtool.exe" sign /fd SHA256 /debug /f "C:\Users\Test\Desktop\MySelfSignedCertXYZ.pfx" /du https://www.revenera.com /t http://timestamp.digicert.com /p "<PFXPassword>" "C:\InstallShield 2022 Projects\BasicMSIDigitalSigningTest\Product Configuration 1\Release 1\DiskImages\DISK1\setup.exe"​
7. Build an uncompressed release.

Outcome

If this issue is resolved, the uncompressed .msi file should have a Digital Signatures Tab and list the digital certificate information indicating that the MSI is digitally signed and the uncompressed setup.exe should have a Digital Signatures Tab and list the digital certificate information indicating that the setup.exe is digitally signed.

More Information

To download the sample project used to test the troubleshooting steps in this article, see the BasicMSIDigitalSigningTest.zip file attached to this article. Note: In the Windows batch files included with the sample project, make sure to change <PFXPassword> to your PFX password for your digital certificate.

If this article did not resolve the digital signing issue(s), contact Support.

For more information about digital signing with Extended Validation (EV) digital certificates, specific to InstallShield 2023 R2, please review the documentation here.

BasicMSIDigitalSigningTest.zip