Mystery files being added to project

[SteveEisner]

Has anyone seen this happen?

From the build logs:

Skipping .NET scan for component Askme_External_CRRedist2005_x86: KeyPath is not a file
File table successfully built
Building MsiFileHash table
Getting FileHash information for file _006396ED149B4B1AA0506B54377A9150.
Getting FileHash information for file _009A1C3EBD7B461AA5D9890A5A40A797.
Getting FileHash information for file _009F16B1CE244989BBF8A86F4ADA2694.
Getting FileHash information for file _00A3BB8853C244678E66C7D770AD2B4F.
Getting FileHash information for file _00A53CFD161E41F3B1EBDC1A85FFDD00.
Getting FileHash information for file _00BB897C03F0498A8BBF0C4FB25185FC.
Getting FileHash information for file _00BE3450A99F485E94D5D875D86FC06C.
Getting FileHash information for file _00FEF4374F0F4F1D9B97DF6C0F218EDB.
Getting FileHash information for file _010A340403BE433280DCDD21C7055FC3.
Getting FileHash information for file _015F47C6A1BE4E37B487E28A473ABEB8.
Getting FileHash information for file _018E992E80AD4095938A38206D1A6CEE.
Getting FileHash information for file _01BE56A4CBBD45F08EE436930E3ABA2F.
Getting FileHash information for file _01CEFC823785422FA52B73787237127D.
Getting FileHash information for file _01DE0BBEFE534B519E01F4C1F6A6DA43.
Getting FileHash information for file _01F787AE5EC5447FA213253DA7F51E9B.
Getting FileHash information for file _01F79D50471A49CE81285B3F2D95CBF5.
Getting FileHash information for file _01FC3C8397B545A38D8F18F5222E6C4F.
Getting FileHash information for file _0243411EB9B44E0C89929E91B66FCDC5.
Getting FileHash information for file _024C100AAB0C4E189423234EA3B06BAD.
Getting FileHash information for file _027326C9F3654548AC4D09BFC4384FD8.
Getting FileHash information for file _027FD35E03064AC1B163DCFF5F2082B6.
Getting FileHash information for file _02D1CA1F80B340BE9D96CAAEDA96A68F.
Getting FileHash information for file _02E698D80CB44040A1A6888003C35395.
Getting FileHash information for file _02F3046375AE4519823AF9BDC1ED8C28.
Getting FileHash information for file _02F55EBB7754460687140B5E3BC6B4D7.
Getting FileHash information for file _03072636AF5D4FD5B70117EBFA5948CC.
Getting FileHash information for file _0320031E97A54B80B9F1DCA0A4171AD6.
Getting FileHash information for file _03382220A666487D8098300A876E11E7.
Getting FileHash information for file _035329D243454757A8F3341F10F43BDF.
...etc... dozens of pages of these

From the build report:

Components
...snip legitimate components listed first...
_005DA50D5982B1BF7C3FDDA4FAA2EB53
_0110260595ACD4A42390601CA4E29762
_025BFAFC141C00400D50509998D7222D
_026743829BAF4D4236744634B635BDBF
_03148D6A8AE9369083EE683B7A9639B1
_03C8271CCFA9BFE4256FE3ED1ADB09B3
_0541021084E932BD736F24BC102D1D72
_0592D2A3A5AF924030966EE0FBBC1211
_0724058EE8B1E888DA6359515D166356
_07F287BE69078F828149DE33AEC6DA49
_08456FFEF5B28EFE535974A71AF1D12B
_0A19D9424DA21C22DEFB40277FEB9532
_0AEC5DA2C6629DF30BB1464218322200
_0BB757E5C66531FDD77E43406CA9240D
_0C5982E0AA98B11F562928976F9F66CB
_0D1D8A0B630888996067A24D2FA888A1
_0D4557A8EEC473BC19DC343F78FB5F83
_0F611FEA0689BACA3015FC1EB1E926B0
_0FB08025473489566427C853F24DA19B
_11B7CBCDF96AE9184B923FE7D68F9D06
_123691653273E24EABA6839B85FE58C2
_124FE9C8EA371FA20514D4560B66DEC4
_13F07D473BE4FFDC4F907EF2FCF0ABC6
_141EF71115C7D9353E82E9CB23367E8B
_14926A5B1DDFD2D98B9363CBAADEC767
_1522B8A1ACE2906AA34367DFA11442A5
_15D089844C3840C917AD152D3BA26F0A
_172F3E672EC17DBF0E0040E929CF1F86
_1793DC835D00EED6D0DAF19790031C96
_17DE0C3AF50D39B6599613054F2BE9E8
_18B305E304AD4C6FEEED4FE564821593
_1B6203ECB5A96784CF2EF09BC4F13BF9
_1CF42205BC2B4AEFBB9BBBF5A0B534C5
_1D4C2A32D7FBE2D747F8A142EF448155
_1DFB4BB1F43AACB712210215679CB9E6
...etc... followed by more legitimate components

Example from the build report file Files section:

Schema_6.7.6_6.8.sql
{_9DBC4EFABE4442BCA836F0AB15065271}
[ProgramFilesFolder]AC\AE\dbsetup\schema\incremental
_9AF1977D96F8960F46DB081F0A787FC5
8/6/2008 3:28:22 AM
7317

In other words, both the file name references and the Component references seem to have been turned into long hex strings. The projects do build correctly and produce a valid Setup.exe file. But then Setup fails during file copy because those hex files are missing.

I have no idea where these are coming from. When I load the ISM into the InstallShield IDE and do a "full project validate", these same files show up as errors ICE60, but when I click on the error it does not take me to any file list that relates to those files...

Any ideas? I'm totally stumped. Thanks!
Steve

[dan_galender]

You probably have ".NET Scan at Build" set to something other than "None" for one or more components. These new files are not in the .ism but are being put into the .msi by the build process.

[SteveEisner]

Thanks for the suggestion, Dan. I looked through the components and I did indeed have .NET Scan set on some of them. But I've since changed them all to "None" and am still seeing these files being added. Also note that as far as I can tell, they do not correspond to any file on disk...

Still stumped... 😞

[dan_galender]

How about MergeModules?

[SteveEisner]

Thanks - where would I look for that? Sorry, I'm completely new to InstallShield and inherited this project. I know what Merge Modules are but not necessarily how that would affect my project or what I can do about it 🙂

[dan_galender]

In the Redistributables view you would look for checked (nerge module) items. The contents of the tables (with file references) are merged into rhe msi file that results from a build.

[SteveEisner]

OK, thanks. I looked into that and there are no modules checked in the redistributables view.

I've started doing drastic tests like one by one deleting the components so I can figure out which one is introducing the files. Well, it turns out that as I remove components the list of mystery files shrinks, but does not go away with any single component. That seems to indicate that it's something that's being applied to each component.

Here's an interesting thing I've found. This is two file listings from the report...

restoreDB.vbs
{DBSETUP2} [ProgramFilesFolder]AC\AE\dbsetup
Database
8/6/2008 3:27:43 AM
6991

Job_ImportAndProcessReportData.sql
{_21E94C16DF6C49B78705B168476C3129} [ProgramFilesFolder]AC\AE\dbsetup\Reporting\schema\config
_E7C7E073E7086E55C72AD20DAF1AAF25
8/6/2008 3:27:39 AM
4890

What's interesting is that these files are both specified in the same Component in the installer, but when they appear in the report, the 2nd file says it is in the component _E7C7... All files from the directory "{DBSETUP2} [ProgramFilesFolder]AC\AE\dbsetup" correctly report that they're from the "Database" project, and all subsequent files from the directory "[ProgramFilesFolder]AC\AE\dbsetup\Reporting\schema\config" also report the same _E7C7... component.

Could it be that somehow each subdirectory specification is resulting in a new subcomponent?

[SteveEisner]

I've confirmed that it's somehow related to the "dynamic links", when I remove a component with dynamic links, the number of mystery hex files decreases by the same count as the number of folders + subfolders in that dynamic link. I did some searching and it turns out that this is the method by which InstallShield 2008 adds dynamic links - "one component per folder", so maybe this is actually expected behavior. That seems to match the behaviors I'm seeing.

Well, I wouldn't really care about the specifics of packaging IF it were working properly. Unfortunately when we use the produced installer, it says: "Error 1334: The file cannot be installed because the file cannot be found in cabinet Data1.cab" That's what led me to try to find this problem in the first place.

My last desperate guess is that there is some sort of problem with the compression or packaging of the .exe, so the whole thing is bad data, rather than this specific file being troublesome........

Help! 😞
Thanks! 🙂

[joshstechnij]

The only way this is typically possible is if there are more than 65,535 total files being added to data1.cab. If including the dynamically linked files causes this limitation to be exceeded, files will "randomly" start disappearing from the built CAB. If this is the case, you may be able to work around the behavior by compressing per-feature or per-component to try reducing the total number of files per CAB (note that CAB files are also limited to 2GB in total size; if this is exceeded, failures can occur).

Another possibility occurs during patching. If you are creating a patch from this setup, and the "previous package" setting is not being used in the release to build the latest setup, the file keys will change in the patch but the CAB file in the MSI being patched will not have the same file keys. However, since you didn't mention building a patch, I'm not sure this is the cause of the error you are seeing.

[SteveEisner]

Thanks, Josh.

I don't believe that we are adding more than 65535 files to the CAB,
and I know the resulting CAB is less than 2gb.

Random thought: is it possible that this could happen if the setup.exe included an older version of the installer that somehow could not read the generated compressed CAB? Or some other OS-specific conflict between the MSI generated by InstallShield and the OS-supplied MSI installer?

You suggested compressing per-component. Two questions about that: 1) does that result in CAB1, CAB2, etc. being generated and then stuffed invisibly into the same setup.exe? (in other words, is the effect transparent to me or the end user?) and 2) how do I do it? 🙂

Thx
Steve

[joshstechnij]

The CABs built by InstallShield use the only version of the CAB SDK that is available from Microsoft. The Windows Installer uses the copy of cabinet.dll installed with the OS to extract files from a CAB. I only recall one issue that cabinet.dll caused on certain versions of Windows 2000 without a specific security hotfix installed. Installing the hotfix resolve the problem (which caused MSI 1335 'cabinet is corrupt' errors). The version of MSI installed on the machine InstallShield is on has no impact on how CABs are built.

The only other possible causes of this behavior that I can think of would happen if the files were present when the component containing the dynamic file link is being built, but are then missing later during the build when the physical media (i.e. CABs) are being built, or, the source files are located on a network location and access to this location is sporadic to due network saturation (which we have seen at least one before).

Building multiple CABs is not possible with a compressed setup.exe release. InstallShield 2009 does include the ability to cap the size of individual CAB files in compressed setup.exe/MSI releases, which can be used to create multiple CABs in a compressed setup.

Can you attach a verbose build log and verbose install log?

[SteveEisner]

I've attached a build log (zipped to bring it down from 1mb) and setup log (zipped from 3mb.) If you search for "Data1.cab" in the setup log, you'll see the error; everything after that is just rollback from the failed install.

Thanks for looking into this

Steve

[joshstechnij]

There seems to be some inconsistency between the build log and the installation log. The build log indicates the file AskMeDB.dbp is an uncompressed source file and is copied as such to the source media:

Copying from "c:\perforce\drop\retail\Program Files\AC\AE\dbsetup\AskMeDB.dbp" to "c:\src\build\Setup\AE_SetupIS\AE Product\AESetup\DiskImages\DISK1\program files\AC\AE\dbsetup\AskMeDB.dbp"

However, the MSI log indicates this file is compressed (as indicated by the file record's attribute value 16384):

MSI (s) (34:00) [12:09:50:312]: Executing op: FileCopy(SourceName=AskMeDB.dbp,SourceCabKey=_48BB554EF4DA79662A55707578583682,DestName=AskMeDB.dbp,Attributes=16384,FileSize=4661,PerTick=32768,,VerifyMedia=1,,,,,CheckCRC=0,,,InstallMode=58982400,HashOptions=0,HashPart1=1701329203,HashPart2=-1493369878,HashPart3=153602589,HashPart4=-1794772898,,)
MSI (s) (34:00) [12:09:50:352]: File: C:\Program Files\AC\AE\dbsetup\AskMeDB.dbp; To be installed; Won't patch; No existing file
MSI (s) (34:00) [12:09:50:352]: Source for file '_48BB554EF4DA79662A55707578583682' is compressed

The problem then turns into a 2356 error because there is no CAB file named data1.cab streamed on the MSI.

The inconsistency between what these logs are saying is essentially impossible. The build code sets a file to be compressed or uncompressed depending upon the current release's settings (compressed build all files are compressed, mixed/custom build files are compressed based on their containing feature's compression settings in the release). If a file was is to be compressed, it is added to a CAB file during the latter part of the build process. The attached build log contains no CAB builder references, so this would indicate no CAB files are being built. In addition, once the compression is determined for a file, it does not change at any subsequent point in the build.

Based on this information, I would recommend trying to create a new product configuration and release with the Release Wizard as the current one may contain invalid combinations of release settings.

[SteveEisner]

Interesting. I'll go try to rebuild the project now...
but could this be a result of using the command-line builder? We use IsSABld.exe to do the build, and pass in flags that determine paths, etc.
Right now we're passing: -c UNCOMP -v -w -x -p
plus some -l paths.
Also the release is set as "Uncompressed". So I don't know where anything could have been set to Compressed (although we have in the past used -c COMP)
I'll also check to make sure that the setup.exe is actually ending up in the right place and that the one I run isn't an old, compressed version

Thanks for the hint, I'll keep looking

[joshstechnij]

You may also try passing:
-a "Product Configuration Name" -r "Release Name"

in place of the -c UNCOMP parameter (add the rest of your parameters). This will build the product config/release specified on the command line.