multiple builds in parallel:error -6003: An error occurred streaming...into setup.exe

[np_install_user]

Hello,

Just upgraded to IS 2012 Spring hoping to benefit from the fixes which allow running multiple Standalone Build instances in parallel to improve our build time. Unfortunately we've encountered a problem.

The parallel building does work as long as we build projects which don't have any setup resources/prerequisite runtimes in common. If the projects do share some setup resources then they will periodically (~quarter of the time) fail with the following error :

ISDEV : error -6003: An error occurred streaming '[name of shared file here, for example a common bitmap image]' into setup.exe

We tracked the problem down to the Standalone Build app (IsCmdBld.exe version 19.0.0.160) opening resources for exclusive read. This prevents any other instance of IsCmdBld from reading the file while the first one is reading it. If it called CreateFile() with the FILE_SHARE_READ flag enabled for the dwShareMode parameter then this error would not occur. Since it is just reading the file it is not clear why exclusive mode would be necessary - perhaps it is not?

We've verified the source of the issue by using the sysutils Process Monitor tool to capture the "SHARING VIOLATION" errors which occur when more than one instance of IsCmdBld.exe tries to read a resource at the same time (the swShareMode parameter is set to "0" for exclusive access).


We'd really like to avoid duplicating things like runtime installers, bitmaps, etc for every project (there are ~8 projects which share a handful of resources).

We'd love to see Flexera fix this issue of course, but for now does anyone know of potential workarounds besides duplicating the setup resources?

Thanks

[MichaelU]

Hmm. I wonder if this comes down to MsiRecordSetStream possibly opening the file for exclusive read as you describe. I would have to trace the exact scenario to know for certain this is the API in question, but I think there's a high likelihood. If you can confirm that, we can start thinking of alternatives (like making a temporary copy of resources that need to be streamed into tables, though I don't like that idea much).

[np_install_user]

Thanks MichaelU


We have some additional information, plus a simple test harness that should allow you to reproduce the issue.


It appears the issue will only occur if compression is turned on for the release (and you have multiple installers trying to draw from common setup resource files). Interestingly, when compression is not enabled then a temporary, local copy of the file is created. The local copy behavior does not occur when compression is enabled.


Extract the attached zip file and run "build_installers.bat" to build the two parallel. They share a single common (large) bitmap for a splash screen.

* You may need to update this line at the start of the batch file

@rem point this var to your IsCmdBld.exe path, for example :
set _IS_IS2012BUILDLOC=c:\Program Files\InstallShield\2012Spring\System\IsCmdBld.exe

* And this line as well if you're on XP/2003

@rem throttle file checking to once every 10 seconds
sleep.exe 10 > NUL
@rem If on XP/2003 then sleep isn't included with the os, this can be used instead :
@rem npsleep.exe 10s > NUL

* The following project setting is required to trigger the error.
-> GUI -> Installation Designer
-> Releases -> Product config -> SINGLE_EXE_IMAGE
-> compression-> enable -> type -> LZX or MSZip

Without compression then IsCmdBld.exe will first create a local working copy of the desired file in :
\SINGLE_EXE_IMAGE\SINGLE_EXE_IMAGE\DiskImages\DISK1\


* The file behavior for streaming a common resource differs between the compression settings :

* No compression : Issue does not occur
1. (read source : shared) -> (write local copy)
2. (local copy) -> streamed to .msi

* Compression - MSZIP : Yes, issue occurs
1. (open file to read its attributes and close : non-shared)
2. (read source : non-shared) -> streamed to setup.exe

* Compression - LZX : Yes, issue occurs
1. (open file to read its attributes and close : non-shared)
2. (read source : non-shared) -> streamed to setup.exe

[np_install_user]

Michael,

Checking in to see if there have been any updates or findings on this issue since your original post. We would really like to utilize building in parallel, but we won't be able to do that easily without this issue resolved.

We look forward to your response,
Thanks

[MichaelU]

I hope to get to verifying this shortly (I was on vacation), but I can't promise any timeframe on a resolution. It definitely sounds odd that we would have different behaviors for different kinds of compression. That makes me suspect something other than a Binary table entry and MsiRecordSetStream; perhaps it's the difference between a file copy and a cabinet build, or something of that nature.

[MichaelU]

Thanks for the great test harness; it made this much easier to trace the problem. As such, I've filed an issue to track this (IOA-000073286) and hope we can schedule it soon.

[np_install_user]

We just upgraded to 2012 Spring Service Pack 1 which includes the fix for IOA-000073286. So far everything looks good with our testing, we'll try rolling it out in production soon.

Please pass our thanks along to all those at Flexera who contributed to the fix.