Parallel build support

[LarryBoy]

Hi all,

I asked this question last year when evaluating IS 2011, and now I'm back with the same question for IS 2012:

Hi all,

Way back when, in the time of ISWI 2.0 and Developer 8, running more than one instance of the standalone build facility could/would cause indeterminate and generally bad behavior. Accordingly, I created a system-wide arbiter to ensure only one instance of SAB ran at a time.

Fast forward to IS 2011 (having migrated from Dev8 to IS 2011) -- can I scrap my "workaround" after more than 9 years? i.e. Is it possible now to run more than one instance of SAB at the same time? I would think this would be ever more critical now than it was back then, in these days of many-core servers...

Thanks in advance,
John

i.e. Can we, on our multi-core, super duper turbo boosted servers run more than one instance of IS 2012 standalone build at a time, or will the arbitrer I wrote more than 10 years ago live on?

Thanks,
John

[MichaelU]

While we strive to make this possible, it's really hard to prove the negative (that we don't have any race conditions that would prevent this). If it's easy for you to test without your workaround, give it a shot and see what you find.

[Christopher_Pai]

MichaelU is being modest. That and he probably can't commit to an answer to this question like I can.

I too have witnessed the concurrent build errors in years gone by and here is an observation that I previously had sent to an unnamed IS employee:

I've got a cheap $350 laptop here. Asus K52F-BBR5 Intel P6100 dual core x64 with 8GB of ram and whatever 250GB HDD they threw in it.

I have a Basic ISM with 44 Merge Modules that spits out 55 product configurations. Total output is 322MB at the moment. I created a batch
file to call the stand alone build 55 times and it took 10 minutes. Only one core seemed spiked. I split the .bat in half and it took 6 minutes and now both cores were pretty spiked.

Looks like the concurrent build is stable and if a machine had enough cores
and DISK I/O, I'm thinking concurrent performance would be a serious win.

[LarryBoy]

Hi Chris,

Thanks for your reply -- initial testing seems to indicate things working fairly well (three views building the same branch, started "simultaneously" on a 12 core system) -- so I have hope that I may be able to retire my arbitrer soon...

The one question remains whether I have to arbitrate projects that extract COM info during the build (see my other post) -- those are at least tiny projects, and serializing things there would have a negligible impact compared to my larger projects.

Thanks,
John

[Christopher_Pai]

IMO, I don't use COM Extract at Build unless I have a really, really good reason. I like the declarative, deterministic approach of MSI. I'd rather extract the COM once at design time, check it into source control and build / test.

When you do COM extract at build time, any number of things could go wrong and the result is non-deterministic behavior.

And of course, the .NET purist in me doesn't want to use COM at all these days. 🙂

[LarryBoy]

Yeah, I was forced into a bit of a corner with the extract at build time (there's only one project where I use it because of issues I had seen in the past)... however, based on your comment (and the unpleasantness I saw yesterday with the server crash), I went ahead and simply extracted the info to the MSI manually.

Thanks!
John

[BrHartmann]

I've been experimenting with parallel builds recently, but I've configured my projects to sign both setup.exe and msi (these are Installscript MSI projects), and even if I run multiple IsCmdBld instances in silent mode, one of the projects always gets stuck with the auto handling of the Private Key Password entry dialog (so one of those dialogs sits there waiting for password input which was apparently sent to the wrong window handle -- if Im running non-silent I see the password entered in the other cmd window after that build completes). Kind of a deal breaker for an automated build process.

Any suggestions? Would you recommend just turning off the signing during the build, and manually sign only the final setup.exe post-build (as a side question, why does IS need to sign 20 or so temp intermediate files when I configured signing the setup.exe and .msi file?) ? I'm not using "COM Extract at Build" in any components, but are there other areas where a race condition could corrupt the build (how about .NET COM Interop or .NET Scan at Build ?) ?

[MichaelU]

For any case but an OCI build (which requires spc+pvk) I would suggest converting your certificate to a pfx file. This uses a different (saner) method to specify the password, and I wouldn't expect that method to result in a race condition.