Problem with a component and more than 32767 files

[lbhafen]

We have setups that have more than 32767 files in them, and have always had them get the warning -6651: The setup you are building contains more than 32,767 files. Automatically switching setup package to appropriate MSI schema.

We are now trying to build a new setup that has more than 32767 in a single component. When trying to build we get:

ISDEV : fatal error -5023: Error building table File

We have been able to work around this by copying:
...\Support\0409\IsMsiPkgLarge.itp

to
...\Support\0409\IsMsiPkg.itp

So we have surmised that the automatic switching the MSI schema has a bug in it that does not work for a single large component. I wanted to post, so that others would be aware of this, and find out if what we did was the proper workaround, or if there is something else to be done.

[Christopher_Pai]

You have 32K+ files in a single directory installed by a single component?


Really?

[lbhafen]

No, we have a single component that dynamically links a directory structure with 40,000 files or so spread out in it. The directory structure is not fixed, so defining a component at the top level and letting dynamic file linking handle the structure was how we decided to handle it.

[Christopher_Pai]

Components can only express files for a single directory. So if you are using dynamic linking you are actually emitting a number of components not just one.

So a better description of your problem would be problems using dynamic file linking with more then 32K files.

I'm not a huge fan of dynamic linking. Is the names of your files and directories changing much? I don't mean the contents but the actual names. I would avoid dynamic linking at all cost.

[lbhafen]

You are right, it is dynamic linking that has the problem. With about 100,000 files in the entire setup and no control over the files that come through from our documentation group, I am a fan of dynamic linking. There is no way we will switch from using dynamic file linking, so we will use this workaround until Flexera fixes dynamic linking.

[GarrettDyer]

Christopher Painter wrote:
I would avoid dynamic linking at all cost.

At all cost? Sounds pretty serious...

By the way, is there a succinctly-worded/linkable write-up of the perils of dynamic linking? We have been using it for *all* of our projects for many years and without it we'd had run up quite a hefty sum maintaining files at the whim of our product teams, so to hear it spoken of in such severe terms makes me think there's a time bomb ready to explode. Should I be worried about such a time bomb?

[Christopher_Pai]

I've been developing installers for nearly 15 years. As we say in Texas, this isn't my first rodeo. 🙂

In my experience, dynamic linking doesn't end well. Most simply put it's non-deterministic and that is never a good thing when it comes to Software Configuration Management. Put another way, when a file appears or disappears you can get a run-time failure of the application and I avoid that at all costs. I'd much rather have a build failure that tells me a new file has appeared that wasn't accounted for or a file disappeared that used to be in the install and wasn't accounted for.

I use an agile / collaborative process at work to make the people who develop the content also responsible for updating the installer. They are expected to consider the installer impact from their very first HLA and design documents all the way through to integration testing.

It's a very lean process and I've had no complaints. They used to beg for dynamic linking to "make it easy" but after I explained why we don't allow it they have accepted it.

I've written many times about it over the years. Here is one of them:

http://blog.deploymentengineering.com/2007/06/dealing-with-very-large-number-of-files.html