Is there any way that a setup program would check the Data Execution Prevention setting? If the setting is "Turn on DEP for all programs and services except those I select" it would automatically add my application to the list or display a message to the user that he needs to manually add the application to the list. Is that possible?
This isn't exactly answering your question, but rather providing thoughts...
Our application requires a similar step on some OS's, but when they asked me to do that, I pushed very hard to NOT automatically add the exe. I didn't like the idea of an installation routine creating holes in someone else's security without his say-so. We ended up putting some extra text in the InstallCompleteSuccess dialog to the effect that they would need to do this manually.
As an alternative, is there any way to prompt the user to add the app being installed to the DEP exceptions list?
I'd rather prompt to turn it off except for windows programs and services etc..., but I'd settle for being able to add my exe to the list if the user approves.
I highly recommend trying to get updated components that are not subject to DEP problems. However, if you have already been down that route and been ignored by the company's developers, or if it is not your code, then I guess you need to deal with DEP. I agree with you that when you are opening up a potential hole, you should ask the user before doing anything.
See the below link for a way to add a program to DEP using program compatibility settings in the registry. I haven't tried it myself, so I don't know if it works.
