I want to call a Custom Action at end of install via a DoAction on the Finish button. Does the Custom Action have to be an MSI dll or could that be a standard dll as well?
It just has to be a custom action, so the Standard DLL custom action will work fine. There are some limitations to what a custom action can do when run from a Control Event, mostly with regards to logging, but I'm not aware of any limitations on the action type.
