Microsoft Windows Installer Service attempts to preserve the security on objects that already exist on the system (
LockPermissions table for reference). You will have to make sure the permission is set on the topmost registry key so that when subkeys are created, they will inherit security descriptor of their parent registry key.
The other thing is to make sure the registry key with explicit permissions is created before any of the subkeys is created. If my memory serves me well, it's the same order as it appears in Registry table. Unfortunately, I don't have any known document on that.