I have been investigating a setup crash issue on one of our Windows Server 2012 R2 servers. The affected patient is an InstallShield 6 setup from a Siemens SCADA software.
We had a suspicion that EMET might be involved from the get-go thus uninstalling EMET from the impacted server. The crash, however, has been remaining thus leading me astray to suspect other policy/security settings to be in play.
After a long and tedious research on a lab VM using snapshots and step-by-step approaches I've been able to find the culprit:
Telling EMET (by GPO for example) to use System DEP in opt-out mode (which would also be the Server OS default setting) two things happen:
1. Expected: The system control panel DEP setting is enforced to be on for all programs/services (aka BCD NX OptOut option)
2. Unexpected: EMET service switches the last four bits of "HKLM\System\CurrentControlSet\Control\Session Manager\Kernel\MitigationOptions" from 2 to 1 (0b0001).
Thus on operating systems of kind Windows 8.1+ the kernel-level DEP (as part of Control Flow Guard, CFG) will be enabled, too. However,
MitigationOptions will not be reset on uninstall of EMET! Thus even when turning off DEP the old way (System Control Panel/BCD NX) you will still have DEP enabled on an OS level. Even the System Control Panel option does only change the BCD
NX option but leaves the kernel settings alone.
The CFG/kernel DEP seems to impose stricter checks because with BCD NX at OptOut but CFG DEP at OptIn the InstallShield setup still works. But as soon as CFG DEP is at OptOut and the server restarted the setup fails referring to IKernel.exe.
To some extent I can see the reasoning behind a restricted manipulation of the CFG Registry options but this is a hell-of-a-trap on earlier OS' that do not yet have manageability/PowerShell cmdlets for CFG! And I couldn't find any mention of this specific
behavior in the EMET 5.52 User Guide.
Thanks go to 0xdabadd00's
EMET 4.1 Uncovered Guide which explains the structure of MitigationOptions in detail and more.
According to
https://blog.ensilo.com/documenting-the-undocumented-adding-cfg-exceptions there should already be an infrastructure for CFG exclusions in place - but not exposed/no
UI in OS like Windows Server 2012 R2?
Thus if you want to revert back to the OS install defaults of NX OptOut but CFG DEP set to OptIn you can only resort to manipulating the Registry directly (and best only manipulate the bits required). In the EMET GPO you can only set System DEP back to "Not
configured" because any other option would manipulate the BCD setting and MitigationOptions in unfavorable ways.
As an aside there is still no way (or not a way I know of) to exclude such subprocess/DLL calls like InstallShield's setup.exe does from DEP. I tried adding IKernel.exe to the exclusion list with no success.
And to conclude this journey to security wonderland - CFG DEP also overrides the BCD NX setting. Even having BCD NX set to AlwaysOff CFG DEP set to OptOut still works and crashes the setup.
Does CFG replace the NX option? Is there even DEP active when only the NX option is set accordingly? Intriguing questions.
Anyone being able to elaborate more on this/give some backgound would be welcome. The workaround and situation is clear now but the whole interoperability and side effects are awkward.
