none
Upgrade to Windows 10 1709: IIS fails. RRS feed

  • Question

  • I have my little server running under Windows 10 with IIS 10 installed.

    After having installed Windows 10 Fall 1709, I could not start my site in IIS 10 anymore. After some research, the cause seems to be that the WAS-service (Windows Process Activation Service) will not start anymore. Trying to start that service manually results in an error 13 (data not valid). There is no further specification of which data is supposed to be invalid.

    After removing the 1709 installation, IIS worked perfectly once again.

    I would like to install 1709 on my server with IIS working.

    Can anybody help me with that?

    Monday, November 20, 2017 9:56 PM

Answers

  • Thank you so much for providing the logs.

    Unfortunately, I cannot reproduce the exact same issue on my computer - I tested the upgrade using both Win10 Enterprise and Windows Server 2016, and neither had this problem. So I'm not sure what is the unique condition to trigger it, and this needs further investigation. But I will definitely help you resolve the issue and unblock your usage.

    Here is what I suspect based on the log:

    The failure happened because WAS could not access the machine keys during startup. Upon first start after upgrade, WAS will try to create new machine keys if there isn't any, or access the old machine keys left from the old OS. In this case, there are old machine keys exist but WAS unfortunately cannot access them for some unclear reason. These machine keys are used to encrypt sensitive information in applicationHost.config or web.config (e.g. user password). WAS will not be able to start if there is no machine key it can use.

    Deleting these keys and letting WAS regenerating them could solve the problem with a side effect:

    Previously encrypted configuration (before the OS upgrade), if there is any, needs to be reconfigured and re-encrypted again, since WAS won't be able to decrypt the original ones (encrypted by the old keys) using the new keys.

    The following steps of deleting several machine keys used by IIS might help to allow WAS to start - WAS will simply recreate these keys upon start.

      • Go to your RSA machine keys folder: C:\Users\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys
      • Find a machine key (file) whose name starts with d6d986f09a1ee04e24c949879fdb506c_*. If you open it using notepad, you should see plain text "NetFrameworkConfigurationKey".
      • Backup this file to some other folder.
      • Delete this file.
      • Following the same steps as 2-4 to backup and delete iisWasKey: 76944fb33636aeddb9590521c2e8815a_*
      • Following the same steps as 2-4 to backup and delete iisConfigurationKey: 6de9cb26d2b98c01ec4e9e8b34824aa2_*
      • Manually start WAS
      • Open a command prompt through "run as administrator".
      • net start was

    • Edited by Yanbing Shi Monday, March 19, 2018 6:11 PM
    • Proposed as answer by Yanbing Shi Monday, March 19, 2018 6:33 PM
    • Marked as answer by McSnor Saturday, March 24, 2018 5:09 PM
    Monday, March 19, 2018 6:05 PM

All replies

  • Try the following:

    1. Open a Windows PowerShell window by using the Run as administrator option.
    2. Run the following commands:
       
      Stop-Service -Force WAS
      Remove-Item -Recurse -Force C:\inetpub\temp\appPools\*
      Start-Service W3SVC

    S.Sengupta,Microsoft MVP Windows and Devices for IT, Windows Insider MVP

    Monday, November 20, 2017 11:55 PM
  • Hi McSnor,

    Microsoft has published a fix about Web applications return HTTP Error 503 and WAS event 5189 on Windows 10 Version 1709. You could follow the steps in the link below to fix it.

    https://support.microsoft.com/en-us/help/4050891/error-http-503-and-was-event-5189-from-web-applications-on-windows-10

    Hope it will be helpful to you


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Tuesday, November 21, 2017 9:05 AM
    Moderator
  • Hi McSnor,

    Haven't received your message a few days, was your issue resolved?
    I am proposing previous helpful replies as "Answered". Please feel free to try it and let me know the result. If the reply is helpful, please remember to mark it as answer which can help other community members who have same questions and find the helpful reply quickly.
    Best regards,
    Carl


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Friday, November 24, 2017 1:20 PM
    Moderator
  • Hi McSnor,

    Any update?


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    Monday, November 27, 2017 2:40 PM
    Moderator
  • Hello,

    This doesn't appear to work.

    Remove-Item -Recurse -Force C:\inetpub\temp\appPools\*

    throws this error:

    Remove-Item : There is a mismatch between the tag specified in the request and the tag present in the reparse point
    At line:1 char:1
    + Remove-Item -Recurse -Force C:\inetpub\temp\appPools\*
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [Remove-Item], Win32Exception
        + FullyQualifiedErrorId : System.ComponentModel.Win32Exception,Microsoft.PowerShell.Commands.RemoveItemCommand

    Thursday, November 30, 2017 4:20 PM
  • Same issue app running in IIS on Windows 10 upgrade to 1709.

    Windows Process Activation Service cannot start Error 13 dialog box, this trigger appspools not works at all. I cannot run nothing even the default web site

    Uninstall IIS, reinstall IIS did not work

    Uninstall IIS & WPAS service from windows features did works!

    But my app does not work now. I need to redeploy again


    MCP soon MCDBA, DCE 2 Stars, DCE 2k5 2 Stars, Technet 4 Stars

    Friday, January 12, 2018 2:59 PM
    1. Open a Windows PowerShell window by using the Run as administrator option.
    2. Run the following commands: 
      Stop-Service -Force WAS
    3. Manually remove files under "C:\inetpub\temp\appPools\"
    4. Go to power shell prompt and run command: Start-Service W3SVC

    If you perform above steps, your issue should be resolved.


    Tuesday, January 16, 2018 6:14 AM
  • Hi, I have the same problem with my IIS after upgrading my Windows to the 1709 version.
    I tried the powershell commands in a administrator prompt, but I'm not able to start W3SVC service and also WAS service.
    In my case, the directory "C:\inetpub\temp\appPools\" didn't have any files or symbolic links before and after the upgrade to 1709. Also, one of my website is not in the C:\inetpub\wwwroot folder but in another folder in my system drive.

    When I start the W3SVC service, I have the following errors (XML View because my computer is in french) with the Event IDs:

    Event ID 5215, WAS

    <Event xmlns="">
    <System>
      <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
      <EventID Qualifiers="49152">5215</EventID>
      <Version>0</Version>
      <Level>2</Level>
      <Task>0</Task>
      <Opcode>0</Opcode>
      <Keywords>0x80000000000000</Keywords>
      <TimeCreated SystemTime="2018-01-17T02:41:59.856976500Z" />
      <EventRecordID>726</EventRecordID>
      <Correlation />
      <Execution ProcessID="0" ThreadID="0" />
      <Channel>System</Channel>
      <Computer>Katana2</Computer>
      <Security />
      </System>
    <EventData>
      <Binary>0D000780</Binary>
      </EventData>
      </Event>

    Event ID 5005, WAS

    <Event xmlns="">
    <System>
      <Provider Name="Microsoft-Windows-WAS" Guid="{524B5D04-133C-4A62-8362-64E8EDB9CE40}" EventSourceName="WAS" />
      <EventID Qualifiers="49152">5005</EventID>
      <Version>0</Version>
      <Level>2</Level>
      <Task>0</Task>
      <Opcode>0</Opcode>
      <Keywords>0x80000000000000</Keywords>
      <TimeCreated SystemTime="2018-01-17T02:41:59.857478200Z" />
      <EventRecordID>727</EventRecordID>
      <Correlation />
      <Execution ProcessID="0" ThreadID="0" />
      <Channel>System</Channel>
      <Computer>Katana2</Computer>
      <Security />
      </System>
    <EventData>
      <Binary>0D000780</Binary>
      </EventData>
      </Event>

    Event ID 7001, Service Control Manager

    <Event xmlns="">
    <System>
      <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" />
      <EventID Qualifiers="49152">7001</EventID>
      <Version>0</Version>
      <Level>2</Level>
      <Task>0</Task>
      <Opcode>0</Opcode>
      <Keywords>0x8080000000000000</Keywords>
      <TimeCreated SystemTime="2018-01-17T02:42:00.858328900Z" />
      <EventRecordID>729</EventRecordID>
      <Correlation />
      <Execution ProcessID="940" ThreadID="9964" />
      <Channel>System</Channel>
      <Computer>Katana2</Computer>
      <Security />
      </System>
    <EventData>
      <Data Name="param1">Service de publication World Wide Web</Data>
      <Data Name="param2">Service d'activation des processus Windows</Data>
      <Data Name="param3">%%13</Data>
      <Binary>570033005300560043000000</Binary>
      </EventData>
      </Event>

    Event ID 7023, Service Control Manager

    <Event xmlns="">
    <System>
      <Provider Name="Service Control Manager" Guid="{555908d1-a6d7-4695-8e1e-26931d2012f4}" EventSourceName="Service Control Manager" />
      <EventID Qualifiers="49152">7023</EventID>
      <Version>0</Version>
      <Level>2</Level>
      <Task>0</Task>
      <Opcode>0</Opcode>
      <Keywords>0x8080000000000000</Keywords>
      <TimeCreated SystemTime="2018-01-17T02:42:00.858328900Z" />
      <EventRecordID>728</EventRecordID>
      <Correlation />
      <Execution ProcessID="940" ThreadID="6468" />
      <Channel>System</Channel>
      <Computer>Katana2</Computer>
      <Security />
      </System>
    <EventData>
      <Data Name="param1">Service d'activation des processus Windows</Data>
      <Data Name="param2">%%13</Data>
      <Binary>5700410053000000</Binary>
      </EventData>
      </Event>

    I know if i rollback the Windows version to 1703, my IIS will be working, but I want to find the problem and to be at the last version :).

    Is there another solution to this problem?

    Thanks in advance,

    Jonathan A.
    Wednesday, January 17, 2018 3:20 AM
  • Hello,

    Same issue.

    Have a good day

    • Proposed as answer by Paweł _Poland Thursday, January 18, 2018 8:31 PM
    • Unproposed as answer by Paweł _Poland Thursday, January 18, 2018 8:31 PM
    Wednesday, January 17, 2018 10:21 AM
  • Is there a solution for running IIS
    I've done all the steps but I still can not run WAS and IIS.

    Reinstall IIS does not help.
    Can someone help or indicate repair steps.

    Version 1709(16299.192)

    Friday, January 19, 2018 8:56 PM
  • How to clear all information about IIS from your computer. This can help you install this service correctly. However, this service depends on the WAS service and I have to fix it too. Returns error 5215 or 5005
    Friday, January 19, 2018 9:22 PM
  • I was having the same issue after a windows update I installed 2 days ago, I tried running all of the commands in the replies to this as well as utilizing old config files instead of the current applicationhost etc. ones. The only thing that worked for me was rolling back to a previous version of windows (just back prior to the updates I installed). I submitted the issue to Microsoft and hopefully the next update includes a fix for whatever they broke. As soon as I rolled back my windows updates all of the errors (exact same you were seeing) went away and things on my machine are working as expected again.
    Friday, January 26, 2018 12:19 AM
  • I have the very same issue. Cannot start WAS after the recent windows update. Any help would be very useful.
    Tuesday, February 13, 2018 11:11 PM
  • I'm having the same issue, Scott Hanselman managed to fix it some time ago, https://www.hanselman.com/blog/FixedWindowsProcessActivationServiceWASIsStoppingBecauseItEncounteredAnError.aspx

    but the bug seems to have "evolved". I cleared the inetpub\temp\appPools folder, gave it permissions as noted here:

    https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc734935(v=ws.10)

    but I still cannot get WAS to start: it now says "The Windows Process Activation Service service terminated with the following error:
    The data is invalid." Who writes these error messages?

    So now I'm using IIS Express, which is ok.

    Thursday, March 1, 2018 5:23 PM
  • I have the same problem.

    Funny (in an ironic sort of way) that we're advised to Stop-Service -Force WAS when WAS isn't even running, and that's the whole problem!  It won't start due to Error 13, which is not resolved by copying over older files.

    We're also advised to delete temporary AppPool files, which also has nothing to do with WAS Error 13

    Proof: go to C:\Windows\System32\inetsrv\Config and move the applicationHost.config file to your Desktop, then try to start WAS again - you'll get "Error 2: The system cannot find the file specified."  Now undo the move, and when you try to start WAS, you'll get "Error 13: The data is invalid" again.

    Thus, the problem is some conflict between WAS and applicationHost.config, since everything is fine before the update, but not after.  Either the Update is "broken" because it is not changing applicationHost.config to match the changes made to the system, or WAS itself is broken in the update.


    • Edited by rickwict Tuesday, March 13, 2018 4:48 PM
    Tuesday, March 13, 2018 4:48 PM
  • Carl,

    With all due respect, I don't think a reply is "helpful" when it includes instructions to stop a service that isn't even running and was listed in the OP as not even being startable (OP wrote: "After some research, the cause seems to be that the WAS-service (Windows Process Activation Service) will not start anymore. Trying to start that service manually results in an error 13 (data not valid)." (emphasis added) 

    Is there any internal MS information on how to fix the applicationHost.config file that does NOT include copying from a backup?  When rolling back the 1709 update causes IIS to work without restoring from a backup, then it doesn't appear to be the .config file's fault, at least not in isolation.




    • Edited by rickwict Tuesday, March 13, 2018 9:31 PM
    Tuesday, March 13, 2018 4:59 PM
  • Hi, after some researchs in the web about the problem, i found something interesting to solve the problem of the WAS-service: https://serverfault.com/questions/891302/windows-process-activation-service-failing-windows-10 

    On my side, I made these steps to resolve my problem:

    • I took a backup of files of IIS in the C:\windows\system32\inetsrv\ in case of rollback. I used a PowerShell elevated prompt to made the backup.

    • I went to the Windows features to uncheck all things related to IIS (I took screenshots of the options I check before that).

    • I reboot my computer after that.

    • I went to the Windows features to uncheck all things related to Windows Process Activation Service.

    • I reboot my computer after that.

    • I deleted all the content of the "C:\windows\system32\inetsrv\". In my case, I took the ownership of the directory "inetsrv" with my running user to be able to delete all the files. Also, I stopped the running services using the files in the directory (in my case "Application Host Helper Service") to be able to delete all the files.

    • I deleted the inetsrv directory to not keeping the ownership of my user.

    • I went to the Windows features to check all things related to IIS (In my case I rechecked all my settings took in screenshots).

    • I checked if the default web site is working (in my case, Yes :) ), put my configuration files before to retrieve my settings and restarted the web server to check if it's working. In my case, it's worked.

    • I reboot my computer to check if the IIS is working (also working).

    • I went to the Windows features to recheck all things related to Windows Process Activation Service.

    • I reboot my computer after that.

    • I tested my IIS and checked if the WAS-Service is working. On my side, all is working fine.

    I don't know if it will resolve all problems related to the WAS-Service with the message "data is invalid", but in my case it was my solution.

    Jonathan A.

    Thursday, March 15, 2018 1:09 AM
  • could you please check whether registry key HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WAS\Parameters exits?

    This posting is provided "AS IS" with no warranties, and confers no rights

    Friday, March 16, 2018 5:32 PM
  • First, apologize for not seeing this 1709 IIS upgrade issue before. I'm from IIS team, and we didn't receive any report on such failure until I happened to see this discussion. I can fully understand all your complaints and concerns raised here for the issue.

    I believe the issue of WAS start failure with "Error 13: The data is invalid" along with event 5215 is a DIFFERENT one from the issue that can be resolved by the KB4050891 (HTTP Error 503 and WAS event 5189 due to symbolic links).

    The info Jonathan_Auger and Pawel _Poland provided are particularly interesting - the WAS event 5215.

    Here are the info I'd like to gather from you to further narrow down the issue:

    1. Check whether you have this registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WAS\Parameters\NanoSetup

    2. Check whether you have a file named applicationhost.config.tmp under C:\windows\system32\inetsrv\config (the folder where your applicationHost.config sits).

    3. Manually start WAS:

    - Launch cmd through "run as administrator"

    - Net start WAS (it should fail again with error code 13)

    Now open c:\windows\iis.log, scroll down to the end of the file, and search the most recent errors in the log (whose time stamp should correlate to the time you run "net start WAS". We appreciate if you could paste any error log message here.

    4. Does your server happen to have any Broadcom hardware/driver/service installed? In particular, could you check if you have bcmihvsrv64.dll or bcmihvsrv.dll under c:\windows\system32? I know this is an ODD request for diagnosing an IIS upgrade failure, but there is a known issue that Broadcom service can cause IIS WAS service fail to create CNG key, and hence fails to start.

    Thanks,

    Yanbing



    • Edited by Yanbing Shi Friday, March 16, 2018 11:56 PM update
    Friday, March 16, 2018 10:50 PM
  • 1. Check whether you have this registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WAS\Parameters\NanoSetup

    Yes, it is a REG_DWORD value 1

    2. Check whether you have a file named applicationhost.config.tmp under C:\windows\system32\inetsrv\config (the folder where your applicationHost.config sits).

    No, though I may have deleted this following previous forum instructions.

    3. Manually start WAS:

    - Launch cmd through "run as administrator"

    - Net start WAS (it should fail again with error code 13)

     Yes it failed.

    Now open c:\windows\iis.log, scroll down to the end of the file, and search the most recent errors in the log (whose time stamp should correlate to the time you run "net start WAS". We appreciate if you could paste any error log message here.

    Here is the log:

    [03/19/2018 10:12:51] [ ***** IIS 10.0 Component Based Setup ***** ]
    [03/19/2018 10:12:51] .\inetsrv\iissetup.exe  /install SharedLibraries /nano
    [03/19/2018 10:12:51] Setting Installation Type to Nano
    [03/19/2018 10:12:51] Successfully added IIS_IUSRS ACE to DACL at %ProgramData%\Microsoft\Windows\WER\ReportQueue.
    [03/19/2018 10:12:51] < !!FAIL!! > Failed to create the NetFrameworkConfigurationKey key container (result=0x8009000f)
    [03/19/2018 10:12:51] < !!FAIL!! > Install of component SharedLibraries result=0x8009000f
    [03/19/2018 10:12:51] < !!FAIL!! > COMPONENT::ExecuteCommand result=0x8009000f
    [03/19/2018 10:12:51] [ End of IIS 10.0 Component Based Setup ]

    4. Does your server happen to have any Broadcom hardware/driver/service installed? In particular, could you check if you have bcmihvsrv64.dll or bcmihvsrv.dll under c:\windows\system32?

    No, my main workstation does not have any Broadcom hardware. My Surface does, but that seems to be OK, the Broadcom chipset has caused other problems in the past, now resolved.

    I'm happy to help diagnose this, I can provide dumps etc, but not publically on the forum. 

    Monday, March 19, 2018 10:23 AM
  • Thank you so much for providing the logs.

    Unfortunately, I cannot reproduce the exact same issue on my computer - I tested the upgrade using both Win10 Enterprise and Windows Server 2016, and neither had this problem. So I'm not sure what is the unique condition to trigger it, and this needs further investigation. But I will definitely help you resolve the issue and unblock your usage.

    Here is what I suspect based on the log:

    The failure happened because WAS could not access the machine keys during startup. Upon first start after upgrade, WAS will try to create new machine keys if there isn't any, or access the old machine keys left from the old OS. In this case, there are old machine keys exist but WAS unfortunately cannot access them for some unclear reason. These machine keys are used to encrypt sensitive information in applicationHost.config or web.config (e.g. user password). WAS will not be able to start if there is no machine key it can use.

    Deleting these keys and letting WAS regenerating them could solve the problem with a side effect:

    Previously encrypted configuration (before the OS upgrade), if there is any, needs to be reconfigured and re-encrypted again, since WAS won't be able to decrypt the original ones (encrypted by the old keys) using the new keys.

    The following steps of deleting several machine keys used by IIS might help to allow WAS to start - WAS will simply recreate these keys upon start.

      • Go to your RSA machine keys folder: C:\Users\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys
      • Find a machine key (file) whose name starts with d6d986f09a1ee04e24c949879fdb506c_*. If you open it using notepad, you should see plain text "NetFrameworkConfigurationKey".
      • Backup this file to some other folder.
      • Delete this file.
      • Following the same steps as 2-4 to backup and delete iisWasKey: 76944fb33636aeddb9590521c2e8815a_*
      • Following the same steps as 2-4 to backup and delete iisConfigurationKey: 6de9cb26d2b98c01ec4e9e8b34824aa2_*
      • Manually start WAS
      • Open a command prompt through "run as administrator".
      • net start was

    • Edited by Yanbing Shi Monday, March 19, 2018 6:11 PM
    • Proposed as answer by Yanbing Shi Monday, March 19, 2018 6:33 PM
    • Marked as answer by McSnor Saturday, March 24, 2018 5:09 PM
    Monday, March 19, 2018 6:05 PM
  • 1. Check whether you have this registry key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WAS\Parameters\NanoSetup

    Yes, Dword, value = 1

    2. Check whether you have a file named applicationhost.config.tmp under C:\windows\system32\inetsrv\config (the folder where your applicationHost.config sits).

    No.

    3. Manually start WAS:

    - Launch cmd through "run as administrator"

    - Net start WAS (it should fail again with error code 13)

    It does :)

    Now open c:\windows\iis.log, scroll down to the end of the file, and search the most recent errors in the log (whose time stamp should correlate to the time you run "net start WAS". We appreciate if you could paste any error log message here.

    [03/19/2018 16:52:07] [ ***** IIS 10.0 Component Based Setup ***** ]
    [03/19/2018 16:52:07] .\inetsrv\iissetup.exe  /install SharedLibraries /nano 
    [03/19/2018 16:52:07] Setting Installation Type to Nano
    [03/19/2018 16:52:07] Successfully added IIS_IUSRS ACE to DACL at %ProgramData%\Microsoft\Windows\WER\ReportQueue.
    [03/19/2018 16:52:07] Created NetFrameworkConfigurationKey key containter
    [03/19/2018 16:52:08] Created NetFrameworkConfigurationKey user key
    [03/19/2018 16:52:08] Set ACLs on NetFrameworkConfigurationKey
    [03/19/2018 16:52:08] Created iisWasKey key container
    [03/19/2018 16:52:08] Created iisWasKey user key
    [03/19/2018 16:52:08] Created iisConfigurationKey key container
    [03/19/2018 16:52:08] < !!FAIL!! > Failed to generate iisConfigurationKey user key (result=0x80070005)
    [03/19/2018 16:52:08] < !!FAIL!! > Install of component SharedLibraries result=0x80070005
    [03/19/2018 16:52:08] < !!FAIL!! > COMPONENT::ExecuteCommand result=0x80070005
    [03/19/2018 16:52:08] [ End of IIS 10.0 Component Based Setup ]

    4. Does your server happen to have any Broadcom hardware/driver/service installed? In particular, could you check if you have bcmihvsrv64.dll or bcmihvsrv.dll under c:\windows\system32? I know this is an ODD request for diagnosing an IIS upgrade failure, but there is a known issue that Broadcom service can cause IIS WAS service fail to create CNG key, and hence fails to start.

    No Broadcom devices here.

    Monday, March 19, 2018 10:09 PM
  • In response to my own post, I re-removed the iisConfigurationKey file and when I tried "net start was", it worked! I was then able to start IIS.

    THANK YOU for getting me on the right track with the MachineKeys and the iis.log file!
    Monday, March 19, 2018 11:18 PM
  • Glad the workaround unblocks you. But the root cause isn't clear yet.

    Definitely something related to the machine keys was messed up during the upgrade (and IIS has no control on it). The old machine keys were created by WAS before the upgrade, and Windows upgrade carries these keys to the new OS. However, it is completely unexpected that WAS running under SYSTEM account would fail to access the machine keys previously created by itself.

    If anyone in this thread followed my workaround steps and backup the three old RSA machine keys:

    NetFrameworkConfigurationKeyd6d986f09a1ee04e24c949879fdb506c_*

    iisWasKey: 76944fb33636aeddb9590521c2e8815a_*

    iisConfigurationKey: 6de9cb26d2b98c01ec4e9e8b34824aa2_*

    where "*" is a machine-specific GUID.

    I would appreciate if you could provide me the following info:

    If you compare each old key (the backup) with the new one recreated by WAS, do they have the same name - e.g. whether the "*" part (GUID) are the same?

    In addition, if the machine happens to be a VM with a checkpoint before the upgrade (I know the probability is small), could you provide the OS version number before and after upgrade. The detailed version number can be found from the following registry:

    HKLM\Software\Microsoft\Windows NT\CurrentVersion\BuildLabEx

    Please also provide the OS Edition (e.g. Home, Professional, Enterprise etc)

    HKLM\Software\Microsoft\Windows NT\CurrentVersion\CompositionEditionID

    • Edited by Yanbing Shi Tuesday, March 20, 2018 6:51 AM
    Tuesday, March 20, 2018 6:35 AM
  • Forgive my silly question: did you mean the folder C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys ?
    I do have a file, d6d986f09a1ee04e24c949879fdb506c_43cd7ff9-a310-48a1-9cd7-bfd2e2aeb254.sys
    but I'm unable to open it, I get a permissions error, which seems to fit with the scenario you describe.
    I right button clicked, viewed security, it said I could take ownership, so I re-assigned it to the local administrator. Still couldn't move it to a backup folder, The Advanced security settings dialog said it was unable to display the current owner, I added myself as a new principal and gave it full control, same for the other two. I was then able to move all three files to a backup folder.
    The security settings for each of those files was IIS_IUSRS - Read and WMSVC - Read
    I then ran cmd.exe as admin, NET START WAS and it succeeded.
    I checked the machine specific guid part of the filename, it was the same as the old one.
    I was able to start the WWW Publishing service, and launch IIS - default website is now running.
    The version numbers from the registry:
    BuildLabEx 16299.15.amd64fre.rs3_release.170928-1534
    CompositionEditionID Professional
    Here is the relevant section from IIS.Log
    [03/20/2018 10:12:53] [ ***** IIS 10.0 Component Based Setup ***** ]
    [03/20/2018 10:12:53] .\inetsrv\iissetup.exe  /install SharedLibraries /nano
    [03/20/2018 10:12:53] Setting Installation Type to Nano
    [03/20/2018 10:12:53] Successfully added IIS_IUSRS ACE to DACL at %ProgramData%\Microsoft\Windows\WER\ReportQueue.
    [03/20/2018 10:12:53] Created NetFrameworkConfigurationKey key containter
    [03/20/2018 10:12:53] Created NetFrameworkConfigurationKey user key
    [03/20/2018 10:12:53] Set ACLs on NetFrameworkConfigurationKey
    [03/20/2018 10:12:53] Created iisWasKey key container
    [03/20/2018 10:12:53] Created iisWasKey user key
    [03/20/2018 10:12:53] Created iisConfigurationKey key container
    [03/20/2018 10:12:53] Created iisConfigurationKey user key
    [03/20/2018 10:12:53] Set ACLs on iisConfigurationKey
    [03/20/2018 10:12:53] Found AesProvider and skipping creation
    [03/20/2018 10:12:53] Found IISWASOnlyAesProvider and skipping creation
    [03/20/2018 10:12:53] Failed to create iisCngConfigurationKey key container (result=0x8009000f)
    [03/20/2018 10:12:53] Opened existing iisCngConfigurationKey key container
    [03/20/2018 10:12:53] iisCngConfigurationKey key container already exists
    [03/20/2018 10:12:53] Generated (or already exists) IIS CNG Configuration Key Container
    [03/20/2018 10:12:53] Failed to create iisCngWasKey key container (result=0x8009000f)
    [03/20/2018 10:12:53] Opened existing iisCngWasKey key container
    [03/20/2018 10:12:53] iisCngWasKey key container already exists
    [03/20/2018 10:12:53] Generated (or already exists) IIS CNG WAS only Key Container
    [03/20/2018 10:12:53] Install of component SharedLibraries succeeded!
    [03/20/2018 10:12:53] Success!
    [03/20/2018 10:12:53] [ End of IIS 10.0 Component Based Setup ]
    Let me know if you need anything else, I'm up and running now, thanks. I'll reboot and make sure everything starts.
    Tuesday, March 20, 2018 10:34 AM
  • I checked each key and the backups match the newly-created ones.  Each backup and new file pair is also the same size (2.16K or 2.17K, depending on the file).

    The machine is not a VM.

    Investigating what "LegacyOfHerot" wrote about the folders, I have matching keys in both the C:\Users\All Users\Application Data\Microsoft\Crypto\RSA\MachineKeys folder and the C:\ProgramData\Microsoft\Crypto\RSA\MachineKeys folder, with the same date/time modified.  Is one folder a mirror of the other?


    • Edited by rickwict Tuesday, March 20, 2018 5:07 PM
    Tuesday, March 20, 2018 4:02 PM
  • This may not be a new issue.  https://forums.iis.net/p/1148509/1865753.aspx
    Tuesday, March 20, 2018 5:09 PM
  • Many thanks for all the information provided by both of you.

    As rickwict pointed out, these two folders are actually mapped to the same one :)

    It is expected that the GUID part of the old/new machine key file name are the same, so we eliminate the possibility that the upgrade messed up this part. It is also expected that you have to take full control in order to move it.

    If the GUID part remains the same, then I suspect the ACL got messed up.

    The info LegacyOfHerot provides is interesting. Could you compare the ACLs of the old keys VS new ones by checking the "Security" tab in the "Properties" window? May need to click the "Advanced" button in "Security" tab.

    You mentioned that the old one only has [IIS_IUSRS - Read] and [WMSVC - Read]. I was surprised it doesn't have [SYSTEM - Full control]. Could you double check?


    • Edited by Yanbing Shi Tuesday, March 20, 2018 6:33 PM
    Tuesday, March 20, 2018 6:26 PM
  • Bug hunt...

    OK, so my machine doesn't have a C:\Users\AllUsers folder - may not be relevant.

    The old versions definitely DONT have SYSTEM - Full control, just the two mentioned above.

    The new versions do: as in image below:

    Hope this helps.

    Tuesday, March 20, 2018 7:13 PM
  • This is the problem, and no wonder WAS cannot access these keys upon startup - SYSTEM account is not even in ACL.

    Although these machine keys were created and used by IIS, it was the Windows upgrade that carried all the keys in old OS to the new OS. Definitely the ACL was somehow messed up during this process - I don't think you guys manually changed any property of the keys, right? (Likely you are not even aware the existence of them before this issue). But the exact reason is still mysterious at this point.

    We will escalate this issue to the partner team. Meanwhile, if anyone in this thread (or anyone sees this thread in the future) has the information of the old OS version number, providing it to us could help further narrow down the condition to reproduce this issue, which will be critical for further investigation.

    We will also make some efforts to provide more informative and accurate event log and error message for such error condition.
    • Edited by Yanbing Shi Wednesday, March 21, 2018 4:54 AM
    Wednesday, March 21, 2018 4:48 AM
  • Glad we have a solution, even if we dont have the root cause. More diagnostic information would be good: "The data is invalid" -really?

    Taking a step back, though, this is another example of just how complex and labyrinthine Windows has become.

    Take IIS authentication for a corporate web site - first we had Forms & Windows authentication - simple. Then along came ADFS  and Fedutil. Fedutil was dropped, and we had to try and build the entries in web.config by comparing with another site. Now we have Owin, and we need 30 or 40 classes/files/nuGet packages just to bounce off some third party ID provider. Edge doesn't even cope with a site using Windows auth - it always presents the 401 login dialog. IE 11 on the same box is fine (yes I've raised that "feature request" - as have many others).

    I've been developing on the Windows platform for a decade or three, and I couldn't fix this by myself, shame on me. I do think the OS needs a dose of simplication and addition of lightness.

    Oh well, back to debugging my other issue - the Visual Studio team have decided packages.config is no longer flavour of the month, and we have to migrate our solution to package references. No migration utility provided, so its time to poke and fiddle in the csproj files.

    Wednesday, March 21, 2018 10:16 AM
  • Hello Yanbing Shi.

    I'm sorry for my very late reaction. The reason is that I decided to undo my Windows 1609 update for I simply had to activate my server instantaneously.

    Yesterday, Windows 10 surprised me with an unnattended upgrade with the same error occurring again. This time however there was some time to investigate.

    Your solution worked very good although I had a hard time deleting the files you mentioned (not allowed, no owner, ...). Thank you for that; I wouldn't have guessed a solution like that.

    I still find it very strange that an upgrade error like this still has not been solved in months.

    Thanks a lot.

    McSnor

    Saturday, March 24, 2018 5:16 PM
  • This thread was a huge help in solving this issue.  And it was this insight about the ACLs that makes the solution easier than the Answer that is marked for this thread.  The marked Answer gives instructions to remove the three relevant keys.  This is not actually necessary.  You can simply update the "Security" config of each of these three keys by adding SYSTEM to the ACL list and giving it "Full Control".  After that, the WAS service able to be started with no problem (and no more of that vague "error 13 (data not valid)" message).  With WAS started, IIS could now start.  Too bad the info in this thread wasn't around a few months ago when my computer decided to update to build 1709 (and cause problems). I had to do a rollback to get IIS working. But then my computer recently all by itself eventually re-updated back to build 1709, so I had no choice but to continue to research a solution. Fortunately this time, this thread was here to help.
    Tuesday, March 27, 2018 3:53 PM
  • Right, granting SYSTEM to the ACL of the machine keys will also work.

    The reason I suggested delete all three RSA machine keys is because at that time we didn't know that the root cause was the ACL :) Accessing machine key is NOT as simple as file access. There are many possibility that can lead to machine key access failure - ACL is just one of them. Once the root cause is clear, the work around of granting ACL looks simpler.


    Yanbing Shi


    • Edited by Yanbing Shi Thursday, March 29, 2018 7:52 AM
    Thursday, March 29, 2018 7:51 AM
  • Hi, I had the same problem on 19-12-2017, when I updated my Windows 10 Pro to 1709. I also went back to 1703 with 'Reset my PC'. Yesterday, I was again updated to 1709, and had the same problem again. I could solve it now by adding SYSTEM to the ACL of these three files. I found the files in:
    c:\ProgramData\Microsoft\Crypto\RSA\MachineKeys\
    I first had to take ownership of each file.

    I have two identical laptops, where IIS was installed on exactly the same way, and only one of them had this problem.

    Thanks!

     
    • Edited by Zeerob Sunday, April 1, 2018 7:18 PM
    Sunday, April 1, 2018 7:00 PM
  • I am just adding the IIS error messages, so that this solution can be found easier.

    Websites cannot be started unless both the Windows Activation Service (WAS) and the World Wide Web Publishing Service (W3SVC) are running. Both services are currently stopped.

    Cannot start service W3SVC on computer '.'.


    • Edited by Zeerob Monday, April 2, 2018 9:47 AM
    Monday, April 2, 2018 9:46 AM