none
MDT 2013 - Windows 10 Deployment Fails Randomly - Incorrect Function RRS feed

  • Question

  • Hello,  we've been having issues with deploying Windows 10 since it first became available.  It's becoming increasingly problematic as more users request Windows 10, deployments only succeed about 50% of the time.  The other 50% of the time, the task sequence fails at random points during the State Restore phase.  

    All that is happening during the state restore is pretty much the norm, some applications are set to install, it sets some variables, joins the domain and restarts.  Windows 7 deployments have no issues with these tasks, Windows 10 is highly unpredictable.  Currently the only solution I have for the help desk guys is to reimage the machine until it succeeds.  

    Now when it fails, the only actual 'error' I can find in any logs is in the smsts.log.  I decided to make every single step in the State Restore phase of the task sequence continue on error so I can get a better idea of what it is doing.  It'll start the State Restore phase normally, but after several steps in it'll skip a whole lot of steps, then it'll successfully do other steps, then skip some more.  Each machine I've seen succeeds and skips completely different steps, and they all skip with the error of 'Incorrect Function'.  Any type of step will fail with this error also.  Though since most of the steps are for application installs, they apply mostly to those, but it also will randomly fail on 'run command line' steps, and also 'set task sequence variable' steps.  

    For testing purposes I have been reimaging a Dell Latitude E7440 and Latitude E7450.  MDT version is current 2013 Update 2, however this problem was also present in Update 1.  My troubleshooting has resorted to my last ditch effort of recreating an entirely new deployment share using the latest MDT build, creating a bare bones windows 10 image to deploy and test and the results are the same as the normal production deploymentshare.  Reimaging these two laptops will literally either half the time fail out at some random time, or will process all steps successfully.  

    Logs from two different deployments:

    http://1drv.ms/1Po41sF


    The smsts.log is literally littered with random steps that either succeed or fail.  the ZTIApplications.log doesn't provide any log information regarding the applications that succeeded or failed.  Likewise, ZTISetVariable.log has no information about the variables it couldn't set, and the ZTIDomainJoin.log has no information (or wasn't even created), as well with ZTIGroups.log and ZTIUserState.log.  These failures are only randomly occurring in the State Restore phase, after the image has been loaded on the drive and the OS boots up and logs in.  The error for any script it's calling in Incorrect Function, I don't know what it's doing and ran out of things to try and do to make it work.  And it's gotta work correctly...  this is all the information I can think to provide, but do let me know if more information would be useful.  

    Thanks!



    Friday, January 29, 2016 11:01 PM

Answers

  • OK, looks I have found the issue and the solution...

    Based on my previous screenshots, it was clear to me it was a driver issue.  The issue being that Windows 10 likes to check the internet to update drivers, and when it updates the network driver during the MDT task sequence it completely interrupts the process and causes it to crap out.

    As it's unreasonable for me to try and stay on top of network driver version updates across all hardware, considering Dell clearly isn't supplying that latest version, I have to disable the Windows 10 feature to automatically search for driver updates on the internet.

    The fix I have found is, before the OS boots for the first time, change this registry key to 0:

    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\DriverSearching]
    "SearchOrderConfig"=dword:00000000

    This seems to prevent Windows 10 from doing what it does, and lets the imaging process complete reliably...

    • Marked as answer by Mike McConnell Tuesday, February 16, 2016 11:54 PM
    Tuesday, February 16, 2016 11:54 PM

All replies

  • Can you post your actual log files (bdd.log and smsts.log) to something like OneDrive and share the link here?

    Logs are very important. https://keithga.wordpress.com/2014/10/24/video-mdt-2013-log-files-basics-bdd-log-and-smsts-log/ Mention any customizations you have made.

    Friday, January 29, 2016 11:03 PM
    Moderator
  • Yes.

    http://1drv.ms/1Po41sF


    The logs ending with YZ1 belong together in the same deployment.  The machine that produced the N32.log deleted all the other logs and never backed them anywhere.  I found that smsts.log by looking in the local admin \appdata\local\temp folder.  



    Friday, January 29, 2016 11:13 PM
  • Thank you.  This makes it much easier to look at the logs.  CMTrace > Notepad in this instance ;)

    Logs are very important. https://keithga.wordpress.com/2014/10/24/video-mdt-2013-log-files-basics-bdd-log-and-smsts-log/ Mention any customizations you have made.

    Friday, January 29, 2016 11:16 PM
    Moderator
  • I can tell from your log that you have a very mature MDT infrastructure.

    I can also see there are quite a few customizations.

    Some of your msi installs you could append /lvx* %LOGFILE% and most of the 'Incorrect Function' errors seem to be in batch files.  I would look at your batch files first.

    I know at least one failure had to do with an incorrect password.  I am also curious if you are using the decoded passwords in any of your batch files.


    Logs are very important. https://keithga.wordpress.com/2014/10/24/video-mdt-2013-log-files-basics-bdd-log-and-smsts-log/ Mention any customizations you have made.


    Friday, January 29, 2016 11:59 PM
    Moderator
  • Yeah, the DNS app failure is not of a concern.  No passwords are used in any batch files.  I'm actually not yet convinced there is any problems with the batch files or any of the applications (aside from the DNS command line) as half the time every application installs just fine.  That, and other steps not related to the app installs also are randomly failing with 'Incorrect Function', mostly as shown in the smsts-N32.log, there's a bunch of interesting 'Incorrect Functions' there that I don't even know how to begin to explain.  

    None of these failures are in any way consistent, sometimes the deployment is fine, and other times a bunch of random deployment steps will fail with 'Incorrect Function'.  From what I can tell, this is only happening with the Windows 10 deployment.  Windows 7 and 8.1 seem to be deploying correctly every time.  

    Thanks

    Monday, February 1, 2016 3:09 PM
  • Is it possible that some machines have stale scripts? Or even if your deployment share has some?

    Logs are very important. https://keithga.wordpress.com/2014/10/24/video-mdt-2013-log-files-basics-bdd-log-and-smsts-log/ Mention any customizations you have made.

    Monday, February 1, 2016 5:42 PM
    Moderator
  • Upon more experimenting, it looks like something possibly with something the OS is doing with the network card drivers that may be causing it to randomly lose its network connection.  Only in Windows 10, only on laptops...  

    This photo is from the ethernet adapter events on a machine that had errors... at 1:47 is approximately when the machine booted up from MDT's reboot out of winPE.  It calls for a device installation.  About 10 minutes later when the task sequence is already running it reports that the device is installed, at 1:57:40.

    Then looking at the error logs for this particular deployment, the first incorrect function error happens at precisely 1:57:42 pm where it then proceeds to skip a bunch of steps with the same error.  This time however I decided to use the MDT debugging tool to try and do a little debugging of Microsoft's own scripts, except the tool doesn't run when when the errors occur.  It works fine on the instances where the deployment makes it through though.  On machines that errors out it provides a different error other than 'incorrect function' on the steps the debugging tool is present, it instead shows 'The system cannot find the file specified', indicating the network connect is lost.

    Now that I believe I have narrowed down the source of the errors, I still don't actually know why this is occurring or what to do to correct it.  I've been testing using the latest and greatest network drivers Dell has available to also not injecting any drivers what so ever and using the ones Windows 10 installs on its own.  But for some reason it appears something with the systems network card configuration is causing issues.     

    Wednesday, February 3, 2016 10:06 PM
  • OK, looks I have found the issue and the solution...

    Based on my previous screenshots, it was clear to me it was a driver issue.  The issue being that Windows 10 likes to check the internet to update drivers, and when it updates the network driver during the MDT task sequence it completely interrupts the process and causes it to crap out.

    As it's unreasonable for me to try and stay on top of network driver version updates across all hardware, considering Dell clearly isn't supplying that latest version, I have to disable the Windows 10 feature to automatically search for driver updates on the internet.

    The fix I have found is, before the OS boots for the first time, change this registry key to 0:

    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\DriverSearching]
    "SearchOrderConfig"=dword:00000000

    This seems to prevent Windows 10 from doing what it does, and lets the imaging process complete reliably...

    • Marked as answer by Mike McConnell Tuesday, February 16, 2016 11:54 PM
    Tuesday, February 16, 2016 11:54 PM
  • The fix I have found is, before the OS boots for the first time, change this registry key to 0:

    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\DriverSearching]
    "SearchOrderConfig"=dword:00000000

    This seems to prevent Windows 10 from doing what it does, and lets the imaging process complete reliably...

    Can I ask where in the Task Sequence you put this registry edit command? I can't seem to work out how to apply this before the OS boots for the first time and all my problems start.
    Thursday, April 7, 2016 4:14 PM
  • The fix I have found is, before the OS boots for the first time, change this registry key to 0:

    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\DriverSearching]
    "SearchOrderConfig"=dword:00000000

    This seems to prevent Windows 10 from doing what it does, and lets the imaging process complete reliably...

    Can I ask where in the Task Sequence you put this registry edit command? I can't seem to work out how to apply this before the OS boots for the first time and all my problems start.

    In the 'Postinstall' phase I added 3 new steps for this, before the next phase / restart computer steps:

    1. Run Command Line - Load Registry HKLM

    reg.exe load HKLM\MountedHive %OSDisk%\Windows\System32\config\SOFTWARE

    2. Run Command Line - Disable Driver Download

    regedit /s %ScriptRoot%\disabledriverdownload.reg

    disabledriverdownload.reg contents:

    Windows Registry Editor Version 5.00

    [HKEY_LOCAL_MACHINE\MountedHive\Microsoft\Windows\CurrentVersion\DriverSearching]
    "SearchOrderConfig"=dword:00000000

    3. Run Command Line - Unload Registry HKLM

    reg.exe unload HKLM\MountedHive


    Thursday, April 7, 2016 6:27 PM
  • I know this is old but have a quick question.

    Thanks for posting this fix, I am having this same issue on our laptops but sporadically. I have a stupid question, why do you mount Windows\System32\config\SOFTWARE instead of just running the regedit /s? Is it because Windows is not loaded yet?

    Also if I wanted to "undo" this once the image is laid down, can I just repeat the above setting it back the default level? If what I assume above is correct that you load Windows\System32\config\SOFTWARE before your regedit command because windows is not running yet, you could skip steps 1 and 3 and just change the value in HKLM to set it back, right?

    Sorry if that's confusing.

    Thanks,

    Scott

    Wednesday, August 30, 2017 4:21 PM
  • I know this is old but have a quick question.

    Thanks for posting this fix, I am having this same issue on our laptops but sporadically. I have a stupid question, why do you mount Windows\System32\config\SOFTWARE instead of just running the regedit /s? Is it because Windows is not loaded yet?

    Also if I wanted to "undo" this once the image is laid down, can I just repeat the above setting it back the default level? If what I assume above is correct that you load Windows\System32\config\SOFTWARE before your regedit command because windows is not running yet, you could skip steps 1 and 3 and just change the value in HKLM to set it back, right?

    Sorry if that's confusing.

    Thanks,

    Scott

    The point I am running the regedit is in WinPE, before the actual OS has even booted, so you have to mount the registry of the offline OS first.  If you wait until the OS loads you can skip the mount stage, however that is when the issue presents itself and it can often happen too quickly for the task sequence to kick in, and the change also doesn't take effect until the machine is rebooted.  

    To remove the fix you'd could create a task sequence step near the end that reverts the registry back to how it was pre-fix, and you can do that from within the main OS so no need to mount any registry hives as it's already mounted.  Change the registry value location to reflect that it's to modify the real OS registry instead of a mounted hive location though. 

    Thursday, August 31, 2017 3:37 PM
  • Thats what I thought. Thanks for taking the time to reply back!
    Thursday, August 31, 2017 4:31 PM