none
Problems getting BITS to work with PowerShell cmdlets RRS feed

  • Question

  • I'm attempting to get a PS script working to upload .NET 5 apps to IIS. Using an approach by piping ZIP file content directly to the server failed because the ZIP file is too large. I'm attempting to use BITS, but I'm running into problems, especially with a connection error and an invalid CA error (because I'm using a self-signed cert). I've tried two approaches with an Azure VM (WS 2012 R2) with the BITS features installed (including the IIS Extension and compact server):

    $result = Invoke-WsmanAction `
                -Action CreateJob `
                –ResourceUri wmi/root/microsoft/bits/BitsClientJob `
                –ValueSet @{ `
                    DisplayName="CreateJob"; `
                    RemoteUrl="https://xxxxx.cloudapp.net/deployment/output.zip"; `
                    LocalFile="$env:USERPROFILE\AppData\Local\Temp\PublishTemp\output.zip"; `
                    Type=1; `
                } `
                -Credential $mycreds `
                -ComputerName $cloudService `
                -Port 50000 `
                -UseSSL `
                -SessionOption (New-WSManSessionOption -SkipCACheck)

    [where $cloudService = xxxxx.cloudapp.net; $mycreds are known good (the server admin account is used), because the PS session is started with them; 50000 maps to 5986 in Azure; 5986 was opened by the BITS installer on the VM and Remote Address on the port is set to "Any"]

    It doesn't choke on the fact that I'm using a self-signed certificate, but it throws with an exception stating that ...

    "The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol."

    I've changed/removed/added the parameters to every possible combination I can think of.

    The port endpoint is open in Azure, allowed in the firewall, configured in IIS, with all the BITS stuff on the server and the virtual directory configured with the IIS BITS extension. I am a bit confused about permissions on that folder, deployment, that I setup for the vdir (techdocs are very difficult to comprehend on exactly what permissions settings to apply to each group/user). When I manually add an output.zip file to the vdir and hit the address https://xxxxx.cloudapp.net:50000/deployment/output.zip, I do get the file. When I run the PS script for an upload to that vdir, the Event Log only shows a connection attempt to start the job, followed by a second entry immediately showing that the CreateJob was cancelled (for no apparent reason).

    If I try using this ...

    $Job = Start-BitsTransfer `
                -Source $env:USERPROFILE\AppData\Local\Temp\PublishTemp\output.zip `
                -Destination https://xxxxx.cloudapp.net:50000/deployment/output.zip `
                -TransferType Upload `
                -DisplayName MyJob `
                -Credential $mycreds `
                -Priority High

    ... it fails with ...

    "The certificate authority is invalid or incorrect"

    ... no matter if the cert is in the Trusted Root or not and no matter if it's installed on the local machine or not. There is apparently no switch for telling the cmdlet to ignore the CA.

    After about 12 hours of trying to get BITS to work for these large file uploads to work with PS, I'm about to abandon it entirely and try something else (e.g., FTP).

    Does anyone have tips ... esp. on the first approach, where it doesn't seem to choke on the CA problem.


    Luke Latham, CEO GuardRex Corporation


    • Edited by Luke Latham Saturday, September 12, 2015 7:01 PM typo
    Saturday, September 12, 2015 6:40 PM

Answers

  • Start the job with this:
     -Asynchronous -Suspended

    Then enter this:
    bitsadmin /SetSecurityFlags myJob 8

    Then resume the job:

    Get-BitsTransfer|Resume-BitsTransfer

    You will not fail for the cert.


    \_(ツ)_/

    • Marked as answer by Luke Latham Saturday, September 12, 2015 11:38 PM
    Saturday, September 12, 2015 9:03 PM

All replies

  • Run this line before you call to start the upload.

    [Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}


    \_(ツ)_/

    Saturday, September 12, 2015 6:54 PM
  • I put the command in the PS script above each of my approaches.

    Using Approach #1, comes back the same:

    The WinRM client sent a request to an HTTP server and got a response saying the requested HTTP URL was not available. This is usually returned by a HTTP server that does not support the WS-Management protocol."

    Using Approach #2, comes back the same:

    "The certificate authority is invalid or incorrect"


    Luke Latham, CEO GuardRex Corporation

    Saturday, September 12, 2015 7:07 PM
  • WinRM does not like test certs.  Try allocating a real cert. 

    You should also try this:

    New-WSManSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck


    \_(ツ)_/

    Saturday, September 12, 2015 7:21 PM
  • Same result.

    One thing strange here is that under my Approach #1 when I look at Fiddler when the cmdlet is triggered, I don't see anything. Like the request isn't getting off of the client. Earlier, it was hitting the server (at least a few times) because I had some EventLog entries in the BITS log showing a hit and an immediate cancellation of the CreateJob (without reason).

    I'm going to play with it a bit more.

    You're right about the cert, but I don't want to buy one just to test this. All of my certs are on other VM's hooked up production stuff.


    Luke Latham, CEO GuardRex Corporation

    Saturday, September 12, 2015 7:57 PM
  • Start the job with this:
     -Asynchronous -Suspended

    Then enter this:
    bitsadmin /SetSecurityFlags myJob 8

    Then resume the job:

    Get-BitsTransfer|Resume-BitsTransfer

    You will not fail for the cert.


    \_(ツ)_/

    • Marked as answer by Luke Latham Saturday, September 12, 2015 11:38 PM
    Saturday, September 12, 2015 9:03 PM
  • Just to be sure I cloned off a VMM with BITRSS and set up a default test cert.  The upload failed until I set the flag then it works as expected.  I also had to set the "ignore name" bit which is 2 so you might need to use 10 instead of 8.


    \_(ツ)_/

    Saturday, September 12, 2015 9:31 PM
  • You GOT it! Wow ... I wouldn't have figured this out without your help. Here's what finally worked for me:

    New-WSManSessionOption -SkipCACheck -SkipCNCheck -SkipRevocationCheck
    $Job = Start-BitsTransfer `
        -Source $env:USERPROFILE\AppData\Local\Temp\PublishTemp\output.zip `
        -Destination https://${cloudService}:${port}/deployment/output.zip `
        -TransferType Upload `
        -DisplayName MyJob `
        -Priority High `
        -Credential $mycreds `
        -Asynchronous `
        -Suspended
    bitsadmin /SetSecurityFlags MyJob 8
    Resume-BitsTransfer -BitsJob $Job -Asynchronous
    while (($Job.JobState -eq "Transferring") -or ($Job.JobState -eq "Connecting")) {
    Write-Host $Job.JobId $Job.Jobstate $Job.BytesTransferred $Job.BytesTotal;
    sleep 10;
    }
    Switch($Job.JobState) {
    "Transferred" {Complete-BitsTransfer -BitsJob $Job; $IsSuccessfulTransfer = $true;}
    "Error" {Write-Host $Job.ErrorDescription; Write-Host;}
    default {}
    }

    $cloudservice is the Azure Cloud Service endpoint for the VM (e.g., xxxxxxxx.cloudapp.net)

    $port is the SSL port bound to the website (e.g., 50000)
    Of course, you have to open the port in the Azure portal for the VM (public = 50000, private = 50000). You also need to open the port in the firewall on the VM.

    The certificate being used in IIS for the website is the self-signed Azure Cloud Service cert, named xxxxxxx.cloudapp.net and produced when the Cloud Service is created. You can find this cert in your cert store after you click in Azure to connect to the VM via RDP. It will install on your local machine for your RDP connections to the VM.

    $mycreds come from an admin server account and can be produced with the following bit. You can live a life of danger by hard-coding the password into the script ... good luck with that. Note that when you install BITS on the server it will set it up to run under the Local Service account, and this is good, because there are claims on the Net that if you try to run BITS on the server under an account that is used for login that BITS will fail when the logged-in user logs out. Accounts like the Local Service are always logged-in and this is the default for the BITS install, so you don't need to sweat it unless you decide that you need the service to run under some other account ... beware!

    $pw = Read-Host -Prompt "Input the password "
    $secpasswd = ConvertTo-SecureString $pw -AsPlainText -Force
    $mycreds = New-Object System.Management.Automation.PSCredential `
    ($serverName + "\" + $serverUsername, $secpasswd)

    Of course, you also have to have all the BITS bits installed on the server via the Roles & Feature wizard and restart the server. You need both the IIS Extension and the Compact Server.

    You'll setup a virtual directory in the website in IIS. There is a ton of chatter on the Net about changing the permissions on the folder in the OS, but I didn't find that it was necessary. I guess you could do as they suggest and disable Write and Execute permissions for safety. I'm going to test shortly and see if it chokes anything. You will need to leave Anonymous Authentication enabled ... or else it will choke. You should be able to lock the vdir down in additional ways that will work (e.g., by IP address in IIS and/or at the firewall and/or via Azure ACL).

    You just need to select the virtual directory in IIS under the app, click the BITS IIS Extension, and click "Check to allow clients to upload files." If you use custom settings, you can also click there to allow file overwrites. Be sure to "Apply" the changes on the right or else they won't stick. Next, set to Require SSL on the virtual directory in IIS.

    Install PowerShell 5.0. The preview is still out at the time of this writing, so that's at http://www.microsoft.com/en-us/download/details.aspx?id=48729

    Finally, I am still fighting with it a little bit. The call to "bitsadmin" uses the "DisplayName" to find the BITS job. If there is a failure, the BITS job stays active. If you re-run with it remaining active, it breaks there claiming that it can't distinguish between multiple jobs with the same DisplayName. I wish it could use JobId instead, but it doesn't seem to want to take that as an identity parameter when it runs the command. One way around all of this would be, as jrv suggested earlier, to use a real SSL cert with a valid CA. I'll add something later if I get a final workaround that I like in place.

    Thanks again jrv! You saved me.


    Luke Latham, CEO GuardRex Corporation







    • Edited by Luke Latham Sunday, September 13, 2015 10:57 PM content change
    Saturday, September 12, 2015 11:19 PM
  • Yeah. I had to look at it myself to see what was happening.  Then I searched on the error and the answer popped up:

    StartBitsTransfer: The certificate authority is invalid or incorrect

    Always search the text of the error.

    https://www.google.com/?gws_rd=ssl#newwindow=1&q=StartBitsTransfer:+The+certificate+authority+is+invalid+or+incorrect

    Anyone who really knows BITS would have seen that almost immediately. I haven't set up BITS server for a couple of years.  I had to fish for an answer.


    \_(ツ)_/


    • Edited by jrv Saturday, September 12, 2015 11:44 PM
    Saturday, September 12, 2015 11:43 PM
  • I also had to set the "ignore name" bit which is 2 so you might need to use 10 instead of 8.

    I was able to leave it "8" due to using the Azure Cloud Service Url (xxxxx.cloudapp.net) that does match the certificate provided by Azure for RDP. The IIS website SSL binding is to that address, so fortunately there the binding matches the cert CN and I didn't need to change it, but that's a great tip for anyone with a cert with both a CA and a CN that are going to choke.


    Luke Latham, CEO GuardRex Corporation

    Saturday, September 12, 2015 11:58 PM
  • I also had to set the "ignore name" bit which is 2 so you might need to use 10 instead of 8.

    I was able to leave it "8" due to using the Azure Cloud Service Url (xxxxx.cloudapp.net) that does match the certificate provided by Azure for RDP. The IIS website SSL binding is to that address, so fortunately there the binding matches the cert CN and I didn't need to change it, but that's a great tip for anyone with a cert with both a CA and a CN that are going to choke.


    Luke Latham, CEO GuardRex Corporation

    Yes. That is good info.  Also look at all of the other bits.  They are set from top-to-bottom in the listing as right to left.  Use hex to improve:

    bitsadmin job1 /SetSecurityFlags 0xA

    0xA = 10 or 1010 binary.


    \_(ツ)_/



    • Edited by jrv Sunday, September 13, 2015 12:12 AM
    Sunday, September 13, 2015 12:12 AM
  • The sad part is that my post (with your two critical parts) is probably the most complete and clear explanation of what to do, and that just shouldn't be the case for technology this old and simple. The docs for BITS are just horrible IMHO. I just hope others find this before they spend the time with what's currently out there passing for documentation. Note to MS: If BITS is sticking around, please get someone to FIX the docs for it.

    Luke Latham, CEO GuardRex Corporation

    Sunday, September 13, 2015 12:15 AM