none
Powershell - Process Redirect - Polling RRS feed

  • Question

  • Hi

    I found a number of articles on running processes and BAT files and capturing the output and errors, e.g.

    http://www.chinabtp.com/captured-output-of-command-run-by-powershell-is-sometimes-incomplete/
    http://stackoverflow.com/questions/24370814/how-to-capture-process-output-asynchronously-in-powershell

    So I created a test function below and it "works"
    However I wanted to have control over the display of the output, i.e. close to real time. According to the doco the event gets fired for each newline, my tests indicate otherwise.
    i.e. the script may run for a long time and have 1000s of lines of output..

    I did a test where I told the BAT file to sleep. However the results where not as expected.
    There was a delay until the first output occurred, then a further delay which could be accounted for due to the sleep.

    https://msdn.microsoft.com/en-us/library/microsoft.powershell.utility.activities.registerobjectevent_methods(v=vs.85).aspx
    https://msdn.microsoft.com/en-us/library/microsoft.powershell.utility.activities.registerobjectevent_members(v=vs.85).aspx

    Don't help because there are poorly documented, e.g. MaxtriggerCount, provides access to maxtriggercount, none of the items are actually described by function.

    The C# process has ReadToEnd() but this does not apply to Powershell

    So the question :

    >> Is there a way for me fetch the output event data, say in the 1s while loop ?

    SBS_Logger justs adds the timestamp

    Function SBS_RunAProcess
    {
    [CmdletBinding()]
     param(
     		[Parameter(Mandatory = $True)] 
           [String]$Label )
    	   
    		# Setup stdin\stdout redirection
    		$StartInfo = New-Object System.Diagnostics.ProcessStartInfo -Property @{
    			FileName = "F:\ScriptWork\helloworld.bat"
    			Arguments = $Label
    			UseShellExecute = $false
    			RedirectStandardOutput = $true
    			RedirectStandardError = $true
    		}
    		# Create new process
    		$Process = New-Object System.Diagnostics.Process
    		# Assign previously created StartInfo properties
    		$Process.StartInfo = $StartInfo
    		# Register Object Events for stdin\stdout reading
    		$OutEvent = Register-ObjectEvent -InputObject $Process -EventName OutputDataReceived -Action {
    
    		SBS_Logger $Event.SourceEventArgs.Data
    		}
    		$ErrEvent = Register-ObjectEvent -InputObject $Process -EventName ErrorDataReceived -Action {
    
    		SBS_Logger $Event.SourceEventArgs.Data
    		}
    		# Start process
    		[void]$Process.Start()
    		# Begin reading stdin\stdout
    		$Process.BeginOutputReadLine()
    		$Process.BeginErrorReadLine()
    		# Do something else while events are firing
    		do
    		{
    			#Write-Host 'Still alive!' -ForegroundColor Green
    			Start-Sleep -Seconds 1
    		}
    		while (!$Process.HasExited)
    		# Unregister events
    		$OutEvent.Name, $ErrEvent.Name |
    		ForEach-Object {Unregister-Event -SourceIdentifier $_}
     }
    BAT Content
    
    @echo off
    echo "hello world from a BAT file"
    echo "args where %1%"
    rem timeout /t 10 /nobreak > NUL
    SET
    
    ===
    
    2015-09-03 14:04:00 Task test_bat running
    2015-09-03 14:04:01 "hello world from a BAT file"
    2015-09-03 14:04:01 "args where greg"
    2015-09-03 14:04:01 ALLUSERSPROFILE=C:\ProgramData
    2015-09-03 14:04:01 APPDATA=C:\Users\build\AppData\Roaming
    
    
    =====================
    BAT Content
    
    echo off
    echo "hello world from a BAT file"
    echo "args where %1%"
    timeout /t 10 /nobreak > NUL
    SET
    
    ===
    
    2015-09-03 14:08:43 Task test_bat running
    2015-09-03 14:08:54 "hello world from a BAT file"  <= 9s delay
    2015-09-03 14:08:54 "args where greg"
    2015-09-03 14:09:03 ALLUSERSPROFILE=C:\ProgramData <= 9s delay
    
    


    Thursday, September 3, 2015 5:04 AM

Answers

  • This is a shortcoming of running an unattended process.    The program (bat file) is terminating before you can grab the buffer.  There is no way to prevent this.  You need to output the information to a file and then monitor the file.

    .\MyBatch.bat > mybuild.log
    Get-Content mybuild.log -wait -tail 1

    You can modify the output and write to a new file if needed.


    \_(ツ)_/


    • Edited by jrv Friday, September 4, 2015 11:06 PM
    • Marked as answer by Greg B Roberts Monday, September 7, 2015 4:39 AM
    Friday, September 4, 2015 11:05 PM

All replies

  • The event data is not "fetched" it only exists when the event is triggered.

    It is very difficult to understand what you are trying to do .  Can you ask a specific question without so much discussion. Just state what it is you need to do.

    If you output the data it will list at the console. It appears you are using Write-Host. That will throw assay all data after it is displayed.


    \_(ツ)_/

    Thursday, September 3, 2015 5:27 AM
  • phew

    I really tried, it I did too little then I would be asked for more context and examples. We are on different wavelengths

    Existing examples show capturing process output

    I did this and found that quirks, i.e. for each line of BAT file output, and event was not triggered

    Rather it happened later that expected.

    >> Is this approach wrong, and I need some ReadtoEnd() approach from the output ?

    or is the issue that the cmd.exe itself is only firing output events when it thinks it should, not every line ?

    Thanks

    Thursday, September 3, 2015 5:33 AM
  • There is nothing to read.  The text is being sent to you in the event data.  The events are fired when the output of a program is available.

    Change this as it will be more reliable:

    		$StartInfo = New-Object System.Diagnostics.ProcessStartInfo -Property @{
    			FileName = $env:comspec
    			Arguments = @('F:\ScriptWork\helloworld.bat', $Label)
    			UseShellExecute = $false
    			RedirectStandardOutput = $true
    			RedirectStandardError = $true
    		}
    


    \_(ツ)_/

    Thursday, September 3, 2015 6:33 AM
  • Thanks for this tip,

    I modified my test BAT and its contents are below
    While one lot of doc said the call back occues after each line, my tests indicate this is not always the way.

    The conclusion is although the BAT "timeout" does not affect output when run manually, it does when run from powershell.  This would suggest there isn't much we can do about this and the code is not at fault.

    =====

    - Double clicking on the BAT file interactively the first 3 lines are printed straight away, then 10s later the env. variables are printed, then 10s later the end, e.g.

     "hello world from a BAT file"
     "args where greg"
    (10s delay before SET info comes out)
     ALLUSERSPROFILE=C:\ProgramData

    - When running the same BAT via powershell capturing the output

    2015-09-03 16:54:15 Task test_bat running (from powershell)(delay)
    2015-09-03 16:54:36 "hello world from a BAT file"
    2015-09-03 16:54:36 "args where greg"
    2015-09-03 16:54:36 "I am going to sleep for 10s"
    (delay expected)
    2015-09-03 16:54:46 ALLUSERSPROFILE=C:\ProgramData
    ....
    2015-09-03 17:02:23 SystemRoot=C:\WINDOWS
    2015-09-03 17:02:23 "Another 10s"
    (delay expected but not in results)
    2015-09-03 17:02:23 "Done"

    - Now running the same script via powershell with the "timeout" removed (ran time)

    2015-09-03 17:06:02 Task test_bat running
    2015-09-03 17:06:03 "hello world from a BAT file"
    2015-09-03 17:06:03 "args where greg"
    2015-09-03 17:06:03 ALLUSERSPROFILE=C:\ProgramData

    ======================

    This is a BAT file contents

    @echo off
    echo "hello world from a BAT file"
    echo "args where %1%"
    echo "I am going to sleep for 10s"
    timeout /t 10 /nobreak > NUL  <= this is where a delay can be added
    SET
    echo "Another 10s"
    timeout /t 10 /nobreak > NUL
    echo "Done"


    Thursday, September 3, 2015 7:10 AM
  • I am having difficulty understanding your specific question.

    What, exactly and specifically, do you want to do in PowerShell? What is the purpose/goal?

    Also post a short example script that "doesn't work" and explain specifically how it doesn't work. Remember, we don't know what you are trying to do, and we can't see your screen.


    -- Bill Stewart [Bill_Stewart]

    Thursday, September 3, 2015 3:09 PM
    Moderator
  • I updated by previous comment to try to make it clearer.

    The script "does work" i.e. all output is captured

    What doesn't work is this capture is no where near realtime.

    I'll place some sample files in onedrive soon.

    In onedrive http://1drv.ms/1EDJPQf

    I added some test scripts

    I expanded the while check to be

    }

    while(!$Process.HasExited -or$OutEvent.HasMoreData -or$ErrEvent.HasMoreData)

    What I found what that a short time after the process ends, the events events stop even though they still have events in them

    I have not found how to flush the events

    Thanks

    Friday, September 4, 2015 12:33 AM
  • You have a sleep. It will delay the output.

    What is the purpose of this?


    \_(ツ)_/

    Friday, September 4, 2015 3:27 AM
  • I found this in several scripts of the web as an example of this approach

    On removing the sleep more lines get truncated from the results of the BAT

    Removing this completely and using

    $Process.WaitForExit()

    gets a few more lines but some end lines are missing

    Thanks



    Friday, September 4, 2015 3:35 AM
  • Are you saying you want to output to the console and write to a file at the same time?

    You mean like Tee-Object?


    -- Bill Stewart [Bill_Stewart]

    Friday, September 4, 2015 2:23 PM
    Moderator
  • I want to run a process and capture its output

    In this example the process is a BAT file.

    This allows me to add time stamps and do some adjustments to the output as needed.

    NB: I have found in the last few hours that if an output line is empty this upsets the event call back

      $OutEvent = Register-ObjectEvent -InputObject $Process -EventName OutputDataReceived -Action {
      SBS_Logger $Event
    .SourceEventArgs.Data
    }

    and a try - catch resolves this issue around the Logger

    thanks

    Friday, September 4, 2015 2:32 PM
  • What does the shell script (batch file) do? Why not do everything you need in PowerShell?

    Why do you think you need to register a .NET object event?

    what is the overall purpose/design goal of this script? (Tell what you want to do, not how you think you need to do it.)


    -- Bill Stewart [Bill_Stewart]

    Friday, September 4, 2015 2:35 PM
    Moderator
  • Hi Bill

    Background:

    I am developing a framework for running our extensive build system under TFS Build. This extensive system comprises many BAT files driving msbuild and mstest plus doing BAT operations.

    Over time we will move most of the build content into powershell.

    Now there are several ways to attack this, but my probe at the moment was running the BAT files from powershell. This has the advantage that if output is captured then some enhancements can be done to the output, e.g. time stamps. As well as issue tracking reporting.

    So I googled and came across numerous references to ioredirect and Register-ObjectEvent Then I found quirks in this process.

    My status now is I have found that after the process has exited there still are items in the event queue. I have found that Unregister-Event in fact is the item that will force the last of the traces out. The While loop is not needed and a $Process.WaitForExit() will do the job.

    So the intent of this post was to work through how to capture process  redirect successfully. NB: If the BAT files has pauses in it then the capture system becomes undeterministic (see top of post). Possibly related to the BAT input (? peeks in "timeout" to see if a key was pressed)

    regards

    Friday, September 4, 2015 10:33 PM
  • This is a shortcoming of running an unattended process.    The program (bat file) is terminating before you can grab the buffer.  There is no way to prevent this.  You need to output the information to a file and then monitor the file.

    .\MyBatch.bat > mybuild.log
    Get-Content mybuild.log -wait -tail 1

    You can modify the output and write to a new file if needed.


    \_(ツ)_/


    • Edited by jrv Friday, September 4, 2015 11:06 PM
    • Marked as answer by Greg B Roberts Monday, September 7, 2015 4:39 AM
    Friday, September 4, 2015 11:05 PM