none
Parse LogFile Using PowerShell

    Question

  • Hello,

    I am new to PowerShell, and trying to parse Log file for the most recent [ERROR] keyword . 

    Content of my log file is as follows

    [ERROR] 2013-12-23 19:46:32
    [ERROR] 2013-12-24 19:46:35
    [ERROR] 2013-12-24 19:48:56
    [ERROR] 2013-12-24 20:13:07

    Function CheckLogs()

    {
        param ([string] $logfile)
        if(!$logfile) {write-host "Usage: ""<Log file path>"""; exit}
        cat $logfile | Select-String "ERROR" -SimpleMatch | select -expand line |
             foreach {
                        $_ -match '(.+)\s\[(ERROR)\]\s(.+)'| Out-Null 
                        new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]} |
                        where {$_.timestamp -gt (get-date).AddDays(-1)}
                        $error_time = [datetime]($matches[1])
                        if ($error_time -gt (Get-Date).AddDays(-1) )
                        {
                            write-output "CRITICAL: There is an error in the log file $logfile around 
                                          $($error_time.ToShortTimeString())"; exit(2)
                        } 
                     }
      write-output "OK: There was no errors in the past 24 hours." 
    }
    CheckLogs "C:\Log.txt" #Function Call

    After executing above script, I am getting the below error, please advise.

     $error_time = [datetime]($matches[1])
    +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray
     
    Cannot index into a null array.
    At C:\PS\LogTest.ps1:10 char:21
    +                     new-object psobject -Property @{Timestamp = 
    [datetime]$match ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray
     
    Cannot index into a null array.
    At C:\Test\LogTest.ps1:12 char:21
    +                     $error_time = [datetime]($matches[1])
    +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray

    Wednesday, December 25, 2013 2:49 AM

Answers

  • You asked one question. You are now asking for a whole solution.

    The code I posted returns an array of all line dates as datetime objects.  YOu can catche the results and do whatever you need with it.

    In your post you ask nothing about what is on the line after the time string.  You can add the remainder of th eline to the results very easily by capturing it.

    $results=cat logfile.log  |
         ?{$_ -match '^\[ERROR'}|
         %{
             if($_ -match '(?<dt>\d+-\d+-\d+ \d+:\d+:\d+)(?<txt>.*)'){
                  New-Object PsObject -Property @{Time=[datetime]$matches['dt'];Text=$matches['dt']}
             }
          }

    It is still up to you to write the remaining code.  $results will be an array of all lines with an error. 


    ¯\_(ツ)_/¯

    • Marked as answer by BJKP Monday, December 30, 2013 1:27 AM
    Thursday, December 26, 2013 3:11 PM

All replies

  • The -match operator returns value of True or False, and the $matches automatic variable is only populated if -match returned true.  You shouldn't just pipe the value of the -match expression to Null and then assume that it was successful.

    The regular expression pattern you're using doesn't match the sample contents you've listed.  The pattern would match something like "<Date / Time> [ERROR] <Error Message>".

    Wednesday, December 25, 2013 3:03 AM
  • Do you mean, I need to remove |Out-Null  from below

     $_ -match '(.+)\s\[(ERROR)\]\s(.+)'| Out-Null  

    Can you please advise on regular expression "<Date / Time> [ERROR] <Error Message>" is not working


    Wednesday, December 25, 2013 3:15 AM
  • This grabs and converts

    cat logfile.log  |
         ?{$_ -match '^\[ERROR'}|
         %{
             if($_ -match '(?<dt>\d+-\d+-\d+ \d+:\d+:\d+)'){
                  [datetime]$matches['dt']
             }
          }
    


    ¯\_(ツ)_/¯

    Wednesday, December 25, 2013 3:25 AM
  • You should get rid of Out-Null, and put the expression containing the -match operator into an "if" statement (or a Where-Object block, as jrv demonstrated.)
    Wednesday, December 25, 2013 3:35 AM
  • Thanks for your replies, unfortunately this is not working. Kindly provide the full script if possible. 

    Function CheckLogs()
    {
        param ([string] $logfile)
        if(!$logfile) {write-host "Usage: ""<Log file path>"""; exit}
        cat $logfile | Select-String "ERROR" -SimpleMatch | select -expand line |
             foreach {
                      if($_ -match '(.+)\s\[(ERROR)\]\s(.+)')
                       {
                        new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]} |
                        where {$_.timestamp -gt (get-date).AddDays(-1)}
                        $error_time = [datetime]($matches[1])
                            if ($error_time -gt (Get-Date).AddDays(-1))
                            {
                            write-output "CRITICAL: There is an error in the log file $logfile around 
                                          $($error_time.ToShortTimeString())"; exit(2)
                            } 
                         write-output "OK: There was no errors in the past 24 hours." 
                        }
                     }
     
    }
     
    CheckLogs "C:\Log.txt"

    Wednesday, December 25, 2013 4:09 AM
  • Hello,

    Anyone please help on this.

    Wednesday, December 25, 2013 5:28 AM
  • Hello,

    Anyone please help on this.

    I posted the complete solution above.  Why not just use it.


    ¯\_(ツ)_/¯

    Wednesday, December 25, 2013 9:50 AM
  • Wednesday, December 25, 2013 5:23 PM
  • Hello Jrv,

    Could you please elaborate, I am new to power shell.

    If the above snippet is considered as full solution, kindly advise where I could write my output as no errors, are if any errors found where should I locate.

    My objective is to get the most recent [ERROR] and the associated 500 characters if found. When 500 characters are not found after my most recent [ERROR] just store as many as there in the log.
    Thursday, December 26, 2013 9:24 AM
  • You asked one question. You are now asking for a whole solution.

    The code I posted returns an array of all line dates as datetime objects.  YOu can catche the results and do whatever you need with it.

    In your post you ask nothing about what is on the line after the time string.  You can add the remainder of th eline to the results very easily by capturing it.

    $results=cat logfile.log  |
         ?{$_ -match '^\[ERROR'}|
         %{
             if($_ -match '(?<dt>\d+-\d+-\d+ \d+:\d+:\d+)(?<txt>.*)'){
                  New-Object PsObject -Property @{Time=[datetime]$matches['dt'];Text=$matches['dt']}
             }
          }

    It is still up to you to write the remaining code.  $results will be an array of all lines with an error. 


    ¯\_(ツ)_/¯

    • Marked as answer by BJKP Monday, December 30, 2013 1:27 AM
    Thursday, December 26, 2013 3:11 PM
  • Hello Jrv,

    Thanks for your answer. 

    Monday, December 30, 2013 1:27 AM