none
Parsing Log file with PowerShell

    Question

  • Hey Guys, I have the following line in a txt file (log file) 

    2012-08-14 18:00:00 [ERROR] . Exception SQL error 1 

    2012-08-14 18:10:00 [ERROR] . Exception SQL error 2

    2012-08-15 18:00:00 [INFO] . Started 


    - Check the most recent entry(s) the last 24 hours

    - if there's an error [ERROR] write-out a statement that says (Critical) with the date-time of the error

    - If there's no erros write-out (Ok)

    So far I learned to write this much and would like to learn more from you:

    $file = "C:\Users\example\Documents\Log.txt" 

    cat $file | Select-String "ERROR" -SimpleMatch


    Wednesday, August 15, 2012 4:19 PM

Answers

  • $file = "C:\Users\example\Documents\Log.txt" 
    cat $file |
     Select-String "ERROR" -SimpleMatch |
      select -expand line |
       foreach {
                  $_ -match '(.+)\s\[ERROR\]\s\.\s(.+)' | out-null
                  new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]} |
                  where {$_.timestamp -gt (get-date).AddDays(-1)}
                }

    Better?

    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "


    Wednesday, August 15, 2012 5:04 PM
    Moderator

All replies

  • Does this help?

    $file = "C:\Users\example\Documents\Log.txt" 
    cat $file |
     Select-String "ERROR" -SimpleMatch |
      select -expand line |
       foreach {
                  $_ -match '(.+)\s\[ERROR\]\s\.\s(.+)'
                  new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]}
                }


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    Wednesday, August 15, 2012 4:37 PM
    Moderator
  • Rob, the problem with your answer is that it doesn't check only for entries for the last 24 hours.

    Grant Ward, a.k.a. Bigteddy

    Wednesday, August 15, 2012 4:58 PM
  • Here's your code, with that slight amendment:

    $file = "C:\scripts\sqllog.txt" 
    cat $file |
     Select-String "ERROR" -SimpleMatch |
      select -expand line |
       foreach {
                  $_ -match '(.+)\s\[ERROR\]\s\.\s(.+)' | Out-Null
                  if ([datetime]$matches[1] -gt (Get-Date).AddHours(-24)) {
                    new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]}
                    }
                }


    Grant Ward, a.k.a. Bigteddy

    Wednesday, August 15, 2012 5:04 PM
  • $file = "C:\Users\example\Documents\Log.txt" 
    cat $file |
     Select-String "ERROR" -SimpleMatch |
      select -expand line |
       foreach {
                  $_ -match '(.+)\s\[ERROR\]\s\.\s(.+)' | out-null
                  new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]} |
                  where {$_.timestamp -gt (get-date).AddDays(-1)}
                }

    Better?

    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "


    Wednesday, August 15, 2012 5:04 PM
    Moderator
  • Yes.  Notice I put an Out-Null on the regex match to supress "True".  My version also works.

    Grant Ward, a.k.a. Bigteddy

    Wednesday, August 15, 2012 5:48 PM
  • I always forget that if I don't test.

    Will fix.


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "

    Wednesday, August 15, 2012 6:10 PM
    Moderator
  • Thanks to both of you guys, afte spending more time with this, end results are below, this is going to act as a command to use with nagios nrpe to check any log file for the word ERROR.

    Please review and any improvement is welcomed of course. 

    # This script will parse the log file and search for the word ERROR in the past 24 hours. param ($logfile) if(!$logfile) {write-host "Usage: ""<Log file path>"""; exit} cat $logfile | Select-String "ERROR" -SimpleMatch | select -expand line | foreach { $_ -match '(.+)\s\[(ERROR)\]\s(.+)'| Out-Null $error_time = [datetime]($matches[1]).split(",")[0] #needed to do this cause the timestamp was similar to this(2012-08-09 16:49:48,306)

    if ($error_time -gt (Get-Date).AddHours(-24)) { write-host "CRITICAL: There is an error in the log file" $logfile "around "$error_time; exit(2) } } write-host "OK: There was no errors in the past 24h" #;exit(0)}

    Cheers!

    Wednesday, August 15, 2012 8:35 PM
  • I would make these changes:

    # This script will parse the log file and search for the word ERROR in the past 24 hours.
    param ($logfile)
    if(!$logfile) {write-host "Usage: ""<Log file path>"""; exit}
    cat $logfile |
    Select-String "ERROR" -SimpleMatch |
      select -expand line |
       foreach {
                  $_ -match '(.+)\s\[(ERROR)\]\s(.+)'| Out-Null 
                  $error_time = [datetime]($matches[1])
                  if ($error_time -gt (Get-Date).AddDays(-1)  {
                   write-output "CRITICAL: There is an error in the log file $logfile around $($error_time.ToShortTimeString())"; exit(2)
                   } 
                }
      write-output "OK: There was no errors in the past 24 hours." 

    Use the .toshorttimestring() method to get just the timestamp of the error.

    Use write-output rather than write-host.  If you don't do anything else with it, it will be written to the console, but you can store it in a variable to use later if you choose.  With write-host, you don't hve that option.


    [string](0..33|%{[char][int](46+("686552495351636652556262185355647068516270555358646562655775 0645570").substring(($_*2),2))})-replace " "


    Wednesday, August 15, 2012 9:02 PM
    Moderator
  • Cool, Thank you again, very productive first day with PowerShell :D
    Wednesday, August 15, 2012 9:45 PM
  • Hello,

    I am new to PowerShell, and looking for same requirement, here is my function.

    Function CheckLogs()
    {
        param ([string] $logfile)
        if(!$logfile) {write-host "Usage: ""<Log file path>"""; exit}
        cat $logfile | Select-String "ERROR" -SimpleMatch | select -expand line |
             foreach {
                        $_ -match '(.+)\s\[(ERROR)\]\s(.+)'| Out-Null 
                        new-object psobject -Property @{Timestamp = [datetime]$matches[1];Error = $matches[2]} |
                        where {$_.timestamp -gt (get-date).AddDays(-1)}
                        $error_time = [datetime]($matches[1])
                        if ($error_time -gt (Get-Date).AddDays(-1) )
                        {
                            write-output "CRITICAL: There is an error in the log file $logfile around 
                                          $($error_time.ToShortTimeString())"; exit(2)
                        } 
                     }
      write-output "OK: There was no errors in the past 24 hours." 
    }
    CheckLogs "C:\Log.txt" #Function Call

    Content of my log file is as follows

    [ERROR] 2013-12-23 19:46:32
    [ERROR] 2013-12-24 19:46:35
    [ERROR] 2013-12-24 19:48:56
    [ERROR] 2013-12-24 20:13:07

    After executing above script, getting the below error, can you please correct me.

     $error_time = [datetime]($matches[1])
    +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray
     
    Cannot index into a null array.
    At C:\PS\LogTest.ps1:10 char:21
    +                     new-object psobject -Property @{Timestamp = 
    [datetime]$match ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray
     
    Cannot index into a null array.
    At C:\Test\LogTest.ps1:12 char:21
    +                     $error_time = [datetime]($matches[1])
    +                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [], RuntimeException
        + FullyQualifiedErrorId : NullArray


    Wednesday, December 25, 2013 2:01 AM
  • Sorry but this thread has been closed for two years.  Please start a new thread with a complete description of your problem.

    ¯\_(ツ)_/¯

    Wednesday, December 25, 2013 2:26 AM