none
Getting fluentD log file monitoring working RRS feed

  • Question

  • Hi

    Anyone having any luck with fluentd logfile monitoring ? 

    The documentation is somewhat lacking.

    Tried setting it up but I get no indication what so ever if it's even running.

    It's not even creating a .pos file so Im guessing NO but can't find a single output from the agent/fluentd more than some cpu related enumeration.

    SCOM 1807, CENTOS 7.

    Friday, August 16, 2019 6:56 AM

Answers

  • Yes : 

    As you can see from Sample Configuration File, you first define the log file you want to monitor, an event ID and an event description on the linux server :

    <filter scom.log>
    
        # new scom fluentd plugin for simple match - Input – Pattern A;  Action: log record = A à Send Event
    
        type filter_scom_simple_match
    
        # Input Pattern to look for. In this case looking for Invalid guest user
        regexp1 message Invalid user guest
    
        # Event to be generated and sent to SCOM OMED service.
        event_id1 6201
    
        # Event description to be sent to SCOM
        event_desc Failed login attempt. Invalid User guest attempted to log in
    
    </filter>


    These events are then forwarded to the SCOM Management Server, in its event log, using the parameters defined in the configuration file.

    Then the alerting rule uses a Windows event provider to reads these events.

    The only strange concept here is that the alerting rule targets the class Unix.Computer, even though it actually runs on SCOM management servers. That's because instances of class Unix.Computer are "unhosted", which means they do not "live" on an agent but on a SCOM server; therefore every rule and monitor that are targetting this class are actually run on a SCOM server.

    The reason why you still target the rule at Unix.Computer is because it allows you to use properties of that classes as parameters to the datasource, more specifically here the Unix computer name.

    Since the Unix computer name is contained in the event, It allows the rule to trigger only  for the appropriate instance of Unix computer.

    Tuesday, August 20, 2019 2:08 PM

All replies

  • Hi,

    What is the documentation lacking exactly?

    Did you follow the documentation precisely step-by-step?
    Linux Log file monitoring in System Center Operations Manager

    You've might have noticed that there is also a sample management pack and sample configuration file provided in the documentation that you can test to see if it works, did you try them to see if you could get any results?

    Here's also an article explaining about the monitoring:
    System Center 1801 Operations Manager – Enhanced log file monitoring for Linux Servers

    Best regards,
    Leon


    Blog: https://thesystemcenterblog.com LinkedIn:


    Friday, August 16, 2019 11:19 AM
  • Hi

    Yes that is the step-by-step I have been following. 

    Some things i noticed with the guide is:

    It links only to command line installation of Linux agent, is that a must ? Or is the Discovery installation in 1807 enough ? (It says so further down)

    The config file <source> config uses "format" wich by fluentd documentation is depricated ? 

    The match section has a note about disabling server Auth...without saying if it's needed or when its needed.

    The Example management pack has "Target="Windows!Microsoft.Windows.Computer" in one rule and Linux in the other…

    And i can't find a single logfile telling me if the agent is running fluentd at all.

    This article has a screenshot of a seperate OMED event log. Do you know if that is supposed to be in 1807 becouse i dont have it on my scom servers. I can see that they are running the OMED service and are listening on the correct port though

    https://argonsys.com/microsoft-cloud/library/system-center-1801-operations-manager-enhanced-log-file-monitoring-for-linux-servers/ 

    Tuesday, August 20, 2019 6:15 AM
  • The Argonsys article is originated from the Microsoft Tech Community article in the link below and is actually written by the System Center team:
    https://techcommunity.microsoft.com/t5/System-Center-Blog/System-Center-1801-Operations-Manager-Enhanced-log-file/ba-p/351952

    Regarding the installation of the Linux agent, the documentation states the following:

    "Install the new Linux agent on the Linux servers, this can be done through discovery wizard or manually."

    The management pack has a rule that looks for all events from the new data source Microsoft.Linux.OMED.EventDataSource and generates alerts accordingly.

    When taking a look at the Microsoft.Linux.OMED.EventDataSource on the System Center Wiki, it does have a the following configuration: <LogName>System Center OMED Service</LogName>, so yes this event log should exist.

    "The OMEDService on SCOM Management server would receive an event on match of a log record along with the log record context. User would need to import a management pack on SCOM server which would generate alert when there is an event received from Linux Server."

    It might be that you don't have a match and therefore you don't see the event log.



    Blog: https://thesystemcenterblog.com LinkedIn:

    Tuesday, August 20, 2019 6:41 AM
  • Hi

    This is making somewhat more sense now, thanks. 

    Do you have an explenation on why the example MP is targeting Linux in one rule and Windows in the other using the same datasouce ?

    https://docs.microsoft.com/en-us/system-center/scom/manage-sample-management-pack?view=sc-om-1807

    Tuesday, August 20, 2019 9:11 AM
  • It's likely a typo in the doc, especially considering that right after that they use the variable $Target/Property[Type="Unix!Microsoft.Unix.Computer"]/NetworkName$ which implies that the target of the rule is Microsoft.Unix.Computer and not Windows.Computer
    • Edited by CyrAz Tuesday, August 20, 2019 12:10 PM
    Tuesday, August 20, 2019 12:09 PM
  • Hi

    Looks like i managed to the get fluentd part working. There were some problems with the agent that didn't show in the normal monitoring. So a complete purge and reinstall of the agent fixed that part somewhat.

    Then there is a Task for the Agent that is not in the documentation what i could find. 
    "Create FluentD SCOM Workspace" 
    After running that fluentd came to Life and started monitoring my logfile.

    There were still an issue with sending data to scom server:
    "Error:'Net::HTTP.Post raises exception: OpenSSL::SSL::SSLError, 'SSL_connect returned=1 errno=0 state=error: certificate verify failed''

    but "enable_server_auth false" fixed that.

    Not sure why the automaticly created certificates don't work with OMED. Threre are some extra step when using manual agent installation for it (https://docs.microsoft.com/en-us/system-center/scom/deploy-linux-agent-install?view=sc-om-1801#configure-certificates)

    After the SCOM server received the first event from my Linux agent the System Center OMED Service event log where createded and visable.

    Next step is figuring out the MP part. Still not sure what class to target. the eventlog is in windows so Linux as target seems strange.

    Tuesday, August 20, 2019 1:36 PM
  • Not so much when you realize that Linux monitoring is actually done at the SCOM management server level : all the workflows targeting a linux computer are actually run from Windows, which may then query linux agent using winrm if the datasource used involves it.

    So targeting Unix.Computer is till the way to go :)

    • Edited by CyrAz Tuesday, August 20, 2019 1:42 PM
    Tuesday, August 20, 2019 1:41 PM
  • Yea, but this rule reads from the eventlog on the SCOM server ?

    If i look at the datasource  Linux!Microsoft.Linux.OMED.EventDataSource

    It uses Windows!Microsoft.Windows.EventProvider to get the data:

        <DataSource TypeID="Windows!Microsoft.Windows.EventProvider" ID="Provider">
                    <ComputerName>.</ComputerName>
                    <LogName>System Center OMED Service</LogName>
                    <AllowProxying>true</AllowProxying>
                    ....................

    (I'm not saying you are wrong, I'm trying to learn why you aren't)


    Tuesday, August 20, 2019 1:45 PM
  • Yes : 

    As you can see from Sample Configuration File, you first define the log file you want to monitor, an event ID and an event description on the linux server :

    <filter scom.log>
    
        # new scom fluentd plugin for simple match - Input – Pattern A;  Action: log record = A à Send Event
    
        type filter_scom_simple_match
    
        # Input Pattern to look for. In this case looking for Invalid guest user
        regexp1 message Invalid user guest
    
        # Event to be generated and sent to SCOM OMED service.
        event_id1 6201
    
        # Event description to be sent to SCOM
        event_desc Failed login attempt. Invalid User guest attempted to log in
    
    </filter>


    These events are then forwarded to the SCOM Management Server, in its event log, using the parameters defined in the configuration file.

    Then the alerting rule uses a Windows event provider to reads these events.

    The only strange concept here is that the alerting rule targets the class Unix.Computer, even though it actually runs on SCOM management servers. That's because instances of class Unix.Computer are "unhosted", which means they do not "live" on an agent but on a SCOM server; therefore every rule and monitor that are targetting this class are actually run on a SCOM server.

    The reason why you still target the rule at Unix.Computer is because it allows you to use properties of that classes as parameters to the datasource, more specifically here the Unix computer name.

    Since the Unix computer name is contained in the event, It allows the rule to trigger only  for the appropriate instance of Unix computer.

    Tuesday, August 20, 2019 2:08 PM
  • Hi Nice explanation, now I get it! Thanks alot for explaining it all, I'v now got the string I'v configured in fluentd as an alert in scom. Next step is to tweek my source, filter and match to get the info I want.
    Tuesday, August 20, 2019 4:53 PM
  • Yes : 

    As you can see from Sample Configuration File, you first define the log file you want to monitor, an event ID and an event description on the linux server :

    <filter scom.log>
    
        # new scom fluentd plugin for simple match - Input – Pattern A;  Action: log record = A à Send Event
    
        type filter_scom_simple_match
    
        # Input Pattern to look for. In this case looking for Invalid guest user
        regexp1 message Invalid user guest
    
        # Event to be generated and sent to SCOM OMED service.
        event_id1 6201
    
        # Event description to be sent to SCOM
        event_desc Failed login attempt. Invalid User guest attempted to log in
    
    </filter>


    These events are then forwarded to the SCOM Management Server, in its event log, using the parameters defined in the configuration file.

    Then the alerting rule uses a Windows event provider to reads these events.

    The only strange concept here is that the alerting rule targets the class Unix.Computer, even though it actually runs on SCOM management servers. That's because instances of class Unix.Computer are "unhosted", which means they do not "live" on an agent but on a SCOM server; therefore every rule and monitor that are targetting this class are actually run on a SCOM server.

    The reason why you still target the rule at Unix.Computer is because it allows you to use properties of that classes as parameters to the datasource, more specifically here the Unix computer name.

    Since the Unix computer name is contained in the event, It allows the rule to trigger only  for the appropriate instance of Unix computer.

    Marked this as aswer, didn't really know what post to set but since MS proposed it i followed along.
    Even though that sample file also includes an error, "event_desc" should be "event_desc1" ;)

    Thursday, August 22, 2019 7:49 AM
  • You can mark multiple answers, if you feel like that different posts/posters helped you to find a solution :)
    Thursday, August 22, 2019 8:32 AM