Server SBS2008 standard SP2 (non R2) Security log events recording over 1 million + logon/logoff events

Unanswered Server SBS2008 standard SP2 (non R2) Security log events recording over 1 million + logon/logoff events

  • Thursday, January 24, 2013 1:22 AM
     
     


    Hello everyone. I have been trying to tackle this for many months now and hopefully someone here can help.

    Since day 1 with a 2008 SBS server,  the server records 1 million + logon/logoff events a day.
    Now the customer has a requirement to be PCI compliant. (which is almost impossible with that volume of logs.)

    I tried to work with microsoft support on this and they didn't know why this is happening.



    Quick details...

    Server 2008 SBS standard SP2.
    ~30 workstations currently joined to domain.
    stand alone server

    Now I understand that there might be something else going on here but I'm at a loss to where I need to go. 

    Here are a few examples of what I'm swampped with. 

    Also, if I use auditpol to shut off logon/logoff, the logs go almost completely silent.
    But its one of the requirements for PCI to record them and keep them for a year.



    --------------------

    A Kerberos service ticket was requested.

    Account Information:
    Account Name: <dcname>$@<domain>.LOCAL
    Account Domain: <domain>.LOCAL
    Logon GUID: {7c9ea43a-cb21-5989-9790-fce1233a1e9f}

    Service Information:
    Service Name: krbtgt
    Service ID: <domain>\krbtgt

    Network Information:
    Client Address: ::1
    Client Port: 0

    Additional Information:
    Ticket Options: 0x60810010
    Ticket Encryption Type: 0x17
    Failure Code: 0x0
    Transited Services: -

    This event is generated every time access is requested to a resource such as a computer or a Windows service.  The service name indicates the resource to which access was requested.

    This event can be correlated with Windows logon events by comparing the Logon GUID fields in each event.  The logon event occurs on the machine that was accessed, which is often a different machine than the domain controller which issued the service ticket.

    Ticket options, encryption types, and failure codes are defined in RFC 4120.

    -------------------

    An account was successfully logged on.

    Subject:
    Security ID: NULL SID
    Account Name: -
    Account Domain: -
    Logon ID: 0x0

    Logon Type: 3

    New Logon:
    Security ID: SYSTEM
    Account Name: <dcname>$
    Account Domain: <domain>
    Logon ID: 0x108e18b82
    Logon GUID: {fc2c1d5d-ec46-8f87-b78e-ca1b2dff865e}

    Process Information:
    Process ID: 0x0
    Process Name: -

    Network Information:
    Workstation Name:
    Source Network Address: fe80::7c35:6f40:f3af:1e58
    Source Port: 29683

    Detailed Authentication Information:
    Logon Process: Kerberos
    Authentication Package: Kerberos
    Transited Services: -
    Package Name (NTLM only): -
    Key Length: 0

    This event is generated when a logon session is created. It is generated on the computer that was accessed.

    The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

    The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

    The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

    The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

    The authentication information fields provide detailed information about this specific logon request.
    - Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
    - Transited services indicate which intermediate services have participated in this logon request.
    - Package name indicates which sub-protocol was used among the NTLM protocols.
    - Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

    -----------------------------------------

    An account was logged off.

    Subject:
    Security ID: SYSTEM
    Account Name: <dcname>$
    Account Domain: <domain>
    Logon ID: 0x108e18b55

    Logon Type: 3

    This event is generated when a logon session is destroyed. It may be positively correlated with a logon event using the Logon ID value. Logon IDs are only unique between reboots on the same computer.

    -------------------------------------------

All Replies

  • Thursday, January 24, 2013 12:49 PM
    Moderator
     
     

    Can you check this?

    http://support.microsoft.com/kb/262177

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters


    Robert Pearman SBS MVP | www.titlerequired.com | www.itauthority.co.uk

  • Thursday, January 24, 2013 1:12 PM
     
     

    Ok to be an absolute blunt.

    1st thing --- the server records 1 million + logon/logoff events a day -- are these warnings , errors?  even if they are 1 million? Please try to be realistic.
    2nd thing -- Now the customer has a requirement to be PCI compliant. (which is almost impossible with that volume of logs.) --  logon/logoff as far as I remember is/was never a criteria to be PCI compliant.

    3rd coming from point 2 --- its one of the requirements for PCI to record them and keep them for a year?? - logon/logoff events? keep a record of errors, warning informational events ??

    Primary thing what PCI and your client has to understand is SBS is the standalone DC (Secure transactions  are never recommended or were never recommended on domain controllers)

     If you are asking for reducing the security event logs (even if it's warning or error) it's an herculean task to achieve ( Event you posted some of them are by design)

    If you are facing issues passing the PCI compliant test , to best of my knowledge Susan is the best person to contact.

  • Thursday, January 24, 2013 4:53 PM
     
     
    Robert, Turning that on resulted in many of these in event viewer.

    ---------------------------------------------

     
    A Kerberos Error Message was received:

    on logon session 

    Client Time: 

    Server Time: 16:45:33.0000 1/24/2013 Z

    Error Code: 0xe KDC_ERR_ETYPE_NOTSUPP

    Extended Error: 

    Client Realm: 

    Client Name: 

    Server Realm: AWA.LOCAL

    Server Name: krbtgt/AWA.LOCAL

    Target Name: krbtgt/AWA.LOCAL@AWA.LOCAL

    Error Text: 

    File: 9

    Line: e2d

    Error Data is in record data.
  • Thursday, January 24, 2013 5:05 PM
     
     

    1st thing --- the server records 1 million + logon/logoff events a day -- are these warnings , errors?  even if they are 1 million? Please try to be realistic.

    ---  Current count is 2,914,097 since Monday afternoon. They all show a success audit.

    2nd thing -- Now the customer has a requirement to be PCI compliant. (which is almost impossible with that volume of logs.) --  logon/logoff as far as I remember is/was never a criteria to be PCI compliant.

    ---  Requirement 10.2.4 states it. and 10.7 says to keep it for a year. 

    3rd coming from point 2 --- its one of the requirements for PCI to record them and keep them for a year?? - logon/logoff events? keep a record of errors, warning informational events ??

    --- Correct.  I have seen other installations of 2008 and their security events are far more quiet though. The reason the server is involved with compliance is because the systems that is going to handle credit card processing, will be backed up using this server as well as handle authentication to the systems.

  • Friday, January 25, 2013 1:05 AM
     
     

    Can you check this?

    http://support.microsoft.com/kb/262177

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters


    Robert Pearman SBS MVP | www.titlerequired.com | www.itauthority.co.uk

    I dug a little deeper on this and I see a few errors on my (possibly mis-configured) DNS server. 

    Old records weren't being dropped and I see multiple entries of different machine names for one IP.
    I think it might be whats causing this as aging isn't working  I'm going to correct this and report back.


  • Monday, January 28, 2013 7:17 PM
     
     

    Can you check this?

    http://support.microsoft.com/kb/262177

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa\Kerberos\Parameters


    Robert Pearman SBS MVP | www.titlerequired.com | www.itauthority.co.uk

    I dug a little deeper on this and I see a few errors on my (possibly mis-configured) DNS server. 

    Old records weren't being dropped and I see multiple entries of different machine names for one IP.
    I think it might be whats causing this as aging isn't working  I'm going to correct this and report back.


    Ok I cleaned up DNS and reviewed some of the advanced kerberos logs. 

    On the advanced logs I get a couple of  Error Code: 0xe KDC_ERR_ETYPE_NOTSUPP,  Server Name: krbtgt/<domain.local>
    But not too many. Maybe like 5-10 a minute

    On the security logs its still being flooded like mad. (2,000) since I started typing this.
    Many of the logs show machine accounts not actual users. But I do see alot of users as well and Service ID: <domain>\krbtgt

    what do you think?

    -Mike

    P.S.  Thanks again for all your help!!  :)

  • Monday, February 04, 2013 9:46 PM
     
     
    Anyone have any ideas?  I'm at a loss here.  :(
  • Tuesday, February 05, 2013 5:41 AM
    Moderator
     
     
    I meant to check that registry key and make sure it was disabled.

    Robert Pearman SBS MVP | www.titlerequired.com | www.itauthority.co.uk

  • Tuesday, February 05, 2013 5:44 AM
    Moderator
     
     
    How many computers do you have? What is the audit policy set to in the default domain controllers gpo?

    Robert Pearman SBS MVP | www.titlerequired.com | www.itauthority.co.uk

  • Wednesday, February 06, 2013 5:47 PM
     
     

    How many computers do you have? What is the audit policy set to in the default domain controllers gpo?
    --------------------------------------------------------------------------------

    Robert Pearman SBS MVP | www.titlerequired.com | www.itauthority.co.uk

    Well..  Currently ~25-30.  But AD shows more then that as some machines were either sold or retired.

     

    Here is the default domain controller GPO

    I tried my best to keep it clean for you.

    Policies
    --Windows Settings
    ----Security Settings
    ------Local Policies/Audit Policy

    Policy Setting
    Audit account logon events Success, Failure
    Audit logon events No auditing

    ------Local Policies/User Rights Assignment

    Access this computer from the network
    <domain>\QBDataServiceUser19, Everyone, BUILTIN\Administrators, NT AUTHORITY\Authenticated Users, NT AUTHORITY\ENTERPRISE DOMAIN CONTROLLERS, BUILTIN\Pre-Windows 2000 Compatible Access


    Add workstations to domain
    NT AUTHORITY\Authenticated Users


    Adjust memory quotas for a process
    <domain>\SQLServer2005MSSQLUser$<servername>$SCANMAIL, <domain>\SQLServer2005MSSQLUser$<servername>$SQLEXPRESS, NT AUTHORITY\LOCAL SERVICE, NT AUTHORITY\NETWORK SERVICE, <domain>\SQLServer2005MSSQLUser$<servername>$MICROSOFT##SSEE, <domain>\SQLServer2005MSSQLUser$<servername>$SBSMONITORING, <domain>\SQLServer2005MSFTEUser$<servername>$SBSMONITORING, BUILTIN\Administrators, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$ACT7, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$SCANMAIL, SafeTrac, <domain>\DCS_<servername>


    Allow log on locally
    BUILTIN\Administrators, BUILTIN\Backup Operators, BUILTIN\Account Operators, BUILTIN\Server Operators, BUILTIN\Print Operators

    Back up files and directories
    BUILTIN\Administrators, BUILTIN\Backup Operators, BUILTIN\Server Operators

    Bypass traverse checking
    <domain>\SQLServer2005MSSQLUser$<servername>$SCANMAIL, <domain>\SQLServer2005MSSQLUser$<servername>$SQLEXPRESS, <domain>\QBDataServiceUser19, Everyone, NT AUTHORITY\LOCAL SERVICE, NT AUTHORITY\NETWORK SERVICE, <domain>\SQLServer2005MSSQLUser$<servername>$MICROSOFT##SSEE, <domain>\SQLServer2005MSSQLUser$<servername>$SBSMONITORING, <domain>\SQLServer2005MSFTEUser$<servername>$SBSMONITORING, BUILTIN\Administrators, NT AUTHORITY\Authenticated Users, BUILTIN\Pre-Windows 2000 Compatible Access, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$ACT7, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$SCANMAIL, <domain>\DCS_<servername>

    Change the system time
    NT AUTHORITY\LOCAL SERVICE, BUILTIN\Administrators, BUILTIN\Server Operators

    Create a pagefile
     BUILTIN\Administrators

    Debug programs
    BUILTIN\Administrators

    Enable computer and user accounts to be trusted for delegation
    BUILTIN\Administrators

    Force shutdown from a remote system
    BUILTIN\Administrators, BUILTIN\Server Operators

    Generate security audits
    NT AUTHORITY\LOCAL SERVICE, NT AUTHORITY\NETWORK SERVICE, SafeTrac, <domain>\DCS_<servername>

    Increase scheduling priority
    BUILTIN\Administrators

    Load and unload device drivers
    BUILTIN\Administrators, BUILTIN\Print Operators

    Log on as a batch job
    <domain>\SQLServer2005MSSQLUser$<servername>$SCANMAIL, <domain>\SQLServer2005MSSQLUser$<servername>$SQLEXPRESS, <domain>\SQLServer2005MSSQLUser$<servername>$MICROSOFT##SSEE, <domain>\SQLServer2005MSSQLUser$<servername>$SBSMONITORING, <domain>\SQLServer2005MSFTEUser$<servername>$SBSMONITORING, BUILTIN\Administrators, BUILTIN\Backup Operators, BUILTIN\Performance Log Users, BUILTIN\IIS_IUSRS, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$ACT7, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$SCANMAIL, <domain>\DCS_<servername>

    Manage auditing and security log
    <domain>\Exchange Servers, <domain>\administrator, BUILTIN\Administrators

    Modify firmware environment values
    BUILTIN\Administrators

    Profile single process
    BUILTIN\Administrators

    Profile system performance
    BUILTIN\Administrators

    Remove computer from docking station
    BUILTIN\Administrators

    Replace a process level token
    <domain>\SQLServer2005MSSQLUser$<servername>$SCANMAIL, <domain>\SQLServer2005MSSQLUser$<servername>$SQLEXPRESS, NT AUTHORITY\LOCAL SERVICE, NT AUTHORITY\NETWORK SERVICE, <domain>\SQLServer2005MSSQLUser$<servername>$MICROSOFT##SSEE, <domain>\SQLServer2005MSSQLUser$<servername>$SBSMONITORING, <domain>\SQLServer2005MSFTEUser$<servername>$SBSMONITORING, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$ACT7, <domain>\SQLServer2005MSSQLUser$WIN-EUGSO7LO7PY$SCANMAIL, SafeTrac, <domain>\DCS_<servername>

    Restore files and directories
    BUILTIN\Administrators, BUILTIN\Backup Operators, BUILTIN\Server Operators

    Shut down the system
    BUILTIN\Administrators, BUILTIN\Backup Operators, BUILTIN\Server Operators, BUILTIN\Print Operators

    Take ownership of files or other objects
    BUILTIN\Administrators


    ------Local Policies/Security Options

    Domain Controller
    Policy Setting
    Domain controller: LDAP server signing requirements None

    Domain Memberhide
    Policy Setting
    Domain member: Digitally encrypt or sign secure channel data (always) Enabled

    Microsoft Network Serverhide
    Policy Setting
    Microsoft network server: Digitally sign communications (always) Enabled
    Microsoft network server: Digitally sign communications (if client agrees) Enabled

    Network Securityhide
    Policy Setting
    Network security: LAN Manager authentication level Send NTLMv2 response only

    Thanks a bunch!!!!

  • Friday, February 08, 2013 8:00 PM
     
     

    I don't know if this will help or not.
    As I continue to dig around, I shut off all logon/logoff events to see if I see anything else interesting.

    I did find one thing I don't know if it will help the cause. 
    In one period of 5 seconds I about 500 of these 2 items each alternating.
    It shows the client address using an IPV6 loopback address.

    Should I try shutting off IPV6? We don't use it here.


    ------------------------------------------------------------

    A Kerberos service ticket was requested.

    Account Information:
     Account Name:  AWAPDC$@AWA.LOCAL
     Account Domain:  AWA.LOCAL
     Logon GUID:  {00000000-0000-0000-0000-000000000000}

    Service Information:
     Service Name:  krbtgt/AWA.LOCAL
     Service ID:  NULL SID

    Network Information:
     Client Address:  ::1
     Client Port:  0

    Additional Information:
     Ticket Options:  0x60810010
     Ticket Encryption Type: 0xffffffff
     Failure Code:  0xe
     Transited Services: -

    This event is generated every time access is requested to a resource such as a computer or a Windows service.  The service name indicates the resource to which access was requested.

    This event can be correlated with Windows logon events by comparing the Logon GUID fields in each event.  The logon event occurs on the machine that was accessed, which is often a different machine than the domain controller which issued the service ticket.

    Ticket options, encryption types, and failure codes are defined in RFC 4120.

    ------------------------------------------------------------

    And these

    ------------------------------------------------------------

    A Kerberos service ticket was requested.

    Account Information:
     Account Name:  AWAPDC$@AWA.LOCAL
     Account Domain:  AWA.LOCAL
     Logon GUID:  {4a3cffe2-9a22-04b4-006a-ddff14a75f32}

    Service Information:
     Service Name:  krbtgt
     Service ID:  AWA\krbtgt

    Network Information:
     Client Address:  ::1
     Client Port:  0

    Additional Information:
     Ticket Options:  0x60810010
     Ticket Encryption Type: 0x17
     Failure Code:  0x0
     Transited Services: -

    This event is generated every time access is requested to a resource such as a computer or a Windows service.  The service name indicates the resource to which access was requested.

    This event can be correlated with Windows logon events by comparing the Logon GUID fields in each event.  The logon event occurs on the machine that was accessed, which is often a different machine than the domain controller which issued the service ticket.

    Ticket options, encryption types, and failure codes are defined in RFC 4120.

  • Friday, February 15, 2013 5:00 PM
     
     

    Well for giggles I loaded up a virtual machine and installed a clean copy of 2008 SBS.

    After the install completed, 100,000 events in the security logs....

    I let it run for a day, another 100,000. This is with not a single computer joined to the domain.

    I think I'm going to throw in the towel at this point and look at upgrading the OS to a newer version.

    Thanks again for all your help Robert.

    -Mike