Sign in
 
HomeOnline20132010Other VersionsLibraryForumsGalleryEHLO Blog
TechCenter >
> 
> 
Internal Mail from non-Outlook clients does not get tagged SCL -1

Beantwortet Internal Mail from non-Outlook clients does not get tagged SCL -1

  • Tuesday, May 01, 2012 4:02 PM
     
     
    Sign In to Vote
    0

    I'm trying to even out some issues we are having with Junk-Mail filtering and I'm trying to figure out why mail sent via SMTP clients is not tagged as SCL -1 whereas mail sent from Outlook Clients does get tagged that way.

    We have a number of internal users for whom using Outlook is not an option, and when they send mail internally it periodically ends up in the Junk-Mail folder of Outlook users.  We do not want to start maintaining whitelists.

    I know that Outlook respects the SCL -1 Header and won't junk internal mail for that reason, but why isn't that set when the user sends mail via SMTP.  They are authenticating, and using TLS, so it isn't being sent anonymously.

    Here are some sanitized headers where there's no SCL header for the SMTP client

    Received: from [10.10.10.100] (10.10.10.100) by mailserver.domain.local
     (10.1.1.25) with Microsoft SMTP Server (TLS) id 14.1.339.1; Tue, 1 May 2012
     11:37:21 -0400
    Message-ID: <4FA00328.9070103@domain.local>
    Date: Tue, 1 May 2012 11:37:12 -0400
    From: SMTPUser <SMTPUser@domain.local>
    User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120428 Thunderbird/12.0.1
    MIME-Version: 1.0
    To: <OutlookUser@domain.local>
    Subject: Test
    Content-Type: text/plain; charset="ISO-8859-1"; format=flowed
    Content-Transfer-Encoding: 7bit
    Return-Path: SMTPUser@domain.local
    X-MS-Exchange-Organization-AuthSource: mailserver.domain.local
    X-MS-Exchange-Organization-AuthAs: Internal
    X-MS-Exchange-Organization-AuthMechanism: 06
    X-Originating-IP: [10.10.10.100]
    X-MS-Exchange-Organization-AVStamp-Mailbox: MSFTFF;1;0;0 0 0

    Here are the headers from mail sent with the Outlook Client (Note the X-MS-Exchange-Organization-SCL: -1)

    Received: from mailserver.domain.local ([10.1.1.25]) by
     mailserver.domain.local ([10.1.1.25]) with mapi id 14.01.0339.001; Tue, 1
     May 2012 11:47:12 -0400
    Content-Type: application/ms-tnef; name="winmail.dat"
    Content-Transfer-Encoding: binary
    From: "User, Outlook" <OutlookUser@domain.local>
    To: <SMTPUser@domain.local>
    Subject: Test
    Thread-Topic: Test
    Thread-Index: Ac0nsaw8qtmBArjeQvqyJNCzA1h3pw==
    Date: Tue, 1 May 2012 11:47:11 -0400
    Message-ID: <882D80C21B295249A3EF5CA04D4934DE0671A3F9@mailserver.domain.local>
    Accept-Language: en-US
    Content-Language: en-US
    X-MS-Has-Attach: yes
    X-MS-Exchange-Organization-SCL: -1
    X-MS-TNEF-Correlator: <882D80C21B295249A3EF5CA04D4934DE0671A3F9@mailserver.domain.local>
    MIME-Version: 1.0
    X-MS-Exchange-Organization-AuthSource: mailserver.domain.local
    X-MS-Exchange-Organization-AuthAs: Internal
    X-MS-Exchange-Organization-AuthMechanism: 04
    X-Originating-IP: [172.29.3.101]
    X-MS-Exchange-Organization-AVStamp-Mailbox: MSFTFF;1;0;0 0 0


     

All Replies

  • Wednesday, May 02, 2012 7:20 AM
     
     
    Sign In to Vote
    0

    hi,

    What's the smtp client? Is it a app that you own develop? Try to use pop or owa to test, see if the scl value is still here. I suspect that the issue is cause by your smtp client.

    If not, we should analysis some logs.

    The SCL threshold configuration is used by the Content Filter agent, one of the default anti-spam agents included with Exchange 2010. The Content Filter agent uses Microsoft SmartScreen technology to assess the contents of a message and to assign an SCL rating to each message.

    So you can use the agent log and the pipline trace log to check whether the message has been scan by the content filter agent.

    You can send a message that include the logs to me, my email address is v-yutlu@microsoft.com

    hope can help you

    thanks,

    CastinLu

    TechNet Community Support

     
  • Thursday, May 03, 2012 4:24 PM
     
     
    Sign In to Vote
    0

    The SMTP client in my example above is Mozilla Thunderbird, but it's the same for all SMTP clients. 

    I should note that we do not have the Exchange Anti-Spam agents installed.  So I'm unsure of whether the Content Filter agent should, or should not be involved.

    I'll have to look into the agent log and pipeline log.  Is there specific documentation on configuring those for a trace, or is this kind of traffic captured by default?

    Thanks,
    Andrew

     
  • Monday, May 07, 2012 3:58 PM
     
     
    Sign In to Vote
    0

    Since you are not using the Content Filter then you are looking at the wrong thing. The fact that no SCL is being assigned to these emails further confirms that the SCL value (the spamminess rating used by the Content Filter) is not playing any role.

    So this means that emails are being moved to Junk by something else.

    I see 2 possiblities:
    1. EITHER these emails are being moved by the Outlook client side anti-spam filter
    2. OR the emails are being moved by some 3rd party spam filtering software

    To investigate Outlook you might want to try disabling the Junk Folder from the Outlook Side and see what happens.

    I think this is a good starting point for you to understand how the Junk Folder works:
    http://www.exchangeinbox.com/article.aspx?i=155

    IMF Tune - Anti-spam extending the Exchange 2003, 2007, 2010 IMF/Content Filter - http://www.windeveloper.com/imftune/

     
  • Monday, May 07, 2012 4:38 PM
     
     
    Sign In to Vote
    0

    Sorry, I wasn't clear.  We already know that it is the Outlook Junk-Mail filter that's junking the messages.  Due to a change in 2010 SP1, we can no longer add our entire @ourdomain.com to the Safe-Senders list (I understand the rationale here and don't have a problem with it).  Although we could add the individual addresses (someuser@ourdomain.com) to the safe senders list, that doesn't scale well.

    We don't want to disable the Outlook Junk-Mail filter, because it does capture actual junkmail. 

    Since the mail from our Outlook clients is already being tagged SCL -1, which is then not junked by Outlook's Junk-Mail filter, I'd like to figure out why our Authenticated mail from SMTP clients is not also tagged SCL -1.  They are users with mailboxes and are recognized as Internal (not external anonymous) users by the server as per the headers in my original message.

    Thanks,
    Andrew

     
  • Monday, May 07, 2012 5:26 PM
     
     
    Sign In to Vote
    0

    The answer to your question has a lot to do with the definition of Internal/External emails.
    Just because your SMTP client is using an Authenticated connection, that does not make it internal.
    AFAIK Exchange only considers a client as internal if you use a native client.
    So I don't know how you can solve this problem with your approach.

    Why don't you disable the Outlook Filter and use the server side Content Filter?
    IMO the server side solution is a lot easier to manage. For one the Content Filter won't scan authenticated connections by default. So this problem is immediately solved…

     

    IMF Tune - Anti-spam extending the Exchange 2003, 2007, 2010 IMF/Content Filter - http://www.windeveloper.com/imftune/

     
  • Tuesday, May 08, 2012 3:26 PM
     
     
    Sign In to Vote
    0
    I've been following this thread and would like to know how would you "Disable the Outlook Filter and use the server side content filter"??
     
  • Tuesday, May 08, 2012 5:32 PM
     
     
    Sign In to Vote
    0

    You disable the Outlook Filter using Group Policy.

    Enabling and using the Content Filter is explained in the links that follow. These were written for Exchange 2007. But they are still valid for Exchange 2010 since nothing really changed in the Content Filter

    The Exchange 2007 Content Filter Agent
    http://www.exchangeinbox.com/article.aspx?i=104

    Getting Started with the Exchange 2007 Content Filter
    http://www.exchangeinbox.com/article.aspx?i=135

    IMF Tune - Anti-spam extending the Exchange 2003, 2007, 2010 IMF/Content Filter - http://www.windeveloper.com/imftune/

     
  • Friday, May 11, 2012 3:26 PM
     
     
    Sign In to Vote
    0

    In our organization, a different group handles mail filtering (on appliances and via safe lists in Outlook and my group focuses on the mail server itself.  We can't turn off Junk-Mail filtering in Outlook and we don't have the Anti-Spam agents installed on Exchange right now, and it's probably not an area that we can touch right now. 

    I'm specifically limiting the scope of this issue as to how Exchange handles the decision to mark some internal mail as SCL -1 and others get no determination one way or the other.  That's what's causing most of our problem (due to how Outlook's Junk-Mail filter is so unreliable). 

    Thanks,
    Andrew

    • Marked As Answer by Castinlu Monday, May 14, 2012 1:45 AM
    • Unmarked As Answer by AndrewR_JMS Monday, May 14, 2012 4:23 AM
    •  
     
  • Monday, May 21, 2012 8:21 AM
     
     
    Sign In to Vote
    0

    did you ever find the answer to this?

    we're having a similar issue.  I have a transport rule that sets SCL to -1 and it IS being set on all mails, even those sent via SMTP.  However, outlook is still junking the emails sent via SMTP.

    I am using the "safe senders only" setting in outlook 2010 and our exchange server is 2010 SP1

    Bob


    • Edited by Bob Findlay Monday, May 21, 2012 8:22 AM
    •  
     
  • Wednesday, May 23, 2012 6:59 PM
     
     
    Sign In to Vote
    0

    @Andrew
    I believe you are on the wrong track. SMTP emails even if sent from the local network are not considered internal by Outlook. In this case an internal email is one originating from a native exchange client like Outlook or OWA.

     

    @Bob
    The Outlook spam filter makes no use of the SCL rating. So just because you are setting that header it won't stop it from filtering emails.

    If you want to stop Outlook filtering altogether you can use Group Policy.

    IMF Tune - Anti-spam extending the Exchange 2003, 2007, 2010 IMF/Content Filter - http://www.windeveloper.com/imftune/

     
  • Wednesday, May 23, 2012 7:19 PM
     
     
    Sign In to Vote
    0

    But it's not Outlook that sets the SCL of it's own outgoing mail.  The Exchange server makes some determination there doesnt' it?  So in that way, why wouldn't internal authenticated SMTP clients be treated the same way?  Exchange already catergorizes these as internal mail, separately than it categorizes the external SMTP sources. 

    As to Outlook's handling of SCL -1:

    http://blogs.technet.com/b/exchange/archive/2009/11/13/exchange-anti-spam-myths-revealed.aspx

    #3 states

    • Outlook uses its own SmartScreen filter technology separate from Exchange junk mail screening to filter junk mail. While the two sometimes agree Outlook ignores any SCL which Exchange may set on a message and uses its own criteria to determine the "spaminess" of a message. This is the client side filter (It should be noted here that Outlook will honor an SCL of -1. The above only applies to any SCL ratings other than -1.).

    So Bob's config should theoretically work.

     
  • Thursday, May 24, 2012 9:06 AM
     
     
    Sign In to Vote
    0

    Thanks Andrew.  That is borne out by observed results so I was surprised to read that SCL was supposedly totally ignored.

    But what is working for emails sent from outlook etc. is not working for anything that comes in via SMTP, even though SCL=-1 IS being set on those mails too.

    I don't want to stop outlook's filtering altogether, I just want it to honour the SCL of -1 on SMTP emails too.  We use extensive automation via SMTP and a lot of emails are being junked because of this.

    Bob

     
  • Thursday, May 24, 2012 10:01 AM
     
     
    Sign In to Vote
    0

    After some testing, I have found that if I authenticate my SMTP connection, the mail is not junked by outlook.

    Not sure how practical this is for all our applications, but it's something at least.

    Bob

     
  • Thursday, May 24, 2012 10:34 AM
     
     
    Sign In to Vote
    0

    ok, here's the solution

    authenticated SMTP mails had the following header value set

    X-MS-Exchange-Organization-AuthAs:Internal

    whereas non-auth have

    X-MS-Exchange-Organization-AuthAs:Anonymous

    you can therefore fake authenticated emails by using a transport rule to set specific classes of your email to have that header by using the following parameters on a create or set transport rule command

     -SetHeaderName 'X-MS-Exchange-Organization-AuthAs' -SetHeaderValue 'Internal'

    Bob

     
  • Monday, June 11, 2012 2:48 PM
     
     Answered
    Sign In to Vote
    1

    Funny, I was about to post something similar.

    Setting the permission ms-Exch-SMTP-Accept-Authentication-Flag for the specific group of users on the receive connector had the same effect for us.

    The command looks like

    Get-ReceiveConnector "YOURRECEIVECONNECTOR" | Set-ADPermission -User SOMEGROUP -ExtendedRights ms-Exch-SMTP-Accept-Authoritative-Domain-Sender

    We also, set the SCL for these message to -1 by transport rule.  Still in the process of testing to make sure Outlook is dealing with the -1 correctly.

    EDIT:  I've noted that it may be useful to apply that permission to "Authenticated Users".  That should cover all auth'd SMTP users, rather than creating and maintaining a group for them.

    • Marked As Answer by AndrewR_JMS Monday, June 11, 2012 2:48 PM
    •  
     
  • Thursday, October 25, 2012 4:10 PM
     
     
    Sign In to Vote
    0

    I'm trying to resolve an issue where I have IMAP/POP users sending messages through authenticated SMTP (Port 587), but since they're not getting stamped SCL -1 my spam software (GFI) is considering them "fair game" and processing them.

    I tried applying both the 'ms-Exch-SMTP-Accept-Authentication-Flag' and 'ms-Exch-SMTP-Accept-Authoritative-Domain-Sender' extended rights to my Receive Connector (Port 587), but the SCL -1 still isn't being applied to authenticated SMTP sessions (verified in Pipeline tracing). So did you need those permission settings at all?? Slightly confused..

    The only thing that "worked" was the less-than-ideal Transport Rule. I say "less-than-ideal" as GFI will mark it as spam and then the transport rule "fixes it" later, however that could make mail delivery troubleshooting difficult as the GFI logs would be inconsistent with what actually occured.

    Thanks,

    -g

     
  • Friday, October 26, 2012 9:04 AM
     
     
    Sign In to Vote
    0
    I don't recall setting those permissions, no.  My problem was not getting the scl -1 on there, but getting outlook to take notice of it.  sorry.

    Bob

     
  • Wednesday, January 23, 2013 6:11 PM
     
     
    Sign In to Vote
    0

    Bob,

    Did you have to authenticate your SMTP connection in conjunction with something else to get Outlook not to junk your mail?  I've authenticated an SMTP connection from my Domino environment to my Exchange environment.  These e-mails don't get an SCL of -1 and some of them still get moved to Junk in Outlook.  I don't have the anti-spam agents installed on my Hub Transport servers.  Is that the trick?

     
  • Thursday, January 24, 2013 10:07 AM
     
     
    Sign In to Vote
    0

    Just one thing that drove me crazy with the ipallowlist.

    Even if you have enabled and configured such feature, it won't work if you don't enable de "Content Filtering" feature. They appear as sepparate features but they are really dependant!!!. Nobody said that in any place that I can recall, it is a little obvious once you know, butttttttttt.....

    So, if you have configured IP allow list and you don't get the scl -1, check that Content Filtering is enabled....

    Hope this helps someone.