none
FOPE ... weed out e-mail with certain foreign language characters ....

    Întrebare

  • Hello:

    I am trying to fine-tune our FOPE settings.  We had a prior spam filtering company (spamsoap.com) before moving to Exchange Online Plan 2 (Office 365 E-3 Plan).

    I am getting a lot of foreign character e-mail to our info@[our_domain_name] alias.  Most of it goes into our Junk E-mail Box in Outlook 2010 (and is at least primarily labeled "SPAM?" in the subject header by FOPE."

    Is there anyway to configure Outlook 2010 to filter out e-mail based on high propensity of certain foreign-language characters.

    I never receive bona fide e-mail from China and other Asian countries and would be fine with auto-deleting e-mail with a high propensity of such characters.  It appears that of the e-mail has Chinese characters.

    David

    FYI: I am based in the U.S. (English).

    26 februarie 2012 13:24

Toate mesajele

  • Hello HelpStar,

    In Office 365 we have a SPAM Action set to Add X-header which injects the header X-FOSE-spam in the email detected as SPAM by FOPE. Office 365 Servers see the header value injected and delivers these emails to the Junk Email folder in Outlook.

    Now understanding your concern you get emails comprsing a lot of foreign characters which should be deleted or rejected as per your requirement. I am not sure we have this provision in outlook but definitely FOPE can help you to achieve this.

    While configuring a Policy Rule we have provision a FOPE to identify the caharcter set used in an email. Please refer the article to go through the charcter set which can be blocked by FOPE : http://technet.microsoft.com/en-us/library/ff715218.aspx.

    Actio Plan to be followed here is :

    We need to analyze the headers and get the character sets used to draft the SPAM emails received by you.

    Once we get the character set we can select the same while configuring the Policy Rule in FOPE and select the Action to Reject. So any email coming to FOPE having the selected Character Set involved will get Rejected by FOPE.

    Now main point to be highlighted is : Emails rejcted by FOPE canot be retrieved at any cost.

    Here we can use a Smart option we can create a above mention Policy and set the Action to TEST or Quarantine in order to keep a track of False Positive or False Negative ratio implemented by configuring the Policy Rule.

    To know about Quarantine and Test Policy please follow the article (Understandig Policy Rule Settings): http://technet.microsoft.com/en-us/library/ff714983.aspx

    Thus when we are sure that the Policy Rule configured is upto mark and is accurate enough to catch the emails comprising the selected set of charaters we can change the Action to Reject.

    If you are facing any difficulty in understanding the Chaaracter set used in the SPAM emails received in Junk please paste the sample headers of those emails here in the forum so that I can analye them and provide you with the required information. If you require any immediate assistance please feel free to open a Service Ticket with FOPE Technical Supoprt Team using the Microsoft Online Portal.




    4 martie 2012 20:36
  • Thank you Pradeep.  It may take me a few days to digest your posting, but it looks very comprehensive and helpful.

    David

    5 martie 2012 15:55
  • Hi Pradeep:

    The two links you provided are very helpful.  Before selecting which Character Sets to set as FOPE Policies, I checked the e-mail headers of several of the e-mails.

    The first was very clear: the header read "Content-Type: text/plain; charset="iso-2022-jp"

    There were some, however, without the word "charset."  One e-mail read "Content-Type: multipart/mixed;
     boundary="----=_NextPart_000_016D_01E9D395.1C72AA30"

    How does one interpret the Character Set for that e-mail?

    -----------------------

    In any case, I ended up using the Edit button in the "Match – New Policy Rule section" of the Message section of Policy Rules which allowed me to check off Chinese and Japanese character sets em masse.

    QUESTION, where do I view the Quarantine for these messages?  I elected to start with that before moving to the "Reject" setting. Interested to see what is caught first.

    David

    13 martie 2012 17:49
  • Hi Pradeep:

    As a follow-on to my last question, is there a consistent place that I search for the Character Set is used for a particular incoming e-mail?  I noticed that some e-mail headers have a "Charset" entry while others don't.

    Below is a representative sample of one that does not.  I have changed the e-mail addresses for privacy purposes but this is otherwise the full header of a spam e-mail that I received earlier today.

    FYI, this was received after I implemented a Policy Rule that adds "Filter Test" to all Japanese character sets. It made it through the Policy Rule for that (notice that it does not have "Filter Test" in the subject line).  FYI, it hails from Japanese servers.

    ----------------------------

    Received: from CH1PRD0802HT007.namprd08.prod.outlook.com (10.42.110.142) by
     SN2PRD0802HT005.namprd08.prod.outlook.com (10.27.84.48) with Microsoft SMTP
     Server (TLS) id 14.15.45.0; Tue, 13 Mar 2012 20:03:55 +0000
    Received: from CH1PRD0802HT010.namprd08.prod.outlook.com (10.42.111.236) by
     CH1PRD0802HT007.namprd08.prod.outlook.com (10.42.110.142) with Microsoft SMTP
     Server (TLS) id 14.15.45.0; Tue, 13 Mar 2012 20:03:55 +0000
    Received: from mail96-tx2-R.bigfish.com (65.55.88.115) by
     CH1PRD0802HT010.namprd08.prod.outlook.com (10.42.111.236) with Microsoft SMTP
     Server (TLS) id 14.15.45.0; Tue, 13 Mar 2012 20:03:55 +0000
    Received: from mail96-tx2 (localhost [127.0.0.1]) by mail96-tx2-R.bigfish.com
     (Postfix) with ESMTP id 1B84F140378 for <info@domainname.com>; Tue, 13 Mar 2012
     20:03:56 +0000 (UTC)
    X-BigFish: ps49(zzbb2dIc89bhzz1202hzz8275bh1348cbkjz2fh2a8h668h839h8e2h8e3h)
    Subject: SPAM? i-Net Cruiser
    X-SpamScore: 49
    X-Forefront-Antispam-Report: CIP:202.172.28.45;KIP:(null);UIP:(null);IPV:NLI;H:s44.coreserver.jp;RD:s44.coreserver.jp;EFVD:NLI
    Received-SPF: pass (mail96-tx2: domain of s44.coreserver.jp designates 202.172.28.45 as permitted sender) client-ip=202.172.28.45; envelope-from=anonymous@s44.coreserver.jp; helo=s44.coreserver.jp ;oreserver.jp ;
    Received: from mail96-tx2 (localhost.localdomain [127.0.0.1]) by mail96-tx2
     (MessageSwitch) id 133166903462644_32680; Tue, 13 Mar 2012 20:03:54 +0000
     (UTC)
    Received: from TX2EHSMHS012.bigfish.com (unknown [10.9.14.240]) by
     mail96-tx2.bigfish.com (Postfix) with ESMTP id F26401E008A for
     <info@domainname.com>; Tue, 13 Mar 2012 20:03:53 +0000 (UTC)
    Received: from s44.coreserver.jp (202.172.28.45) by TX2EHSMHS012.bigfish.com
     (10.9.99.112) with Microsoft SMTP Server (TLS) id 14.1.225.23; Tue, 13 Mar
     2012 20:03:50 +0000
    Received: (qmail 16727 invoked by uid 10165); 14 Mar 2012 05:03:39 +0900
    Date: Wed, 14 Mar 2012 05:03:39 +0900
    Message-ID: <20120313200339.16726.qmail@s44.coreserver.jp>
    To: <info@ domainname.com>
    From: <ffffff@gmail.com>
    Content-Transfer-Encoding: 7bit
    MIME-Version: 1.0
    Content-Type: text/plain
    Return-Path: anonymous@s44.coreserver.jp
    X-MS-Exchange-Organization-SCL: 6
    X-MS-Exchange-Organization-AVStamp-Mailbox: MSFTFF;1;0;0 0 0
    X-MS-Exchange-Organization-AuthSource:
     CH1PRD0802HT010.namprd08.prod.outlook.com
    X-MS-Exchange-Organization-AuthAs: Anonymous

    --------------

    E-mail body looked like:

    “o˜^‚ªŠ®—¹‚µ‚Ü‚µ‚½¡

    íœ/•ÒW‚̍ۤ•K—v‚ȏî•ñ‚Å‚•¡

    ¥ID:17419

    ¥Ê߽ܰÄÞ:9500

    ¥“o˜^Žó•t“ú
    2012/03/14

    ‚ ‚È‚½‚ÌÍß°¼Þ‚Ɉȉº‚ÌØݸ‚ð‚¨Šè‚¢‚µ‚Ü‚•B

    ƒVƒƒƒh[ƒ{ƒNƒT[’Ê”Ì

    http://ponguefffffff.com/inefft13005

    ‚±‚ÌÒ°Ù‚Í‘åØ‚É•Û‘¶‚•‚鎖‚ð‚¨‚•‚•‚ß‚µ‚Ü‚•B

    13 martie 2012 21:27
  • Hello Yash,

    I am analyzing the headers and wil definitely asure that the next blog wil answer your question.

    Just 20 minutes and thats all.


    13 martie 2012 21:33
  • Hello Yash,

    Thaks a lot fopr sharing the headers. Analyzing these headers the infromation we get is

    Content-Transfer-Encoding: 7bit

    The basic Internet email transmission protocol, SMTP, supports only 7-bit ASCII characters (see also 8BITMIME). This effectively limits Internet email to messages which, when transmitted, include only the characters sufficient for writing a small number of languages, primarily English.

    Other languages based on the Latin alphabet typically include diacritics and are not supported in 7-bit ASCII, meaning text in these languages cannot be correctly represented in basic email.

    Now in this case you are using a Office 365 subscription secured with FOPE. The emails are coming from an email address.

    The email is coming from the email address Return-Path: anonymous@s44.coreserver.jp where we can resolve it is coming from Japenese Server.

    In Japan, Some major mail client does not support Content-Transfer-Encoding: quoted-printable. So in this scenario blocking a character Set wont be a ideal resolution. If we want to blco such kind of emails we can blcok it usign the email address, Connecting Server IP 202.172.28.45 etc. Referencing this we can create a allow or Quarantine Policy. I think this will reosolve your issue.

    With Office 365 we have a default SPAM action set to Add X-header where the SPAM email is been marked with a header X-Fose-spam and referencing that the email is been delivered to Junk Email folder in Outlook

    X-BigFish: ps49(zzbb2dIc89bhzz1202hzz8275bh1348cbkjz2fh2a8h668h839h8e2h8e3h)
    Subject: SPAM? i-Net Cruiser
    X-SpamScore: 49 

    Now this is a SPAM email as the Spam score assigned is 49 anbut we cannot find the header X-Fose-spam so i think the SPAM action for tht domain is change to Modify Subject and the keyword SPAM? is been added to the Subject Line. Addressing your query that you have created a Policy to modify the subject of the emails. This thing can only be done using Test Policy. Real time policies cannot modify the subject of that email. Now when we create or change a Policy Rule it will take 45-60 minutes to propagate and get implemented.

    I hope that addresses your query.


    13 martie 2012 22:12
  • QUESTION, where do I view the Quarantine for these messages?  I elected to start with that before moving to the "Reject" setting. Interested to see what is caught first.

    David

    -----------------------------------------------------------------------------------------------------------------------

    Hello David,

    I would like to inform you that SPAM emails are been delivered depending upon the SPAM action set for that domain. If the SPAM action is set to Spam Quarantine then only the SPAM emails marked by FOPE will be delivered to SPAM Quarantine mailbox. Policy Quarantine Mailbox is located at the same destination.

    To access the Policy or Spam Quarantine mailbox we need to login to quarantine.messaging.microsoft.com or if we login to admin.messaging.microsoft.com on the right hand side corner we take our cursor on the login address we get a drop down option to navigate to the Quarantine mailbox.

    In Office 365 Global Administrators do not have an access to SPAM Quarantine mailbox  by default. We need to perform a workaround and get an access so that we can view our and other users Quarantine mailbox.

    If we want the end users to access the Quarantine Mailbox we  need to set the Quarantine settings on the domain accordingly (just Enabling the Allow User Access).

    Please update if you have any doubts because the matter would extent if explained in detailed.



    13 martie 2012 22:20
  • Hi Pradeep:

    Indeed, a number of e-mail is still getting through to either my Inbox or Junk Mailbox after setting all charsets associated with Japanese, Korean, Chinese etc via a Test Policy.

    In other words, I am receiving some e-mail labeled "Filter Test" which is good and are true positives.

    But, I am still receiving some e-mail with Asian characters and not all of it is labeled "SPAM?" and put in the Junk E-mailbox.

    Since it is coming from multiple and changing IP addresses and e-mail addresses, filtering by one of those is NOT feasible.

    What other way can I set things so I do not receive this junk?

    I had the impression that FOPE was an industry leading product.  Since we did not have this problem with our prior spam filtering service, which correctly filtered out ALL of this junk and had no false positives, my guess it that there must be something we are missing.

    David

    19 martie 2012 15:29
  • Hi Pradeep:

    Have you seen my 3/19/12 reply with questions, thanks.  See below.

    David

    26 martie 2012 12:11
  • Hello David:

    I apologize for the delay caused. We also have an option of ASF which is very effective to block such kind of emails. If possible we can see the Message Body of the emails which are getting delivered and we can troubleshoot accordingly.

    Another thing is, try to find out some common patterns in the Message Body like any common word, common pattern of writting a word, common IP etc. If its possible for you please share the headers of those emails which came through so that I can analyze and try to find the common factor.

    As discussed regarding ASF options please follow this link for further details: http://technet.microsoft.com/en-us/library/ff714985.aspx

    I apologize once again for the delay caused. Hope you accept it.


    7 aprilie 2012 01:25
  • Hi Pradeep:

    >> If possible we can see the Message Body of the emails which are getting delivered and we can troubleshoot accordingly.

    I listed a few earlier in this thread.  I can send more, please advise.

    I did review, however, the ASF options at this URL -- http://technet.microsoft.com/en-us/library/ff714985.aspx .  Which one(s) were you thinking might be of help?  I would be happy to experiment with them (set ot "test").

    David

    9 aprilie 2012 13:09