none
I'm having trouble with basic Exchange 2013 mail flow troubleshooting

    Question

  • We recently upgraded from Exchange 2007 to 2013.  I'm still adjusting to the many changes.  I'm trying to troubleshoot a problem where one of our managers is reporting that an external client isn't getting all expected mail from us.  We have a SharePoint application that sends automated replies.

    This particular end-user made 4 inquiries, and says they only received one response from us instead of 4.

    So in Exchange 2007 I would open the toolbox, open the mail flow troubleshooter, and enter the recipients email address to see how many times my transport server tried to send mail.

    In this case, if I saw 4 entries, I could confirm that our application did indeed hit the relay server 4 times, then I could start looking in other areas, like on their end, as to why they didn't receive them.  

    I have no NDR's coming back from the recipient.  This recipient has been receiving mails from us for years.  

    The mail flow delivery reports lone-tool in EAC only seem to work when there is a mailbox involved, but in this case the local SMTP service on my Sharepoint server is directly contacting my transport server and relaying mail to the outside.

    So i tried looking at the transport logs.  I did a test email and found the below log data from me sending a test email with subject "Test2"

    The only thing i can see that looks fishy is the "No suitable shadow servers,,SMTP,HAREDIRECTFAIL" but I don't even know if that is an issue.  I used the MX record, then tried telnetting to their mail server on port 25 from mine and did a manual test which seemed to work just fine.  Is the below method the only way for me to look and see if the older missing emails were relayed off my server?

    MSGTRK2015092418-1.LOG(473): 2015-09-24T18:11:57.027Z,,,,Arthas,No suitable shadow servers,,SMTP,HAREDIRECTFAIL,5952824672375,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,,15466,1,,,Test2,srubin@ibts.org,srubin@ibts.org,,Originating,,,,S:DeliveryPriority=Normal;S:AccountForest=ibts.org
    MSGTRK2015092418-1.LOG(474): 2015-09-24T18:11:57.137Z,192.168.25.12,ARTHAS.ibts.org,192.168.25.12,Arthas,08D2C509C256A5F3;2015-09-24T18:11:57.012Z;0,ARTHAS\Default Hub connector,SMTP,RECEIVE,5952824672375,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,,15466,1,,,Test2,srubin@ibts.org,srubin@ibts.org,0cI: ,Originating,,192.168.20.145,fe80::1dff:eb3:dbfd:d269%21,S:FirstForestHop=ARTHAS.ibts.org;S:DeliveryPriority=Normal;S:AccountForest=ibts.org
    MSGTRK2015092418-1.LOG(475): 2015-09-24T18:11:57.152Z,,Arthas,,,,,AGENT,AGENTINFO,5952824672375,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,,19735,1,,,Test2,srubin@ibts.org,srubin@ibts.org,,Originating,,192.168.20.145,fe80::1dff:eb3:dbfd:d269%21,S:CompCost=|ETR=0;S:DeliveryPriority=Normal;S:AccountForest=ibts.org
    MSGTRK2015092418-1.LOG(476): 2015-09-24T18:11:57.152Z,,,,Arthas,ContentConversion,,ROUTING,TRANSFER,5952824672376,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,,14682,1,,5952824672375,Test2,srubin@ibts.org,srubin@ibts.org,,Originating,,,,S:DeliveryPriority=Normal;S:AccountForest=ibts.org
    MSGTRK2015092418-1.LOG(477): 2015-09-24T18:11:58.371Z,192.168.25.12,Arthas,207.67.116.86,mx02.rels.info,;250 B56043ce80000 Message accepted for delivery;ClientSubmitTime:,To Internet,SMTP,SEND,5952824672376,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,250 recipient ok <1004cprocessing@rels.info>,14790,1,,,Test2,srubin@ibts.org,srubin@ibts.org,2015-09-24T18:11:56.027Z;SRV=ARTHAS.ibts.org:TOTAL-SUB=0.984|SA=0.968|MTSSDA=0.002|MTSSDC=0.005;MTSS|MTSSD;SRV=ARTHAS.ibts.org:TOTAL-HUB=1.359|SMRDI=0.003|SMRCL=0.082|SMRC=0.082|SMR=0.085|CATRS-Index Routing Agent=0.010|CATRS=0.011|CATRT-Journal Agent=0.003|CATRT=0.003|CCC=0.005|CAT=0.021|QDE=0.213|SMSC=0.244|SMS=0.460,Originating,,,,S:E2ELatency=2.344;S:ExternalSendLatency=1.125;S:Microsoft.Exchange.Transport.MailRecipient.RequiredTlsAuthLevel=Opportunistic;S:Microsoft.Exchange.Transport.MailRecipient.EffectiveTlsAuthLevel=EncryptionOnly;S:DeliveryPriority=Normal;S:AccountForest=ibts.org
    MSGTRKMS2015092418-1.LOG(115): 2015-09-24T18:11:57.012Z,fe80::1dff:eb3:dbfd:d269,ARTHAS.ibts.org,fe80::1dff:eb3:dbfd:d269%21,Arthas,08D2B4C6F8D41B7B,,STOREDRIVER,RECEIVE,0,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,To,14876,1,,,Test2,srubin@ibts.org,srubin@ibts.org,04I: ,Originating,,192.168.20.145,fe80::1dff:eb3:dbfd:d269%21,S:MailboxDatabaseGuid=2a2fec18-ee56-4e39-b985-ca587e33279e;S:ItemEntryId=00-00-00-00-CB-C7-4A-72-27-EE-62-45-8F-7F-ED-9F-2D-4B-6C-0D-07-00-A5-42-E0-1B-80-F1-E2-4E-BF-2A-CF-39-C9-28-19-E1-00-27-4A-D1-BB-D9-00-00-F2-B5-1E-2C-A0-07-36-41-AD-18-07-68-12-29-51-93-00-00-1C-65-59-19-00-00;S:DeliveryPriority=Normal;S:AccountForest=ibts.org
    MSGTRKMS2015092418-1.LOG(116): 2015-09-24T18:11:57.137Z,fe80::1dff:eb3:dbfd:d269%21,ARTHAS,,ARTHAS.ibts.org,"MDB:2a2fec18-ee56-4e39-b985-ca587e33279e, Mailbox:6a63e51b-ad61-4da3-8142-5a2457d693d7, Event:29547146, MessageClass:IPM.Note, CreationTime:2015-09-24T18:11:56.027Z, ClientType:MOMT",,STOREDRIVER,SUBMIT,,<249e53b07a54436e8598c23cc3f20da9@Arthas.ibts.org>,1c3f8cd2-5b83-491a-af72-08d2c50ba240,1004cprocessing@rels.info,,,1,,,Test2,srubin@ibts.org,,2015-09-24T18:11:56.027Z;LSRV=ARTHAS.ibts.org:TOTAL-SUB=1.109|SA=0.968|MTSSDA=0.002|MTSSDC=0.005|SMSC=0.006|SMS=0.085|MTSSDMO=0.092|MTSSDPL=0.003|MTSSDSS=0.004|MTSSD=0.109|MTSS=0.109,Originating,,192.168.20.145,,S:ItemEntryId=00-00-00-00-CB-C7-4A-72-27-EE-62-45-8F-7F-ED-9F-2D-4B-6C-0D-07-00-A5-42-E0-1B-80-F1-E2-4E-BF-2A-CF-39-C9-28-19-E1-00-27-4A-D1-BB-D9-00-00-F2-B5-1E-2C-A0-07-36-41-AD-18-07-68-12-29-51-93-00-00-1C-65-59-19-00-00

    Any help is greatly appreciated!

    Thursday, September 24, 2015 6:44 PM

Answers

All replies

  • Hi,

    Please go through the below in order to troubleshoot this issue:

    1) Check the message tracking logs for emails sent to this recipient. Use Get-MessageTrackingLog: https://technet.microsoft.com/en-us/library/aa997573%28v=exchg.150%29.aspx?f=255&MSPPError=-2147217396. Please post the output of Get-MessageTrackingLog | fl for one of the emails that was not received by the recipient. 

    2) Check that there are no emails stuck in the queues on Exchange: Get-Queue | fl

    3) Check if emails are queued up on your smart host if you are using one. 

    4) Do message tracking on your smart host if you use one.

    5) Check reverse DNS is configured correctly for your internet facing SMTP server: http://markgossa.blogspot.com/2015/09/exchange-2007-2013-reverse-dns.html.

    6) Check you have a valid SPF record configured and that your internet facing SMTP server is a permitted sender: http://markgossa.blogspot.com/2015/08/understanding-spf-records-part-1.html.

    Let me know how you get along.

    Thanks.


    Mark Gossa

    MCSE 2003, MCITP Enterprise Administrator 2008 R2, MCSA 2012 R2, MCTS Exchange 2010

    Blog: http://markgossa.blogspot.com

    Posts are provided “AS IS” without warranty of any kind, either expressed or implied.

    Thursday, September 24, 2015 10:44 PM
  • Hi,

    Please confirm whether the issue only happens to the message sending by SharePoint server. Check if user mailbox in Exchange server can send internal and external messages properly.

    We can follow Mark's suggestion to collect some logs for further analysis. Additionally, please check the related configuration in SharePoint side for the message sending.

    Regards,


    Please remember to mark the replies as answers if they help, and unmark the answers if they provide no help. If you have feedback for TechNet Support, contact tnmff@microsoft.com.

    Winnie Liang
    TechNet Community Support

    Friday, September 25, 2015 8:03 AM
    Moderator
  • Hello,

    Below provide link (Remote connectivity Analyzer) can trace the error.

    https://testconnectivity.microsoft.com/

    Also please check the event viewer and message tracking logs.

    Regards,

    Praveen

    ----------------------------------------------------------------------

    Remember to mark as helpful if you find my contribution useful or as an answer if it does answer your question.That will encourage me - and others - to take time out to help you Check out my latest blog posts @ Techrid.com

    Friday, September 25, 2015 12:28 PM
  • I think you guys all missed my details.  Let me address your questions first.

    #2, yes I checked the queues, the mails aren't there.  It's almost like they either never hit the hub/relay server, or they went out and disappeared into the internet with no NDR.

    #3 No smart host

    #4 see #3

    #5 Reverse DNS for arthas.ibts.org is present and correct

    #6 I didn't know what an SPF record was until I just looked it up.  When I use an external tool to look up spf records, I get "No valid SPF record found of either type TXT or type SPF" for ibts.org

    For your other guys questions, there is no local mailbox.  I have a sharepoint server, we'll call it server.ibts.org, that uses my hub/mb server arthas.ibts.org to relay out to the internet.  There is no other hops or devices, DNS/MX records and internet routing take it from there.

    Also, this client does receive some email from us successfully.  I sent 4 test emails and I confirmed with the client they received them.  That's sending from my Exchange mailbox though, not from the sharepoint server but some sharepoint emails do get to them.

    So in a nutshell, I want to be able to see how many emails have been relayed off of my hub/mb server to this email address or domain.  With 2007 that was easy using the message tracking tool.  I could give it a date range, and the remote email address and it would give me a list of all emails sent to that external address in that time frame.  Then I could match the dates and send-result with what our application thinks it sent.  If I don't see them in the list I can go back to the developers and say "hey these emails didn't go out because my mail server never received them from SharePoint".

    If they did go out, I can say "hey SharePoint sent them, and there is no delivery error, they just disappeared out on the internet or maybe the remote mail system has them queued or has a problem"

    Thanks,

    Friday, September 25, 2015 6:11 PM
  • I think you guys all missed my details.  Let me address your questions first.

    #2, yes I checked the queues, the mails aren't there.  It's almost like they either never hit the hub/relay server, or they went out and disappeared into the internet with no NDR.

    #3 No smart host

    #4 see #3

    #5 Reverse DNS for arthas.ibts.org is present and correct

    #6 I didn't know what an SPF record was until I just looked it up.  When I use an external tool to look up spf records, I get "No valid SPF record found of either type TXT or type SPF" for ibts.org

    For your other guys questions, there is no local mailbox.  I have a sharepoint server, we'll call it server.ibts.org, that uses my hub/mb server arthas.ibts.org to relay out to the internet.  There is no other hops or devices, DNS/MX records and internet routing take it from there.

    Also, this client does receive some email from us successfully.  I sent 4 test emails and I confirmed with the client they received them.  That's sending from my Exchange mailbox though, not from the sharepoint server but some sharepoint emails do get to them.

    So in a nutshell, I want to be able to see how many emails have been relayed off of my hub/mb server to this email address or domain.  With 2007 that was easy using the message tracking tool.  I could give it a date range, and the remote email address and it would give me a list of all emails sent to that external address in that time frame.  Then I could match the dates and send-result with what our application thinks it sent.  If I don't see them in the list I can go back to the developers and say "hey these emails didn't go out because my mail server never received them from SharePoint".

    If they did go out, I can say "hey SharePoint sent them, and there is no delivery error, they just disappeared out on the internet or maybe the remote mail system has them queued or has a problem"

    Thanks,

    Yea, you need to look at this simply. Enable SMTP protocol logging on the Exchange Send connector and the Exchange receive connector.

    https://technet.microsoft.com/en-us/library/bb124531(v=exchg.150).aspx

    So 2 things to verify. The receive connector logs will show what was sent from the Sharepoint Server to Exchange ( If anything)

    The send connector logs will show what the Exchange Server sent.

    Any failures or success should be recorded in those logs.


    Twitter!: Please Note: My Posts are provided “AS IS” without warranty of any kind, either expressed or implied.


    Friday, September 25, 2015 6:16 PM
  • Hi,

    Please follow Andy's advice and check the send and receive connector logs. 

    The receive connectors are often scoped by IP address so certain IPs connect to certain receive connectors. Do you have more than one SharePoint server? It's possible that one can relay through Exchange and the other can't. Also, if the SharePoint server is relaying email using the load balanced IP for multiple Exchange servers then please confirm that the receive connectors have the same configuration on all servers.

    If the SharePoint servers are denied connections or denied relay, you'll see these failures in the receive connector logs. 

    It's also worth checking the SMTP logs on the SharePoint server. Here you'll see the outbound connections to the Exchange server. 

    Let us know how it goes.

    Thanks.


    Mark Gossa

    MCSE 2003, MCITP Enterprise Administrator 2008 R2, MCSA 2012 R2, MCTS Exchange 2010

    Blog: http://markgossa.blogspot.com

    Posts are provided “AS IS” without warranty of any kind, either expressed or implied.

    Saturday, September 26, 2015 12:28 AM
  • Hello 

    Please check if emails are stuck in quarantine in Exchange 2013. If yes check the message header if anything you got not genuin. Also check Transport Rules in exchange 2013 EAC.


    MCSE Exchange 2013, MCSA 2012 Server MCTS Exchange 2007,2010, MCITP Exchange 2007,2010 MCSE 2003 Server, MCSA Exchange 2003 ITIL V3 Foundation https://ranaprem.wordpress.com/ This posting is provided AS IS with no warranties,and confers no rights.

    • Proposed as answer by Prem P Rana Saturday, September 26, 2015 5:00 AM
    Saturday, September 26, 2015 5:00 AM
  • There is only one Sharepoint server involved.

    I created a single receive connector way back when, for all of my SharePoint environments, and called it SharePoint.  All Sharepoint servers are scoped in this one connector, and no other environments are having problems sending mail.

    There are no load balancers involved.

    Thanks for the advice.  I was checking the logs before (my first post in this thread) and did see my tests.  Are those the right logs?

    Also, I'll look for quarantined files.  Not sure where those are but I'll look it up.

    Sunday, September 27, 2015 4:50 PM
  • There is only one Sharepoint server involved.

    I created a single receive connector way back when, for all of my SharePoint environments, and called it SharePoint.  All Sharepoint servers are scoped in this one connector, and no other environments are having problems sending mail.

    There are no load balancers involved.

    Thanks for the advice.  I was checking the logs before (my first post in this thread) and did see my tests.  Are those the right logs?

    Also, I'll look for quarantined files.  Not sure where those are but I'll look it up.

    SMTP protocol logs. Those are the ones you want to check. IF you see the test messages accepted by the Exchange Server, then you should see them in the message tracking logs for any internal Exchange mailboxes.

    If these messages are simply being relayed by the Exchange Server, then the protool logs shoud show they were sent to the next external hop, whatever that is.

    By the way, can you simply telnet on port 25 to the Exchange Server *from* the Sharepoint server and send a message to the outside? You may need to install the telnet client on the Sharepoint server. Doing that will probably tell you almost immediately what the problem is, assuming it fails.

    https://technet.microsoft.com/en-us/library/bb123686%28v=exchg.150%29.aspx?f=255&MSPPError=-2147217396


    Twitter!: Please Note: My Posts are provided “AS IS” without warranty of any kind, either expressed or implied.




    Sunday, September 27, 2015 4:58 PM
  • Thanks!  Again there are no local Exchange mailboxes involved.  Straight from SharePoint,  to the internet.  so the message tracking feature in EAC is useless to me because I have no mailbox to enter.

    Yes I have previously testing telnetting to port 25 from my sharepoint server to my hub/mb server and there are no problems even antering an external address.

    I guess the answer to my problem is to open the logs and find the entries.  My problem is knowing exactly which logs.  Also, there are so many logs.  It looks like my server is making 6 new logs an hour.  They look like this.

    MSGTRKMD2015092816-1

    MSGTRK2015092816-1

    MSGTRKMS2015092816-1

    MSGTRKMS2015092815-1

    CONNECTLOG20150927-1

    I don't really understand why MS would make people manually look through log data and take away tools that made this easy.  Regardless, are those the right logs I should be looking at?  Am I missing any?

    I used Textpad to to a multi-file search for my criteria which came up with the log blurb in my first post.

    Thanks,


    • Edited by Statistic Monday, September 28, 2015 5:41 PM Edit
    Monday, September 28, 2015 5:41 PM
  • You can find the location with powershell:

    Get-TransportService <server> |FL SendProtocolLogPath


    SendProtocolLogPath : C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Hub\ProtocolLog\SmtpSend



    Get-TransportService <server> |FL ReceiveProtocolLogPath


    ReceiveProtocolLogPath : C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\Logs\Hub\ProtocolLog\SmtpReceive


    Twitter!: Please Note: My Posts are provided “AS IS” without warranty of any kind, either expressed or implied.

    Monday, September 28, 2015 6:07 PM
  • Ok thank you.  Are there any tools out there that make these easy to read and search?
    Monday, September 28, 2015 6:08 PM
  • Log Parser  http://blogs.technet.com/b/exchange/archive/2013/06/17/log-parser-studio-2-2-is-now-available.aspx is popular, or excel is actually pretty useful here:

    http://social.technet.microsoft.com/wiki/contents/articles/23182.analyzing-the-protocol-logs-and-message-tracking-logs-in-exchange-2013.aspx



    Twitter!: Please Note: My Posts are provided “AS IS” without warranty of any kind, either expressed or implied.

    Monday, September 28, 2015 6:21 PM
  • I was just installing and configuring that.  Thanks :)
    Monday, September 28, 2015 6:42 PM