locked
Over 700 messages in Shadow Redundancy queue after migrating to Exchange 2016 from Exchange 2010 RRS feed

  • Question

  • We migrated on on-premises Exchange 2010 server to two Exchange 2016 servers this past weekend.  We run a hybrid environment with Office 365.  Two 2016 server roles are EX1 (everything but Edge Transport) and EX2 (Edge Transport).  While troubleshooting a problem with fax automation, we discovered that the Shadow Redundancy queue on EX1 has over 700 messages.  All messages have four things in common.

    1. The recipient is external to our organization

    2. The message status for every message in the queue list Ready.  When viewing the message details, the Recipient Info status is Complete.

    3. The message was generated by an in-house applications and uses a DoNotReply sender address.

    4. All messages have the same Last Error as viewed in Recipient info: [{LED=250 2.1.5 Recipient OK};{MSG=};{FQDN=};{IP=};{LRT=}].  The Last Error field in the queue list is blank.

    The number of messages in the queue is growing.  I've never seen this problem before, so I'm hoping someone has seen it before and solved it.

    TIA

    Tom

    Wednesday, April 3, 2019 11:47 AM

All replies

  • We migrated on on-premises Exchange 2010 server to two Exchange 2016 servers this past weekend.  We run a hybrid environment with Office 365.  Two 2016 server roles are EX1 (everything but Edge Transport) and EX2 (Edge Transport).  While troubleshooting a problem with fax automation, we discovered that the Shadow Redundancy queue on EX1 has over 700 messages.  All messages have four things in common.

    1. The recipient is external to our organization

    2. The message status for every message in the queue list Ready.  When viewing the message details, the Recipient Info status is Complete.

    3. The message was generated by an in-house applications and uses a DoNotReply sender address.

    4. All messages have the same Last Error as viewed in Recipient info: [{LED=250 2.1.5 Recipient OK};{MSG=};{FQDN=};{IP=};{LRT=}].  The Last Error field in the queue list is blank.

    The number of messages in the queue is growing.  I've never seen this problem before, so I'm hoping someone has seen it before and solved it.

    TIA

    Tom

    what happens if you restart the transport service?
    Wednesday, April 3, 2019 12:48 PM
  • Thank you for the quick reply.

    We have restarted the transport service multiple times.  We've rebooted the server.  No apparent change to this queue.  It just keeps growing.

    Wednesday, April 3, 2019 1:02 PM
  • Thank you for the quick reply.

    We have restarted the transport service multiple times.  We've rebooted the server.  No apparent change to this queue.  It just keeps growing.

    Ok, its important to mention all that has been done when posting :) 

    Did this start when you migrated?

    I suspect something is "stuck"

    You could set the discard time to a lower value temporarily

    https://docs.microsoft.com/en-us/powershell/module/exchange/mail-flow/set-transportconfig?view=exchange-ps

    The ShadowMessageAutoDiscardInterval parameter specifies how long a server retains discard events for shadow messages. A primary server queues discard events until queried by the shadow server. However, if the shadow server doesn't query the primary server for the duration specified in this parameter, the primary server deletes the queued discard events.

    To specify a value, enter it as a time span: dd.hh:mm:ss where dd = days, hh = hours, mm = minutes, and ss = seconds.

    Valid input for this parameter is 00:00:05 to 90.00:00:00. The default value is 2.00:00:00 or 2 days.

    or set temporarily to 

    LocalOnly

    The ShadowMessagePreferenceSetting parameter specifies the preferred location for making a shadow copy of a message. Valid values are:

    • LocalOnly: A shadow copy of the message should only be made on a server in the local Active Directory site.

    • RemoteOnly: A shadow copy of the message should only be made on a server in a different Active Directory site.

    • PreferRemote: Try to make a shadow copy of the message in a different Active Directory site. If the operation fails, try make a shadow copy of the message on a server in the local Active Directory site.

    Then restart transport services and see if that pops it and clears up

    • Proposed as answer by Dawn Zhou Monday, April 8, 2019 9:48 AM
    Wednesday, April 3, 2019 1:15 PM
  • Thank you for the quick response.  We will try modifying the discard time.  I am wondering since we went from one on-premises Exchange 2010 server to two on-premises Exchange 2016 servers, and that the purpose of the Shadow Redundancy queue is to ensure a message makes it to the next hop (which would be the Edge Transport server), if we have a communication problem between the Edge Transport and the Hub Transport?

    Thanks,

    Tom

    Wednesday, April 3, 2019 1:36 PM
  • Thank you for the quick response.  We will try modifying the discard time.  I am wondering since we went from one on-premises Exchange 2010 server to two on-premises Exchange 2016 servers, and that the purpose of the Shadow Redundancy queue is to ensure a message makes it to the next hop (which would be the Edge Transport server), if we have a communication problem between the Edge Transport and the Hub Transport?

    Thanks,

    Tom

    I would say for sure its something there on the Edge. Im thinking setting temporarily to LocalOnly is worth trying as well.  
    Wednesday, April 3, 2019 1:59 PM
  • Hi,

    The Ready status means that the message is waiting in the queue and is ready to be processed. It seems that those messages are not being delivered currently.

    Besides what Andy suggested, you can also check the delivery details running the following command or check if there is any related error in Event Viewer.

    Get-MessageTrackingLog -MessageSubject <subject> -Recipients <recipientAddress> -Sender <senderAddress>


    Regards,

    Dawn Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Thursday, April 4, 2019 7:30 AM
  • Hi Tom,

    How is everything going? If there is anything unclear or any update, please feel free to let us know. 

     

    Regards,

    Dawn Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Click here to learn more. Visit the dedicated forum to share, explore and talk to experts about Microsoft Teams.

    Tuesday, April 9, 2019 8:26 AM
  • First, thanks to all who replied with suggestions.  We tried all of them without luck.  One thing notable about message tracking, we could find no records of the messages in message tracking!  We did fix the problem.  We deleted the fax server send connector and re-created it.  The queue emptied in 5 minutes.  

    Thanks to all

    • Proposed as answer by Dawn Zhou Tuesday, April 23, 2019 1:48 AM
    Monday, April 22, 2019 7:04 PM
  • Glad that you solve the issue! Always feel free to mark your solution as answer. That will be helpful for other users that have the same issue.

    Thanks for your sharing.

    Regards, 

    Dawn Zhou


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnsf@microsoft.com.

    Click here to learn more. Visit the dedicated forum to shareexplore and talk to experts about Microsoft Teams.

    Tuesday, April 23, 2019 1:48 AM