locked
Retention Policy not working?? RRS feed

  • Question

  • I've created a retention policy + tag (for testing purposes) to delete old email older than 1 day from the deleted items and allow recovery assigned it to a user and I've run "start-managedfolderassistant -identity user".

    There are emails in the deleted items that are 4 weeks old that it is not deleting... What could be wrong? This is Exchange 2010 SP1.

    Monday, November 8, 2010 6:42 PM

Answers

  • I've been provided with an update to how items in Deleted Items are treated all the way up to 2010 SP1 RU2. RU2 is the latest RU out as of this writing.

    When items are moved from folders with RPTs or a DPT on them to the Deleted Items folder their Start Date is not re-evaluated and the original Start Date the Managed Folder Assistant (MFA) stamped on the item will be retained. This means it is possible items moved to Deleted Items could be expired/deleted immediately the first time the MFA scans the folder. This does not affect items moved to DI from folders without RPTs or a DPT on them.

    Example when item is moved to DI from a folder with an RPT or DPT...

    1. Item comes into Inbox on 1/24/2010 and Inbox has a DPT of 1095 days. The item is stamped by the MFA with a Start Date of 1/24/2010.
    2. The item is deleted by a user on 3/15/2010 and moved to Deleted Items, which has a RPT of 30 days on it. The Start Date remains at 1/24/2010 because moves to DI don't re-evaluate/change the Start Date.
    3. MFA comes around and scans the mailbox, sees the item in Deleted Items with a Start Date of 1/24.
    4. Since 1/24 is >30 days it deletes the item.

    Example when an item is moved to DI from a folder w/o a RPT or DPT...

    1. Item comes into Inbox on 1/24/2010 and Inbox has no RPT/DPT. No Start Date is stamped, Start Date remains $Null.
    2. The item is deleted by a user on 3/15/2010 and moved to Deleted Items, which has a RPT of 30 days on it. The Start Date remains $Null because moves to DI don't re-evaluate/change the Start Date.
    3. MFA comes around and scans the mailbox, sees the item in Deleted Items with a Start Date of $Null so it uses "Present Date + RPT" instead.
    4. Item has an expiration date of 3/15 + 30 days stamped on it.
    5. Once MFA scan the mailbox on the 30th day after 3/15, it deletes the item from DI.

    Microsoft Premier Field Engineer, Exchange
    MCSA 2000/2003, CCNA
    MCITP: Enterprise Messaging Administrator 2010
    Former Microsoft MVP, Exchange Server
    My posts are provided “AS IS” with no guarantees, no warranties, and they confer no rights.
    Wednesday, January 26, 2011 2:50 PM

All replies

  • There are several steps involved:

    1. Creating the tag (New-RetentionPolicyTag)
    2. Creating the policy (New-RetentionPolicy)
    3. Applying the policy (e.g. Get-Mailbox -id user | Set-Mailbox -RetentionPolicy "MyPolicy")
    4. (Optional) Manually forcing the Managed Folder Assistant to apply the policy (Start-ManagedFolderAssistant)

    Have you performed all 4 steps?  If so, please can you post the commands you issued for each.

    Alexei

    Monday, November 8, 2010 7:10 PM
  • I did each step (exactly as you described) inside of the EMC console, the last step i did in the shell.
    Monday, November 8, 2010 7:35 PM
  • Please can you post the output of the following:

    Get-RetentionPolicyTag -id MyPolicyTagName | fl

    Alexei

    Monday, November 8, 2010 8:36 PM
  • Please can you post the output of the following:

    Get-RetentionPolicyTag -id MyPolicyTagName | fl

    Alexei


    RunspaceId           : bf061c08-23c9-4783-8df3-0e028864089e
    MessageClassDisplayName     : All Mailbox Content
    MessageClass          : *
    Description           : Managed Content Settings
    RetentionEnabled        : True
    RetentionAction         : DeleteAndAllowRecovery
    AgeLimitForRetention      : 1.00:00:00
    MoveToDestinationFolder     :
    TriggerForRetention       : WhenDelivered
    MessageFormatForJournaling   : UseTnef
    JournalingEnabled        : False
    AddressForJournaling      :
    LabelForJournaling       :
    Type              : DeletedItems
    SystemTag            : False
    LocalizedRetentionPolicyTagName : {}
    Comment             :
    RetentionId           : dcacc4bb-f972-4cd5-9fbf-df769b19dd43
    LocalizedComment        : {}
    MustDisplayCommentEnabled    : False
    LegacyManagedFolder       :
    AdminDisplayName        :
    ExchangeVersion         : 1.0 (0.0.0.0)
    Name              : Deleted Items Older than 1 Day
    DistinguishedName        : CN=Deleted Items Older than 5 Days,CN=Retention Policy Tag Container,CN=domain,CN=Mi
                     crosoft Exchange,CN=Services,CN=Configuration,DC=domain,DC=org
    Identity            : Deleted Items Older than 1 Day
    Guid              : dcacc4bb-f972-4cd5-9fbf-df769b19dd43
    ObjectCategory         : domain.org/Configuration/Schema/ms-Exch-ELC-Folder
    ObjectClass           : {top, msExchELCFolder}
    WhenChanged           : 11/8/2010 10:37:40 AM
    WhenCreated           : 11/8/2010 10:37:25 AM
    WhenChangedUTC         : 11/8/2010 3:37:40 PM
    WhenCreatedUTC         : 11/8/2010 3:37:25 PM
    OrganizationId         :
    OriginatingServer        : dc1.domain.org
    IsValid             : True
    Monday, November 8, 2010 8:48 PM
  • I can't see anything wrong with that.  Let's look at the retention policy itself:

    Get-RetentionPolicy -Identity MyRetentionPolicyName | fl RetentionPolicyTagLinks, IsValid

    Alexei

    Monday, November 8, 2010 9:03 PM
  •  A few users have exactly the same issue - we have already thread going with no apparent answer from MS:

     

    http://social.technet.microsoft.com/Forums/en/exchange2010/thread/56260407-4b15-407e-9079-2a97c79abf0c

     

    It apprears only affect Deleted Items folder... other folders, such as "Sent Items" with similar policy applied work fine.

    Monday, November 8, 2010 10:43 PM
  • Interesting!  I get the same result in my testing.  Looks like it might be a bug.

    Alexei

    Tuesday, November 9, 2010 6:10 AM
  •  A few users have exactly the same issue - we have already thread going with no apparent answer from MS:

     

    http://social.technet.microsoft.com/Forums/en/exchange2010/thread/56260407-4b15-407e-9079-2a97c79abf0c

     

    It apprears only affect Deleted Items folder... other folders, such as "Sent Items" with similar policy applied work fine.


    What number would I call to contact MS about the bug and see if they can confirm it without cost?
    Tuesday, November 9, 2010 12:09 PM
  • You can find the number through the portal here (depends on the country you are in):

    http://support.microsoft.com/select/Default.aspx?target=assistance

    If you don't have an Enterprise Agreement you will likely be asked for your credit card details to log the call, although I believe they refund you if it turns out to be a bug.

    Alexei

    Tuesday, November 9, 2010 8:08 PM
  • If you don't have an Enterprise Agreement you will likely be asked for your credit card details to log the call, although I believe they refund you if it turns out to be a bug.

     

    Correct. If you're a TechNet subscriber you also I think get 2 free incidents per subscription-year. :)


    Microsoft Premier Field Engineer, Exchange
    MCSA 2000/2003, CCNA
    MCITP: Enterprise Messaging Administrator 2010
    Former Microsoft MVP, Exchange Server
    My posts are provided “AS IS” with no guarantees, no warranties, and they confer no rights.
    Tuesday, November 9, 2010 9:11 PM
  • My customer has the same problem. Is the symptom a bug? Please update to us after contacting Microsoft.

    Wednesday, November 10, 2010 7:59 AM
  • We've got an EA so i've submitted the question via an email and waiting for an email back. Will let everyone know what the response is.
    Wednesday, November 10, 2010 1:29 PM
  • Removed for better clarification below.

    Wednesday, November 10, 2010 2:08 PM
  • That's useful info Brian.  Is is publicly available somewhere?

    Alexei

    Wednesday, November 10, 2010 6:52 PM
  • Well, I have items in my Deleted Items folder that have dates from 2008 and 2009. I didn't change any default retention time on Deleted Items, so I assume it's still 14 days? If we add 14 days from default retention time, and 21 days from the retention policy, it would be 35 days. I see all these emails that are over two years old still there, with "This item is expired" tag.
    Wednesday, November 10, 2010 7:39 PM
  • Any updates on this?
    Thursday, November 11, 2010 4:30 PM
  • When were the items moved to the deleted items folder?


    Microsoft Premier Field Engineer, Exchange
    MCSA 2000/2003, CCNA
    MCITP: Enterprise Messaging Administrator 2010
    Former Microsoft MVP, Exchange Server
    My posts are provided “AS IS” with no guarantees, no warranties, and they confer no rights.
    Thursday, November 11, 2010 4:39 PM
  • Where or when? If "when", it was before the policy was applied. Does it make a difference? It would only apply to items that were moved there after the policy was created, even though it marks all the items in the Deleted Items as it should with expiration etc?
    Thursday, November 11, 2010 8:56 PM
  • The help information for the New-RetentionPolicyTag cmdlet suggests that the date the item is moved to the Deleted Items folder is irrelevant:

    -AgeLimitForRetention <EnhancedTimeSpan>
            The AgeLimitForRetention parameter specifies the age at which retention is enforced on an item. The age limit corresponds to the number of days from the date the item was delivered, or the date an item was created if it wasn't delivered. If this parameter isn't present and the RetentionEnabled parameter is set to $true, an error is returned.

    Alexei

    Thursday, November 11, 2010 9:24 PM
  • *edited*

    Please see post on 1/26 for updated information.

    Thursday, November 11, 2010 9:43 PM
  • Hi Brian

    I don't disagree with the logic, but it would be nice to see it explictly stated in the help (or anywhere authoritative really).  As it stands it is misleading at best.

    Interesting.  So what you state implies that a soft deletion from a folder (Del instead of Shift+Del) is different to simply moving an item to the Deleted Items folder?

    Alexei

    Thursday, November 11, 2010 10:01 PM
  • *edited*

    Please see post on 1/26 for updated information.

    Thursday, November 11, 2010 10:17 PM
  • So I dont get it...is there a bug or no?  Same problem.
    Friday, November 12, 2010 7:51 PM
  • I haven't forgot about you guys.... I've been emailing back and forth with Microsoft regarding this and the technician has reported he experiences the same thing. He asked to get back to me in a few days as he wanted to do some more testing. I will update everyone once I hear back.
    Monday, November 15, 2010 12:17 PM
  •  

    I finished my testing. I found the Retention Policy could work. In my test, I deleted 4 mails, 3 of them are old mails which existed at inbox more than 3 days, one is new which send on myself and delete it at the same day. My RPT is same as yours, the retention day is 1 day only, I got the result 3 old mails were deleted one day after, but the new mail was delete two days after. Then I talked with our premium Engineer and got response, The retention age of mails in deleted items works as: Deleted date + retention age. i.e. If you delete an item on 10th November and retention age is 1, the item will be emptied from deleted items folder after 11th November.

     

    If you have any questions or concerns, feel free to let me know. I will be glad to help.

     

    Best Regards,

    James Sun

     

    Messaging Support Professional

    Microsoft Professional Web Support

    The above was their response. I have responded with the following as I still don't believe it works.

    "I have 8 items in my deleted items and for the past 3 days they all say “This item will expire in 0 days. To keep this item longer, apply a different Retention Policy.” and they still have not been deleted."

     

    Wednesday, November 17, 2010 1:17 PM
  • Maybe this will be escalated ASAP. Exactly the same issue here!
    Thursday, November 18, 2010 3:35 PM
  • I've informed the tech i'm still having the issue and he requested some more info and said he would do further testing on his side and let me know.
    Thursday, November 18, 2010 6:56 PM
  • Very odd.... I just checked my deleted items and the emails that were in there were deleted.... 

    Still doesn't make sense though... I had a retention tag of 1 day and I sent myself an email on 11/15/2010 at 10:08PM. I deleted the item at 10:15PM the same night. The email didn't get permanently deleted until 11/18/2010.

     

    Friday, November 19, 2010 12:09 PM
  • Very odd.... I just checked my deleted items and the emails that were in there were deleted.... 

    Still doesn't make sense though... I had a retention tag of 1 day and I sent myself an email on 11/15/2010 at 10:08PM. I deleted the item at 10:15PM the same night. The email didn't get permanently deleted until 11/18/2010.

     


    has anyone checked to see if the retention policy works on different types of items and not emails? Seems like the appointment items in the Deleted items folder get permanently deleted...
    Wednesday, November 24, 2010 1:49 PM
  • I'm still getting this issue.  Items that are expired are still in the Deleted Items folder.  This includes a mix of calendar items and emails.  The policy is applied, and the countdown appears correct, but nothing is deleted.
    Friday, November 26, 2010 10:50 AM
  • I'm still getting this issue.  Items that are expired are still in the Deleted Items folder.  This includes a mix of calendar items and emails.  The policy is applied, and the countdown appears correct, but nothing is deleted.

    out of curiousity, how long have you waited after your countdown?
    Tuesday, November 30, 2010 12:48 PM
  • I created a policy and set it to 30 days.  It has been 3 weeks now, and still the items remain 'expired'.  Not re-visited the policy, but will keep an eye on deleted items.
    Tuesday, December 7, 2010 4:56 PM
  • I created a policy and set it to 1 day and applied it to myself for testing purposes... if I deleted an item on 12-9, it deletes it on 12-11 i think...

    Thursday, December 9, 2010 2:14 PM
  • Ok.  So all my emails that had expired were deleted today.  Exactly 30 days after creating the policy.  All is good. :)
    Thursday, December 16, 2010 11:33 AM
  • So if I am reading this correctly.  I add a 90 day retention tag on the deleted items folder.  It marks all the items past 90 days as expired but it doesn't remove them until 90 days after I apply the policy.  If that is correct then I think there is a bug.
    Tuesday, January 25, 2011 10:32 PM
  • I've been provided with an update to how items in Deleted Items are treated all the way up to 2010 SP1 RU2. RU2 is the latest RU out as of this writing.

    When items are moved from folders with RPTs or a DPT on them to the Deleted Items folder their Start Date is not re-evaluated and the original Start Date the Managed Folder Assistant (MFA) stamped on the item will be retained. This means it is possible items moved to Deleted Items could be expired/deleted immediately the first time the MFA scans the folder. This does not affect items moved to DI from folders without RPTs or a DPT on them.

    Example when item is moved to DI from a folder with an RPT or DPT...

    1. Item comes into Inbox on 1/24/2010 and Inbox has a DPT of 1095 days. The item is stamped by the MFA with a Start Date of 1/24/2010.
    2. The item is deleted by a user on 3/15/2010 and moved to Deleted Items, which has a RPT of 30 days on it. The Start Date remains at 1/24/2010 because moves to DI don't re-evaluate/change the Start Date.
    3. MFA comes around and scans the mailbox, sees the item in Deleted Items with a Start Date of 1/24.
    4. Since 1/24 is >30 days it deletes the item.

    Example when an item is moved to DI from a folder w/o a RPT or DPT...

    1. Item comes into Inbox on 1/24/2010 and Inbox has no RPT/DPT. No Start Date is stamped, Start Date remains $Null.
    2. The item is deleted by a user on 3/15/2010 and moved to Deleted Items, which has a RPT of 30 days on it. The Start Date remains $Null because moves to DI don't re-evaluate/change the Start Date.
    3. MFA comes around and scans the mailbox, sees the item in Deleted Items with a Start Date of $Null so it uses "Present Date + RPT" instead.
    4. Item has an expiration date of 3/15 + 30 days stamped on it.
    5. Once MFA scan the mailbox on the 30th day after 3/15, it deletes the item from DI.

    Microsoft Premier Field Engineer, Exchange
    MCSA 2000/2003, CCNA
    MCITP: Enterprise Messaging Administrator 2010
    Former Microsoft MVP, Exchange Server
    My posts are provided “AS IS” with no guarantees, no warranties, and they confer no rights.
    Wednesday, January 26, 2011 2:50 PM
  • Thanks Brian.  I had to read it about 3 times but I got it now.  I also ran a test and your explanation was 100% correct.  Since I moved all my user over to exchange 2010 last month I will have to wait for old deleted items to get done.  The new stuff goes every night.

    Wednesday, January 26, 2011 10:29 PM
  • You're very welcome, and yeah it isn't the easiest to put into words. :)

    Also... when using Outlook 2010 there is an interesting UI thing that may happen. When the item is <24 hours from expiration you may already see "This item has expired" on the message, even though it doesn't truly expire until the next day. You can verify this by opening the same item in OWA and it should say "0 days left", which means the item is on its last day before expiration.


    Microsoft Premier Field Engineer, Exchange
    MCSA 2000/2003, CCNA
    MCITP: Enterprise Messaging Administrator 2010
    Former Microsoft MVP, Exchange Server
    My posts are provided “AS IS” with no guarantees, no warranties, and they confer no rights.
    Wednesday, January 26, 2011 10:38 PM
  • So what happens to the items that are in DI before the RPT of "Delete after 30 days" was put in place?  I have this situation where the test mailboxes have deleted items in them back to 2006 and they get marked as expired but do not actually delete from their account in the Outlook client or OWA.  Running Exchange 2010 SP1 RU2 at this time with client version SP1.  We have not started purging deleted items from all mailboxes yet but would need to know how to purge those items if they are already in their deleted items as well.

    Thanks

     
    Friday, May 11, 2012 5:39 PM
  • Outlook 2010, which is where we were looking at the items, was running in cached mode.

    We then looked at some items in OWA, and then looked at them with Outlook in online mode, and there we saw that the old items that were showing as expired with Outlook in cached mode, were showing that they would expire in 4 days. When we first applied the policy, and it was early the previous week - 10 days prior.

    So, what is occurring is that Outlook in cached mode pulls the Policy Tag information for the folder and then does the calculation for expiration itself. If we're in the initial 'x' number of days after the policy was originally applied to the items (they were just moved into the folder, etc.), we may have items that Outlook in cached mode calculates as expired, however, since they were just tagged within the expiration period, they are not expired or deleted on the server.

    Best way to troubleshoot the Retention Policy issue is from OWA  to check the actual days left

    http://blogs.technet.com/b/richardroddy/archive/2010/12/14/exchange-2010-sp1-outlook-2010-cached-mode-and-retention-policies-why-do-my-items-show-expired-in-outlook-but-are-not-being-removed-by-exchange.aspx 


    Regards,

    Anil Yadav





    Wednesday, May 16, 2012 10:37 PM
  • I've created a retention policy + tag (for testing purposes) to delete old email older than 1 day from the deleted items and allow recovery assigned it to a user and I've run "start-managedfolderassistant -identity user".

    There are emails in the deleted items that are 4 weeks old that it is not deleting... What could be wrong? This is Exchange 2010 SP1.


    I am on Exchange 2010 SP1 and Retention Policy works. But not as advertised. When you configure a folder to clear items after one day, it doesnt work. After spending hours with Microsoft, depending on when the managed folder assistant starts and ends, It varies. So we had to tell legal that items will be deleted in 1-3 days... we dont really know : (

    Z-Hire -- Automate IT Account creation process ( AD / Exchange / Lync )
    Z-Term -- Automate IT account termination process ( AD / Exchange )

    Thursday, May 17, 2012 3:52 AM
  • Hello Denny,

    In Exchange 2010 Retention Calculation for the Deleted Items folder is calculated in a diffrent way, specifically for deleted Items folder Retention calculation starts from the date Retention Policy is Applied to the User mailbox.

    For Example: If we have 1year Old Emails in the deleted items folder and if you have added the (RPT) Retention Policy Tag to Retention Policy to delete the Items older then 7 day from the deleted Items folder and if you are applying the Retention Policy today to the User "UserA' mailbox, It will start the Retention Calculation from date the Retention Policy is applied to 'UserA' mailbox only for deleted Items folder.

    All 1 year old Emails won't be deleted immediatly because it will wait for another 7 days to coross the retention period.

    If you want to verify this behaviour, Kindly check the actual retention Period of all the Email items from the OWA. In Outlook 2010 cached mode for 1 year old Email it will show item as expired but in real sense it will wait for another 7 days to deleted those expired items.

    Outlook in cached mode pulls the Policy Tag information for the folder and then does the calculation for expiration itself. If we're in the initial 'x' number of days after the policy was originally applied to the items (they were just moved into the folder, etc.), we may have items that Outlook in cached mode calculates as expired, however, since they were just tagged within the expiration period, they are not expired or deleted on the server.

    Best way to troubleshoot the Retention Policy issue is from OWA  to check the actual days left

    Regard,

    Anil


    Thank you, Anil Yadav


    • Edited by anya_exchange Thursday, May 17, 2012 12:59 PM
    • Proposed as answer by Arve Synd Thursday, May 2, 2013 6:06 AM
    Thursday, May 17, 2012 10:41 AM
  • Wow, so if I'm understanding this correctly I've just setup a Deleted items policy to permanently delete items in the DI folder older than 6 months.  I now have to wait 6 months from the creation and application of this policy before it will do anything?!!

    I confirmed this by looking at my deleted items folder in OWA.  All my items older than 6 months from today's date are set to expire (via the policy) in August this year.  Roughly 6 months from when I created the policy.  Yet items I deleted a few days ago will by cleaned out a few days in advance of this in August.

    I just find this kind of backward.  Usually people create these archive policies as they are running out of space either on their server or with mailbox limits for their users.  Having to wait the period of the policy before it does anything defeats the purpose in my opinion...

    Wednesday, March 6, 2013 3:08 AM
  • Bump.  Is this "bug" still the behavior in 2010 sp3?  How about 2013cu1?
    Saturday, May 4, 2013 3:48 AM
  • What an poor senseless logic from MS Exchange Team! They are definitely out of touch. Something as simple as "Emptying the Deleted Folder after X number of days" should work right out of the box, without any fuss. The quality of work by the developers are sub par - to put it simply. The term frequently used for this type of programming is "Code Junkies"  and they are exactly that!

    Friday, December 13, 2013 7:40 PM