none
Journal Wrap error has occurred..... RRS feed

  • Question

  • Hello,

    I have DPM 2012 R2 protecting 9 DB's on a SQL server [SQL 2008].  for the last 2 months I have been getting a "Journal wrap error has occurred on the change journal and therefore DPM is unable to track any more changes or may have missed some changed for the data source" on every protected Database when DPM is scheduled to do an express full backup.  after the failure, a consistency check is run automatically which succeeds and then all incremental backups succeed. [or at least I don't see any errors].  I am also protecting data on this SQL server in another protection group but I don't get any errors on these backups.

    After doing some research, I have increased the amount of backups [shortened time between backups], I have increased the USN Journal manually on the SQL server from 300 MB to 1 GB [as you don't get the protected computer tab when you right click and "modify disk allocation" on a protected database].  I have also increased the USN journal on the protected computer tab on the data protection.

    Does anyone have any guidance on the journal wrap errors on a protected SQL server?

    Thanks

    Rob

    Thursday, June 25, 2015 3:13 PM

Answers

  • Hi,

    DPM uses the NTFS USN change journal to help locate files that have changed since the last synchronization or recovery point.  The USN journal is a cyclic file, meaning that older records are overwritten as needed based on the USN journal file size. It seems a process is changing something about files that USN tracks that is causing the USN to wrap, and once that occurs, you will get the error in DPM.

    Everything you ever wanted to know about the NTFS USN journal and were afraid to ask can be read about here and here.

    You have an offending application (most likely anti-virus) that is either touching lots of files, or is updating a single file or set of files constantly which is causing the USN journal to wrap between DPM Synchronization jobs. 

     To see what files may be causing the USN journal to wrap, perform the following steps,

    1) Download the DPM 2010 diagnostic and install it on the protected sever.

    http://www.microsoft.com/en-us/download/details.aspx?id=9462

    2) After it's installed from an administrative command prompt cd to: C:\Windows\MPSReports\DPM\bin

    3) Run dumpusn.exe X: -e -o C:\temp\usnlog.txt (Where X: is the protected volume drive letter)

    4) Press CTRL+C to stop it otherwise the file will be too large to open in notepad.

    5) Open the file and see if there is a common set of files that are constantly being updated.  If you find some, you can use resource monitor to see what application or process that is responsible.  Update or un-install that application.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, June 25, 2015 5:59 PM
    Moderator

All replies

  • Hi,

    DPM uses the NTFS USN change journal to help locate files that have changed since the last synchronization or recovery point.  The USN journal is a cyclic file, meaning that older records are overwritten as needed based on the USN journal file size. It seems a process is changing something about files that USN tracks that is causing the USN to wrap, and once that occurs, you will get the error in DPM.

    Everything you ever wanted to know about the NTFS USN journal and were afraid to ask can be read about here and here.

    You have an offending application (most likely anti-virus) that is either touching lots of files, or is updating a single file or set of files constantly which is causing the USN journal to wrap between DPM Synchronization jobs. 

     To see what files may be causing the USN journal to wrap, perform the following steps,

    1) Download the DPM 2010 diagnostic and install it on the protected sever.

    http://www.microsoft.com/en-us/download/details.aspx?id=9462

    2) After it's installed from an administrative command prompt cd to: C:\Windows\MPSReports\DPM\bin

    3) Run dumpusn.exe X: -e -o C:\temp\usnlog.txt (Where X: is the protected volume drive letter)

    4) Press CTRL+C to stop it otherwise the file will be too large to open in notepad.

    5) Open the file and see if there is a common set of files that are constantly being updated.  If you find some, you can use resource monitor to see what application or process that is responsible.  Update or un-install that application.


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread. Regards, Mike J. [MSFT] This posting is provided "AS IS" with no warranties, and confers no rights.

    Thursday, June 25, 2015 5:59 PM
    Moderator
  • Hi Mike,

    thanks for the guidance.  all i get int he usn dump is a bunch of this.....

    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CEED8,Frn:0xFE4B000000004E8D,PFrn:0x0094000000004599,File Name:6E^77d1dad1-93e9-4208-865b-f7bb4a3cac9f.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF000,Frn:0xFE4C000000004E8D,PFrn:0x0094000000004599,File Name:6E^77d1dad1-93e9-4208-865b-f7bb4a3cac9f.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF098,Frn:0xFE4C000000004E8D,PFrn:0x0094000000004599,File Name:6E^77d1dad1-93e9-4208-865b-f7bb4a3cac9f.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF130,Frn:0xFE4C000000004E8D,PFrn:0x0094000000004599,File Name:6E^77d1dad1-93e9-4208-865b-f7bb4a3cac9f.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF1C8,Frn:0xDA01000000004EDA,PFrn:0x0094000000004599,File Name:6E^28ac0311-2103-4345-891b-0edf4b3a2629.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF260,Frn:0xDA02000000004EDA,PFrn:0x0094000000004599,File Name:6E^28ac0311-2103-4345-891b-0edf4b3a2629.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF2F8,Frn:0xDA02000000004EDA,PFrn:0x0094000000004599,File Name:6E^28ac0311-2103-4345-891b-0edf4b3a2629.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E242E5B,Usn:0x0000000DA71CF390,Frn:0xDA02000000004EDA,PFrn:0x0094000000004599,File Name:6E^28ac0311-2103-4345-891b-0edf4b3a2629.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF428,Frn:0x81E2000000004F09,PFrn:0x0094000000004599,File Name:6E^ecee8242-511a-4d21-b167-dfec15e9212b.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF4C0,Frn:0x81E3000000004F09,PFrn:0x0094000000004599,File Name:6E^ecee8242-511a-4d21-b167-dfec15e9212b.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF558,Frn:0x81E3000000004F09,PFrn:0x0094000000004599,File Name:6E^ecee8242-511a-4d21-b167-dfec15e9212b.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF5F0,Frn:0x81E3000000004F09,PFrn:0x0094000000004599,File Name:6E^ecee8242-511a-4d21-b167-dfec15e9212b.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF688,Frn:0xEFE0000000004F79,PFrn:0x0094000000004599,File Name:6E^a83b9e97-ab25-4477-b3a2-31a1279d2478.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF720,Frn:0xEFE1000000004F79,PFrn:0x0094000000004599,File Name:6E^a83b9e97-ab25-4477-b3a2-31a1279d2478.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF7B8,Frn:0xEFE1000000004F79,PFrn:0x0094000000004599,File Name:6E^a83b9e97-ab25-4477-b3a2-31a1279d2478.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF850,Frn:0xEFE1000000004F79,PFrn:0x0094000000004599,File Name:6E^a83b9e97-ab25-4477-b3a2-31a1279d2478.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF8E8,Frn:0x2056000000004480,PFrn:0x0094000000004599,File Name:6E^c0191dbf-10c1-46aa-bf32-789bc9b2c1fd.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CF980,Frn:0x2057000000004480,PFrn:0x0094000000004599,File Name:6E^c0191dbf-10c1-46aa-bf32-789bc9b2c1fd.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CFA18,Frn:0x2057000000004480,PFrn:0x0094000000004599,File Name:6E^c0191dbf-10c1-46aa-bf32-789bc9b2c1fd.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CFAB0,Frn:0x2057000000004480,PFrn:0x0094000000004599,File Name:6E^c0191dbf-10c1-46aa-bf32-789bc9b2c1fd.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E28F309,Usn:0x0000000DA71CFB48,Frn:0x7D26000000004F9E,PFrn:0x0094000000004599,File Name:6E^8630b73b-4369-485d-aeab-241333949c53.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71CFBE0,Frn:0x7D27000000004F9E,PFrn:0x0094000000004599,File Name:6E^8630b73b-4369-485d-aeab-241333949c53.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71CFC78,Frn:0x7D27000000004F9E,PFrn:0x0094000000004599,File Name:6E^8630b73b-4369-485d-aeab-241333949c53.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71CFD10,Frn:0x7D27000000004F9E,PFrn:0x0094000000004599,File Name:6E^8630b73b-4369-485d-aeab-241333949c53.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71CFDA8,Frn:0xD3E9000000004690,PFrn:0x0094000000004599,File Name:6E^e38266f1-134d-4d60-9eaf-f804a17cc45f.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71CFE40,Frn:0xD3EA000000004690,PFrn:0x0094000000004599,File Name:6E^e38266f1-134d-4d60-9eaf-f804a17cc45f.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71CFED8,Frn:0xD3EA000000004690,PFrn:0x0094000000004599,File Name:6E^e38266f1-134d-4d60-9eaf-f804a17cc45f.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71D0000,Frn:0xD3EA000000004690,PFrn:0x0094000000004599,File Name:6E^e38266f1-134d-4d60-9eaf-f804a17cc45f.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71D0098,Frn:0xE23C000000004DB9,PFrn:0x0094000000004599,File Name:6E^f0b6a346-47fe-415f-9fbe-be3856fd91e4.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71D0130,Frn:0xE23D000000004DB9,PFrn:0x0094000000004599,File Name:6E^f0b6a346-47fe-415f-9fbe-be3856fd91e4.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71D01C8,Frn:0xE23D000000004DB9,PFrn:0x0094000000004599,File Name:6E^f0b6a346-47fe-415f-9fbe-be3856fd91e4.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71D0260,Frn:0xE23D000000004DB9,PFrn:0x0094000000004599,File Name:6E^f0b6a346-47fe-415f-9fbe-be3856fd91e4.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E2B5560,Usn:0x0000000DA71D02F8,Frn:0x12CA000000004450,PFrn:0x0094000000004599,File Name:6E^8693a8f5-df18-41e0-8a56-0f91bd4fe556.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D0390,Frn:0x12CB000000004450,PFrn:0x0094000000004599,File Name:6E^8693a8f5-df18-41e0-8a56-0f91bd4fe556.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D0428,Frn:0x12CB000000004450,PFrn:0x0094000000004599,File Name:6E^8693a8f5-df18-41e0-8a56-0f91bd4fe556.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D04C0,Frn:0x12CB000000004450,PFrn:0x0094000000004599,File Name:6E^8693a8f5-df18-41e0-8a56-0f91bd4fe556.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D0558,Frn:0x0F3E000000004443,PFrn:0x0094000000004599,File Name:6E^843ed570-0fb8-413a-aa5c-6234ea71cd91.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D05F0,Frn:0x0F3F000000004443,PFrn:0x0094000000004599,File Name:6E^843ed570-0fb8-413a-aa5c-6234ea71cd91.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D0688,Frn:0x0F3F000000004443,PFrn:0x0094000000004599,File Name:6E^843ed570-0fb8-413a-aa5c-6234ea71cd91.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D0720,Frn:0x0F3F000000004443,PFrn:0x0094000000004599,File Name:6E^843ed570-0fb8-413a-aa5c-6234ea71cd91.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D07B8,Frn:0x24B3000000004DC2,PFrn:0x0094000000004599,File Name:6E^f8a7a436-0169-4dc5-917e-2d8a69290081.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D0850,Frn:0x24B4000000004DC2,PFrn:0x0094000000004599,File Name:6E^f8a7a436-0169-4dc5-917e-2d8a69290081.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E2DB7B7,Usn:0x0000000DA71D08E8,Frn:0x24B4000000004DC2,PFrn:0x0094000000004599,File Name:6E^f8a7a436-0169-4dc5-917e-2d8a69290081.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0980,Frn:0x24B4000000004DC2,PFrn:0x0094000000004599,File Name:6E^f8a7a436-0169-4dc5-917e-2d8a69290081.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0A18,Frn:0x14D5000000004470,PFrn:0x0094000000004599,File Name:6E^7ca8dd65-2412-4fb0-96bc-9670c9ac450a.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0AB0,Frn:0x14D6000000004470,PFrn:0x0094000000004599,File Name:6E^7ca8dd65-2412-4fb0-96bc-9670c9ac450a.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0B48,Frn:0x14D6000000004470,PFrn:0x0094000000004599,File Name:6E^7ca8dd65-2412-4fb0-96bc-9670c9ac450a.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0BE0,Frn:0x14D6000000004470,PFrn:0x0094000000004599,File Name:6E^7ca8dd65-2412-4fb0-96bc-9670c9ac450a.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0C78,Frn:0x3527000000004E8F,PFrn:0x0094000000004599,File Name:6E^22f80439-024c-4517-a4e9-e08049a49e48.xml,Attributes:0x00000020,Reason:FileDelete,Close
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0D10,Frn:0x3528000000004E8F,PFrn:0x0094000000004599,File Name:6E^22f80439-024c-4517-a4e9-e08049a49e48.xml,Attributes:0x00000020,Reason:FileCreate
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0DA8,Frn:0x3528000000004E8F,PFrn:0x0094000000004599,File Name:6E^22f80439-024c-4517-a4e9-e08049a49e48.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate
    TimeStamp:0x01D0AF1D5E301A0E,Usn:0x0000000DA71D0E40,Frn:0x3528000000004E8F,PFrn:0x0094000000004599,File Name:6E^22f80439-024c-4517-a4e9-e08049a49e48.xml,Attributes:0x00000020,Reason:DataExtend,FileCreate,Close

    That is just a small portion of the file.  how am I to tell what files this dump correlates with?  

    Rob

    Thursday, June 25, 2015 6:16 PM
  • Got it.  I thought it was jibberish, but it is a bunch of .xml files used with the program Xactimate.

    I will contact Xactware to see if they know of any issues.  It now makes sense, I upgraded Xactimate about the same time this started happening.

    Thanks

    Rob

    Thursday, June 25, 2015 6:22 PM