none
Reporting Database Refresh stuck in "Waiting to be processed (Sleeping)" RRS feed

  • Question

  • Hi all,

    Project Server 2013 - In the queued jobs log, we're having a lot of "Reporting" jobs success but only one job is failing - mainly "Reporting (Project Publish)"  and "Reporting (Project Sync)".

    After some research the suggestion is to force a rebuild of the reporting database by backing up/restoring custom fields, but when I try to do this, the reporting database refresh job sits in the queue with the status of "Waiting to be processed (Sleeping)" and never actually does anything.

    When i publish my Project the custom field values are not coming in my SQL Database but in my Project center it is showing.

    In our environment we have about 20 custom fields. I tried rebuilding the 

    reporting db by performing a backup/restore of these custom fields. The 
    backup jobs went ok, three restore jobs were also ok, but the "Reporting 
    Database Refresh" job is in status "Waiting to be Processed (Sleeping)" and stuck. This job is in queue position 1.

    How can I get around this issue ?? Urgent help....

    Thanks

    Varun

    Tuesday, November 24, 2015 2:19 PM

All replies

  • Hi Varun,

    The place to start is not the sleeping jobs but the failing one.  The sharepoint log files should have further information.  Sleeping isn't such an issue, but sleeping forever is...


    Ben Howard [MVP] | web | blog | book | P2O

    Tuesday, November 24, 2015 2:31 PM
    Moderator
  • Hi Varun, 

    i ve seen the same problem at client.

    after diagnostic. do this

    1- cancel all jobs in your project server queue (filter by Waiting To Be Processed and Waiting To Be Processed(Sleeping))

    2- rebuild your RDB with your account system (not with simple Administrator project server group account)

    Hope this help you.


    Jalal Blali MCTS, MCITP

    Tuesday, November 24, 2015 6:11 PM
  • In my scenario custom field meta data is not sync and stuck and jobs continuously failing...... Even i can publish my project but reporting database is a place where it stcuks...... see below the following queue job error:

    • porting message processor failed:
          • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='950e3386-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
          • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='30c341a3-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
          • ReportingProjectChangeMessageFailed (24006) - Transaction (Process ID 103) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='7be02ad7-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='Transaction (Process ID 103) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.'.
          • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='c0f452fc-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
          • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='a50e30c6-2a93-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
          • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='81dda804-2b93-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
    • Queue:
      • GeneralQueueJobFailed (26000) -ReportingProjectPublish.ReportProjectPublishMessageEx. Details: id='26000' name='GeneralQueueJobFailed' uid='82dda804-2b93-e511-9409-00155d008500' JobUID='aa754245-1e93-e511-9409-00155d008500' ComputerName='ee29ae1e-761e-4806-b11c-ecaf8851f7c1' GroupType='ReportingProjectPublish' MessageType='ReportProjectPublishMessageEx' MessageId='1' Stage='' CorrelationUID='c8bf449d-8ec2-006c-1f7c-1f9a2d5a3565'. For more details, check the ULS logs on machine ee29ae1e-761e-4806-b11c-ecaf8851f7c1 for entries with JobUID aa754245-1e93-e511-9409-00155d008500.

    another queue job error  is:

    General
    • Reporting message processor failed:
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='3e4b55f3-0a93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='0fa2d859-0c93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='78c35bc0-0d93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='f128b026-0f93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='81a2118d-1093-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='03f964f3-1193-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
    • Queue:
      • GeneralQueueJobFailed (26000) -ReportingCustomFieldMetadataSync.ReportCustomFieldMetadataSyncMessageEx. Details: id='26000' name='GeneralQueueJobFailed' uid='04f964f3-1193-e511-9409-00155d008500' JobUID='f2c8179e-ce92-e511-9409-00155d008500' ComputerName='ee29ae1e-761e-4806-b11c-ecaf8851f7c1' GroupType='ReportingCustomFieldMetadataSync' MessageType='ReportCustomFieldMetadataSyncMessageEx' MessageId='1' Stage='' CorrelationUID='c8bf449d-8ec2-006c-1f7c-1f9a2d5a3565'. For more details, check the ULS logs on machine ee29ae1e-761e-4806-b11c-ecaf8851f7c1 for entries with JobUID f2c8179e-ce92-e511-9409-00155d008500.

    Wednesday, November 25, 2015 6:45 AM
  • In my scenario custom field meta data is not sync and stuck and jobs continuously failing...... Even i can publish my project but reporting database is a place where it stcuks...... see below the following queue job error:

    • porting message processor failed:
      • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='950e3386-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
      • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='30c341a3-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
      • ReportingProjectChangeMessageFailed (24006) - Transaction (Process ID 103) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='7be02ad7-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='Transaction (Process ID 103) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.'.
      • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='c0f452fc-2993-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
      • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='a50e30c6-2a93-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
      • ReportingProjectChangeMessageFailed (24006) - One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. Details: id='24006' name='ReportingProjectChangeMessageFailed' uid='81dda804-2b93-e511-9409-00155d008500' QueueMessageBody='Project UID='7d1a8d6b-6c8f-e511-827a-4cbb587da0f8'. PublishType='All'' Error='One or more custom field metadata mismatch were detected while transfering project '7d1a8d6b-6c8f-e511-827a-4cbb587da0f8''.
    • Queue:
      • GeneralQueueJobFailed (26000) -ReportingProjectPublish.ReportProjectPublishMessageEx. Details: id='26000' name='GeneralQueueJobFailed' uid='82dda804-2b93-e511-9409-00155d008500' JobUID='aa754245-1e93-e511-9409-00155d008500' ComputerName='ee29ae1e-761e-4806-b11c-ecaf8851f7c1' GroupType='ReportingProjectPublish' MessageType='ReportProjectPublishMessageEx' MessageId='1' Stage='' CorrelationUID='c8bf449d-8ec2-006c-1f7c-1f9a2d5a3565'. For more details, check the ULS logs on machine ee29ae1e-761e-4806-b11c-ecaf8851f7c1 for entries with JobUID aa754245-1e93-e511-9409-00155d008500.

    another queue job error  is:

    General
    • Reporting message processor failed:
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='3e4b55f3-0a93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='0fa2d859-0c93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='78c35bc0-0d93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='f128b026-0f93-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='81a2118d-1093-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
      • ReportingCustomFieldMetadataChangeMessageFailed (24002) - Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1. Details: id='24002' name='ReportingCustomFieldMetadataChangeMessageFailed' uid='03f964f3-1193-e511-9409-00155d008500' QueueMessageBody='CustomFieldTypeUID='0000783f-de84-434b-9564-284e5b7b3f49'. ChangeType='Add'' Error='Unable to acquire an application lock 'MSP Project Metadata' in mode 'Exclusive'. LockResult = -1'.
    • Queue:
      • GeneralQueueJobFailed (26000) -ReportingCustomFieldMetadataSync.ReportCustomFieldMetadataSyncMessageEx. Details: id='26000' name='GeneralQueueJobFailed' uid='04f964f3-1193-e511-9409-00155d008500' JobUID='f2c8179e-ce92-e511-9409-00155d008500' ComputerName='ee29ae1e-761e-4806-b11c-ecaf8851f7c1' GroupType='ReportingCustomFieldMetadataSync' MessageType='ReportCustomFieldMetadataSyncMessageEx' MessageId='1' Stage='' CorrelationUID='c8bf449d-8ec2-006c-1f7c-1f9a2d5a3565'. For more details, check the ULS logs on machine ee29ae1e-761e-4806-b11c-ecaf8851f7c1 for entries with JobUID f2c8179e-ce92-e511-9409-00155d008500.
    Wednesday, November 25, 2015 6:46 AM