locked
Index not receiving new documents durning feed. RRS feed

  • Question

  • Hello all,

        I'm having some serious feeding issues with my linux based ESP5.3SP4 3 node cluster (admin, index/search, backup index/search). My problem is that when I do a feed everything looks fine, but wen I go to search on it, nothing gets added. The doc counts do not change, but I do get a properly generated doc_id and no errors/warnings.

        A related problem is that if I do a docpush to add a doc, I get a success, but the doc is not really there. Further, I can delete the doc more than once and still get success messages as shown below.

       Has anyone out there seen this before and know how to fix it?

    Thank you!

     [2013-01-08 10:19:03] DEBUG  indexer   <X>  15900   systemmsg   ft::session_holder: Closing session 3  
     [2013-01-08 10:18:58] INFO   docpush   <X>   Test123 All delete operations completed  
     [2013-01-08 10:18:56] DEBUG  indexer   <X>   15900   systemmsg    session_factory_servant: Returning a VALID session 3 for collection Test123  
     [2013-01-08 10:18:56] DEBUG  indexer   <X>   15900   systemmsg    ft::session_holder: Adding session 3 for collection 'Test123', currently 0 registered sessions  
     [2013-01-08 10:18:37] DEBUG  indexer   <X>   15900   systemmsg    Reporter: Fixml fill rate: 0.975597  
     [2013-01-08 10:18:37] DEBUG  indexer   <X>   15900   systemmsg    Reporter: Sampling: steps_per_sample = 9, num_samples = 25027, starting id = 163  
     [2013-01-08 10:18:37] DEBUG  indexer   <X>   15900   systemmsg    Reporter: Calculating fixml fill rate  

     [2013-01-08 10:18:03] DEBUG  indexer   <X>   15900   systemmsg    ft::session_holder: Closing session 2
     [2013-01-08 10:17:58] INFO   docpush   <X>     Test123 All delete operations completed  
     [2013-01-08 10:17:56] DEBUG  indexer  <X>   15900   systemmsg    session_factory_servant: Returning a VALID session 2 for collection Test123  
     [2013-01-08 10:17:56] DEBUG  indexer   <X>   15900   systemmsg    ft::session_holder: Adding session 2 for collection 'Test123', currently 0 registered sessions  
     [2013-01-08 10:02:18] DEBUG  indexer   <X>   15900   systemmsg    ft::session_holder: Closing session 1  
     [2013-01-08 10:02:17] DEBUG  indexer   <X>   15900   systemmsg    Reporter: Fixml fill rate: 0.975597  
     [2013-01-08 10:02:16] DEBUG  indexer   <X>   15900   systemmsg    Reporter: Sampling: steps_per_sample = 9, num_samples = 25027, starting id = 163  
     [2013-01-08 10:02:16] DEBUG  indexer   <X>   15900   systemmsg    Reporter: Calculating fixml fill rate  
     [2013-01-08 10:02:13] INFO   docpush   <X>           Test123 All add operations completed  
     [2013-01-08 10:02:11] DEBUG  indexer   <X>   15900   systemmsg    session_factory_servant: Returning a VALID session 1 for collection Test123  
     [2013-01-08 10:02:11] DEBUG  indexer   <X>   15900   systemmsg    ft::session_holder: Adding session 1 for collection 'Test123', currently 0 registered sessions  

    Tuesday, January 8, 2013 4:02 PM

Answers

  • This behavior sounds related to some corruption of ftstorage directory , if you are using fault tolerant indexing.
    Would suggest, stop indexers, delete ftStorage on each index server ( you can back them up), start indexers.

    Wednesday, January 9, 2013 3:23 PM

All replies

  • This behavior sounds related to some corruption of ftstorage directory , if you are using fault tolerant indexing.
    Would suggest, stop indexers, delete ftStorage on each index server ( you can back them up), start indexers.

    Wednesday, January 9, 2013 3:23 PM
  • Hello all,

       We figured it out.

       Turns out that we had a corrupted $FASTSEARCH/data/ftStorage directory.

       So, after shutting down the indexer and moving this dir out of the way, everything started working as it should.

    Wednesday, January 9, 2013 3:24 PM