Can old (5 months - 5 years) .normalized directories be deleted from data_index? RRS feed

  • Question

  • Hi All,

    This is my first post, I am trying to defrag my index but space is becoming an issue. I need to know what the impact is if any of deleting .normalized folders from the data_index folder. Some of them are really old - up to 5 years without being modified. Is there a process for this or is it just a case of deleting /moving them, or will I have to stop FAST whilst I perform the clear up. If possible I would like to know what I can clean out of these directories or what house keeping I should bee performing in order to avoid this in the future.

    I will be at some point upgrading the drive sizes but for now I need to clean up the arrays that the nodes have mounted.

    Thank you in advance for any help.



    Friday, December 8, 2017 12:20 PM


  • Hi again,

    So I found some documentation regarding this, the data_index can be regenerated using resetindex --column xx --row xx. This seems to make a copy from it's counter part binary index on the adjacent row.

    The Index when regenerated is exactly the same size to the byte, the normalized directories are not duplicated. We have tested by searching on the nodes before and after and the results are the same.

    I would therefore be leaning towards the fact that these folder can be deleted without issue. 

    One thing we noticed was that the regeneration using resetindex would not actually start until the search process was started on the node that needed regeneration even though it was the indexing data that was being copied.

    It should be noted that we are using nodes in pairs so we have a search and index process running mirrored between two nodes, one favoring indexing and the other searches. It took about an hour to regenerated the index consisting of about 40 million records. which is about 250 - 300 GB.

    • Marked as answer by honkeypig Tuesday, December 12, 2017 11:43 AM
    Tuesday, December 12, 2017 11:43 AM