none
WebAnalyzer linkdump & SPRel build_proc folder sizes RRS feed

  • Question

  • We run FS4SP on a SharePoint Foundation 2010 farm (2 x WFE, 4 x APP, 1 x Master + 1 x Backup FAST, Clustered MS SQL).

    We have recently identified that the FAST certificates expired in Nov 2017 (self-signed). We have renewed these with new 5 year self-signed certificates and FAST is now working "better".

    Since the certificate renewal we have noticed that the available hard disk space has significantly reduced and have identified that a huge amount is taken up by the following folders:

    1. <FASTSearch>\data\sprel\worker\build_proc
    2. <FASTSearch>\data\webanalyzer\linkdump\in_use

    We have provided the following output from spreladmin showstatus and waadmin showstatus:

    PS C:\FASTSearch\bin> spreladmin showstatus
    --------------------------------------------------------------------------------
    ShowStatus:
    
    System:
        Overall status   :  Running
        Error status     :  NOERROR
        Split factor     :  4
        End factor       :  2
    
    Processing:
        Status           :  running
        Makefile         :  spcidcollapser
        Progress         :  79.2 %
        Generation       :  1778
    
    Runtime statistics:
        Number of runs   :  1778
        Total runtime    :  4376098 sec
        Average          :  2461 sec
        Max runtime      :  190341 sec
        Min runtime      :  54 sec
        Last 5 runtimes
                            452 sec
                            284 sec
                            458 sec
                            4547 sec
                            2708 sec
    
    Schedule:
        Scheduled start  :  2018-01-17,04:00
        Interval         :  86400 seconds
        Analysis will start as soon as new click log data enters the system
    
    Collections:
        sp:
            cleared                : 2014-11-21T22:08:41
            created                : 2012-06-14T16:10:45
            generation             : 965
            last_time_finished     : 2018-01-24T04:33:47
            last_time_started      : 2018-01-24T03:25:27
            status                 : 1
            timestamp              : 2018-01-26T03:22:51
    
    Stored clicklog dates:
        First            :  2017-12-17
        Last             :  2018-01-15
    
    Workers:
        live-fast.staff.<our_domain>:$FASTSEARCH\data\sprel\worker
        live-fast2.staff.<our_domain>:$FASTSEARCH\data\sprel\worker
    
    LookupDBs:
        live-fast.staff.<our_domain>, #0
        live-fast2.staff.<our_domain>, #1
    PS C:\FASTSearch\bin> waadmin showstatus
    --------------------------------------------------------------------------------
    ShowStatus:
    
    System:
        Overall status    :  Running
        Error status      :  NoError
        Split factor      :  16
        End factor        :  2
    
    Views:
        default:
            Schedule status   :  stopped
            Queue number      :  -
            Makefile          :  walookupdeployer
            Makefile stage    :  processing
            Makefile progress :  0 %
    
    Views beeing created:
        -
    
    Views beeing deleted:
        -
    
    Collections:
        sp:
            last_time_finished      :  -1
            created                 :  2014-11-21T22:08:43
            generation              :  0
            last_preferred_view     :
            last_time_started       :  -1
            number_of_links         :  0
            started                 :  False
            timestamp               :  2018-01-26T07:26:06
            cleared                 :  2014-11-21T22:08:43
    
    
    Collections beeing created:
        -
    
    Collections beeing deleted:
        -
    
    WAWorkers:
            live-fast.staff.<our_domain>:$FASTSEARCH/data/webanalyzer/waworker1
            live-fast2.staff.<our_domain>:$FASTSEARCH/data/webanalyzer/waworker1
    
    WALookupDBs:
            live-fast.staff.<our_domain>, #0
            live-fast2.staff.<our-domain>, #1
    We have already given both the FAST servers 250GB additional storage space - this got eaten up overnight BUT the actual search index was slightly updated (we have an application that consumes the FAST index and it is now showing some correct updates - prior to the certificates being renewed it didn't display any updates. So we think we're heading in the right direction).

    With the storage space filling up we do understand that the indexer starts, tries to index, fills up, aborts, repeats.

    We're looking for some advice, should we throw yet more storage space at the FAST servers? Will they 'eventually' process the backdated files (why hasn't it done this before as we have a number of files dating back to 2015!?)?

    I'm very new to FS4SP and our only previous SharePoint administrator no longer works for us!

    Thank you :)
    Friday, January 26, 2018 11:59 AM