none
How to delete the crawled property or items by jdbc RRS feed

  • Question

  • hi,

        I want to clear up the results which user don't want to see in the result page.But i have no idea how to do it.

        

    "sumsang" is the wrong property now by jdbc ,i want to move it away.

    May be the item "sumsang" above can be removed too.

     

    now in database i have no "samsung" value,but in result i also have "sumsang" crawled before.

     

    i recrawled by jdbc on search server but i also can't remove "sumsang".

     

    please give me your hand.


    In God We Trust
    Wednesday, February 1, 2012 8:25 AM

Answers

  • Hm... May be I do not understand the goal.

    If it is onetime situation, then clear of content collection is the way to go. You reset errorneous value and recrawl database - since database has only correct values - there should not be any problem.

    If it can happen from time to time - then correction of record in database should result in FAST index update on next run of jdbcconnector, and again there should not be any problem.

    If you delete errorneous record from DB instead of update, then it should result in FAST index update if ChangeDetection is enabled, and again there should not be any problem.

    Or I got it wrong?

    • Marked as answer by Fiyoung Wednesday, February 29, 2012 6:05 AM
    Tuesday, February 21, 2012 12:09 PM

All replies

  • Typically, you would want to enable changedetection/item removal within JDBCConnector by setting "Enabled" parameter within "ChangeDetection" group to True.  By default, it is false.  However, I'm not sure this will help you "after the fact", as it really needs to be enabled on the initial crawl to build the needed table with all the checksums.  Records no longer in the db will be removed on subsequent crawls.    You should be able to use it in the future

    http://technet.microsoft.com/en-us/library/ff354942.aspx#group_ChangeDetection

     

    Here is the snippet from the OOTB config file:

      <group name="ChangeDetection" expand="no">
                    <description>
                    <![CDATA[The parameters in the "ChangeDetection" group specify how to detect changes  in the database since last feeding from the database(s), using row checksums  ]]>
                    </description>
                    <parameter name="Enabled" type="boolean">
                                    <description>
                                                    <![CDATA[If true, changedetection will be enabled. <br>Default: false ]]>
                                    </description>
                                    <value>false</value>  
                    </parameter>
                    <parameter name="ChangeDBPurge" type="boolean">
                                    <description>
                                                    <![CDATA[If true, will delete all entries in the checksum database on startup. <br>Default: false ]]>
                                    </description>
                                    <value>false</value>
                    </parameter>

     

     

    For your current situation, you may be able to use something like "docpush" utility, but I think you you'd need to have a path to a file or its URL as one of the inputs.  I can't tell from your screenshot whether this is something that would be possible.  You may want to edit your Core Web part based on the below blog and be able to show all the fields associated with this document and see whether we could use DocPush to delete it.

     

    http://blogs.msdn.com/b/thomsven/archive/2011/01/26/seeing-what-actual-gets-indexed.aspx

     

    http://technet.microsoft.com/en-us/library/ee943508.aspx  (info on Docpush).

     


    Igor Veytskin
    Wednesday, February 1, 2012 2:17 PM
    Moderator
  • Hi Igor

        i also confuse about how to delete the initial item crawled before.

        I got "internalid" by your advice

       http://blogs.msdn.com/b/thomsven/archive/2011/01/26/seeing-what-actual-gets-indexed.aspx

    With that internalid,what's next step with docpush?   

    My "sumsang" item above is crawled by jdbc before,could "docpush" delete it,in my view docpush can only manipulate share file.

    The "sumsang" url is looked like this" http://xxx:9002/Pages/sumsang "   ,does "docpush" ask to supply url not like this?

     

    ps:

    1.Could i find this record in SomeWhere in SQL SERVER Database and delete it directly.But i don't know which table  contains "sumsang" value.

     

    2.In Central Administraion--->FastQuerySSA:Remove URLs From Search Results,may be i can remove the "sumsang",but it not worked.


    In God We Trust
    Thursday, February 2, 2012 9:37 AM
  • Fiyoung,

     

    If you have the URL now, try doing something like this:

     

    <FASTSearchFolder>\bin\docpush -c sp -d http://xxx:9002/Pages/sumsang

     

    The "-d" flag should send a document for deletion instead of pushing it into the index. If the command succeeds and there are no errors thrown, give it a few minutes and check whether this document disappears from search results.

     

    1) No, you cannot find this record in SQL db. This is where FAST for Sharepoint differs from core Sharepoint, as documents are stored in a binary index on FAST index server/servers in %FASTSEARCH\data\data_index directory. You cannot manipulate this index as you would a SQL db, i.e find some specific record very easily and delete them. "Docpush" is best used for deleting specific documents as long as you know the URL or file location.

     

    2) I've never used it, but this might not apply to FAST, but only core Sharepoint search.


    Igor Veytskin
    Thursday, February 2, 2012 2:54 PM
    Moderator
  • Hi,

    As a workaround you could remove the content source, and then re-add it. Removing a content source will send commands to FAST for deleting the item.

    Are you getting new items into the index, or other items which have changed? If not it might be that the indexer has been suspended and you need to kick it alive again "indexeradmin -a resumeindexing".

    You can also delete that one item with "docpush -d -c sp ssic://<id>" where <id> is the id of the item (can be found in the crawl log).

    Regards,
    Mikael Svenson 


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/
    Thursday, February 2, 2012 6:39 PM
  • hi,

        The "sumsang" value is crawled by jdbc,and i could not find the <id> in FastConten SSA of CA  crawled log

        I want find <id> with "docpush -d -c sp ssic://<id>" 

         When i look the sumsang result in core result page i got the contentid  look like "sumsang",other nodes look like ssic://****.

         So how can i find the <id> by jdbc crawling.


    In God We Trust
    Friday, February 3, 2012 7:58 AM
  • Hi Igor

        i try to do like this <FASTSearchFolder>\bin\docpush -c sp -d http://xxx:9002/Pages/sumsang

     

         but nothing happened in powershell and "sumsang" remain in the result page.


    In God We Trust
    Friday, February 3, 2012 8:00 AM
  • Hi,

    Ah.. using JDBC. Then use the contentid listed instead of ssic.

    Regards,
    Mikael Svensin 


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/
    Friday, February 3, 2012 11:03 AM
  • Hi Mikael

        I also can't delete this item,so sad am i.

        this is the value in xml expression:

       

    <All_Results>
        <Result>
          <id>1</id>
          <ubmurl></ubmurl>
          <manufacturer>sumsang</manufacturer>
          <workid></workid>
          <rank>2241</rank>
          <title>sumsang</title>
          <author></author>
          <size></size>
          <path>sumsang</path>
          <description></description>
          <write>12/15/2011</write>
          <sitename></sitename>
          <collapsingstatus></collapsingstatus>
          <hithighlightedsummary>198727 EEPROM, EPROM N/A CERDIP <c0>111</c0> 1M sumsang 0 sa Instruments sumsang GI 87254456-25 8726 HGB 2222 27256-31 ups</hithighlightedsummary>
          <hithighlightedproperties>
            <HHTitle>sumsang</HHTitle>
            <HHUrl>sumsang</HHUrl>
          </hithighlightedproperties>
          <imageurl imageurldescription="Item icon">/_layouts/images/folder.gif</imageurl>
          <contentclass></contentclass>
          <isdocument>True</isdocument>
          <picturethumbnailurl></picturethumbnailurl>
          <url>sumsang</url>
          <serverredirectedurl></serverredirectedurl>
          <fileextension></fileextension>
          <spsiteurl></spsiteurl>
          <docvector>[sumsang, 1]</docvector>
          <fcocount>0</fcocount>
          <fcoid>0</fcoid>
          <pictureheight></pictureheight>
          <picturewidth></picturewidth>
          <internalid>66b33d38818c190f1c1edc3cb3fc5c03_sp</internalid>
          <contentid>sumsang</contentid>
        </Result>
        <TotalResults>1</TotalResults>
        <NumberOfResults>1</NumberOfResults>
      </All_Results>
    

    I try for these steps,all failed and also restart index in  CA ,and i also recrawl jdbc.

    Now i confuse about what  this "sumsang" really is,from jdbc?from other content source?

    It really affect the user experience.i feel so sad.


    In God We Trust
    Monday, February 6, 2012 2:15 AM
  • Hi,

    Could you post some settings from your jdbctemplate.xml? For example, do you crawl with incremental or full indexing each time? And how many items are you indexing?

    Regards,
    Mikael Svenson 


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/
    Monday, February 6, 2012 7:38 AM
  • hi,

       1. maybe i should say that "sumsang" is existed in database before,now it is changed to "samsung" in right way.

         2.i think it is a full crawling.

    <!DOCTYPE FastConfig SYSTEM "dtd/FastConnectorConfig-1.0.dtd">
    
    <!-- For permissions and the most current information about FAST Search Server 2010 for SharePoint configuration files, see the online documentation, (http://go.microsoft.com/fwlink/?LinkId=1632279). -->
    
    <FastConfig>
      <description>
    <![CDATA[This file (or usually a copy of it) need to be configured with the right values for the connector to perform properly.]]>
      </description>
      <group name="Input">
    	<description>
    	<![CDATA[The parameters in the "Input" group are defining properties of the sources from where the data will be fetched ]]>
    	</description>
    	<parameter name="JDBCDriver" type="string">
    		<description>
    			<![CDATA[MANDATORY. JDBC driver class name. See JDBC driver documentation. Examples: <br>Oracle: oracle.jdbc.driver.OracleDriver <br>MS SQL 2k: com.microsoft.jdbc.sqlserver.SQLServerDriver <br>MS SQL 2005: com.microsoft.sqlserver.jdbc.SQLServerDriver <br>DB2 (UDB 8 JDBC): com.ibm.db2.jcc.DB2Driver <br>MySQL: com.mysql.jdbc.Driver <br>Sybase net.sourceforge.jtds.jdbc.Driver <br>Default: (none)]]>
    		</description>
    		<!-- Example driver values. Remove comment around value tag to use them -->
    		<!-- <value>oracle.jdbc.driver.OracleDriver</value> --><!-- Oracle -->
    		<!-- <value>com.microsoft.jdbc.sqlserver.SQLServerDriver</value> --><!--  MS SQL 2000 -->
    		<!-- <value>com.microsoft.sqlserver.jdbc.SQLServerDriver</value> --><!--  MS SQL 2005/2008 -->
    		<!-- <value>com.ibm.db2.jcc.DB2Driver</value> --><!--  DB2 -->
    		<!-- <value>com.mysql.jdbc.Driver</value> --><!--  MySQL -->
    		<!-- <value>net.sourceforge.jtds.jdbc.Driver</value> --><!--  Sybase -->
    		<value>com.microsoft.sqlserver.jdbc.SQLServerDriver</value>
    	</parameter>
    	<parameter name="JDBCURL" type="string">
    		<description>
    			<![CDATA[MANDATORY. JDBC URL. See JDBC driver documentation for format. Examples: <br>Oracle: jdbc:oracle:thin:@yourServerHere:1521:yourDataBaseNameHere <br>MS SQL 2k: jdbc:microsoft:sqlserver://yourServerHere:1433;DatabaseName=yourDataBaseNameHere <br>MS SQL 2005: jdbc:sqlserver://yourServerHere:1433;database=yourDataBaseNameHere(default is 1433) <br>DB2 (UDB 8) JDBC: jdbc:db2://yourServerHere:50000/yourDatabaseNameHere<br>MySQL: jdbc:mysql://host:3306/yourDatabaseHere?param1=value1&amp;param2=value2 <br>jTDS driver against Sybase: jdbc:jtds:sybase://yourServerHere:<port>/yourDataBaseNameHere <br>Default: (none)]]>
    		</description>
    		<!-- Example connection string values. Remove comment around value tag to use them -->
    		<!-- <value>jdbc:oracle:thin:@yourServerHere:1521:yourDataBaseNameHere</value> --><!-- Oracle -->
    		<!-- <value>jdbc:microsoft:sqlserver://yourServerHere:1433;DatabaseName=yourDataBaseNameHere</value> --><!--  MS SQL 2000 -->
    		<!-- <value>jdbc:sqlserver://yourServerHere:1433;database=yourDataBaseNameHere</value> --><!--  MS SQL 2005/2008 -->
    		<!-- <value>jdbc:db2://yourServerHere:50000/yourDatabaseNameHere</value> --><!--  DB2 -->
    		<!-- <value>jdbc:mysql://host:3306/yourDatabaseHere?param1=value1&amp;param2=value2</value> --><!--  MySQL -->
    		<!-- <value>jdbc:jtds:sybase://yourServerHere:<port>/yourDataBaseNameHere</value> --><!--  Sybase -->
    		<value>jdbc:sqlserver://sh-search:1433;database=ubm</value>
    	</parameter>
    	<parameter name="JDBCUsername" type="string">
    		<description>
    			<![CDATA[Database username. <br>Default: (none)]]>
    		</description>
    		<value>sa</value>
    	</parameter>
    	<parameter name="JDBCPassword" type="string">
    		<description>
    			<![CDATA[Database password. <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCSQL" type="string">
    		<description>
    			<![CDATA[This or JDBCSQLFile must be provided. SQL query to crawl against.  Note, any valid SQL is valid here.  <br>Use %TIMESTAMP% where last crawl time gets inserted as a datetime value.  <br>Use %TIMESTAMPSEC% where last crawl time gets inserted as number of seconds since epoch <br> Examples: <br> Oracle: SELECT * FROM tableName WHERE dateField > TO_TIMESTAMP('%TIMESTAMP%','yyyy-MM-DD"T"hh24:mi:ss')&nbsp; <br>Note that the time stamp format used must be as indicated here. <br>MS SQL Server: SELECT * from tableName WHERE dateField > convert(datetime,'%TIMESTAMP%',126)<br>select * from employees <br>Default: (none)]]>
    		</description>
    		<value>SELECT [Manufacturer],[SI Number],[Part Number],'http://sh-sharepoint:8002/sales/devicedetail.aspx?SI_Number=' + Convert(varchar(10),[SI Number])  AS ubmurl
          ,[Size]
          ,[Device Descr.]
          ,[Date Code]
          ,[Technologies]
          ,[Package Type]
          ,[Pkg Top Marking]
          ,[Package Size]
          ,[Intact Packages]
          ,[Die Size]
          ,[Supplier] FROM device</value>
    	</parameter>
    	<parameter name="JDBCSQLFile" type="string">
    		<description>
    			<![CDATA[This or JDBCSQL must be provided. Name of file (absolute path) containing SQL query to crawl against. Use if the query is too long for the GUI. <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>	
    	<parameter name="JDBCDeleteSQL" type="string">
    			<description>
    			<![CDATA[SQL query to delete against.  Note, any valid SQL is valid here.  <br>Use %TIMESTAMP% where last delete time gets inserted as a datetime value.  <br>Use %TIMESTAMPSEC% where last crawl time gets inserted as number of seconds since epoch <br> Examples: <br> Oracle: SELECT * FROM tableName WHERE dateField > TO_TIMESTAMP('%TIMESTAMP%','yyyy-MM-DD"T"hh24:mi:ss')&nbsp; <br>Note that the time stamp format used must be as indicated here. <br>MS SQL Server: SELECT * from tableName WHERE dateField > convert(datetime,'%TIMESTAMP%',126)<br>select * from employees <br>Default: (none)]]>
    			</description>
    
    			<value><![CDATA[]]></value>
    	</parameter>
    	<parameter name="JDBCDeleteSQLFile" type="string">
    			<description>
    			<![CDATA[Name of file (absolute path) containing SQL query to delete against. Use if the query is too long for the GUI. <br>Default: (none)]]>
    			</description>
    			<value></value>
    	</parameter>
    	<parameter name="JDBCPreSQL" type="list" separator=";">
    		<description>
    			<![CDATA[SQL statements that are run prior to running the extract statement. <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCPostSQL" type="list" separator=";">
    		<description>
    			<![CDATA[SQL statements that are run after the extract statement. <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCTimeStampField" type="string">
    		<description>
              <![CDATA[Timestamp field to use as the source for last crawl time when using time stamp based update detection. Give the name of the DB timestamp column as value. The highest value for the given attribute is set as the last crawl time. Note: using this field implies that a timestamp is extracted by the SQL statement. <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCTimeType" type="string">
    		<description>
    			<![CDATA[Choose date or seconds.<br>Default: date]]>
    		</description>
    		<value>date</value>
    	</parameter>
    	<parameter name="JDBCDeleteTimeStampField" type="string">
    			<description>
              <![CDATA[Timestamp field to use as the source for last delete time when using time stamp based deletion. Give the name of the DB timestamp column as value. The highest value for the given attribute is set as the last crawl time. Note: using this field implies that a timestamp is extracted by the SQL statement. <br>Default: (none)]]>
    			</description>
    			<value></value>
    	</parameter>	
    	<parameter name="JDBCDeleteTimeType" type="string">
    			<description>
    			<![CDATA[Time used in the JDBC delete timestamp file. Choose date or seconds.<br>Default: date]]>
    			</description>
    			<value>date</value>
    	</parameter>
    	<parameter name="JDBCUpdateTimeStampOnFailure" type="boolean">
    		<description>
    			<![CDATA[If the connector receives error callbacks from FAST Search Server 2010 for SharePoint or there are missing callbacks, should the timestamp still file be updated? <br>Default: false]]>
    		</description>
    		<value>false</value>
    	</parameter>
    	<parameter name="JDBCUpdateTimeStampOnDeletion" type="boolean">
    			<description>
    			<![CDATA[should the delete timestamp file still needs to be updated upon deletion <br>Default: false]]>
    			</description>
    			<value>false</value>
    	</parameter>
    	<parameter name="JDBCFetchSizeHint" type="integer">
    		<description>
    			<![CDATA[Gives the JDBC driver a hint as to the number of rows that should be fetched from the database when more rows are needed. The number of rows specified affects only result sets created using this statement. If the value specified is zero, then the hint is ignored. <br>Default: 0]]>
    		</description>
    		<value>0</value>
    	</parameter>
    	<parameter name="JDBCCharacterEncoding" type="string">
    		<description>
    			<![CDATA[Encoding used in database. Relevant when the table contains character data (LONGVARCHAR type) <br>Default: UTF-8]]>
    		</description>
    		<value>UTF-8</value>
    	</parameter>
      </group>
    
      <group name="Transformation">
    	<description>
    	<![CDATA[The parameters in "Transformation" group are describing which data, and how data are transformed into the FAST Search Server 2010 for SharePoint items. ]]>
    	</description>
    	<parameter name="JDBCPrimaryKeyField" type="string">
    		<description>
    			<![CDATA[MANDATORY. The database view/table primary key.  Used to create the Fast Document ID. If GetValidFieldNames is true, you can also use the converted key name here (lowercase, only alphanumeric). If GetValidFieldNames is false, make sure that the use of case matches what comes back from the database in your SELECT statement. <br>Example: employeeid <br>Default: (none)]]>
    		</description>
    		<value>SI Number</value>
    	</parameter>
    	<parameter name="JDBCDeletePrimaryKeyField" type="string">
    			<description>
    			<![CDATA[The database view/table primary key.  Used to Delete the Fast Document ID. <br>Default: (none)]]>
    			</description>
    			<value></value>
    	</parameter>
    	<parameter name="JDBCUseBytesForString" type="boolean">
    		<description>
    			<![CDATA[Create byte arrays instead of strings.  Workaround for some encoding issues. <br>Default: false]]>
    		</description>
    		<value>false</value>
    	</parameter>
    	<parameter name="UseMultiRowNormalizer" type="boolean">
    		<description>
    			<![CDATA[If true, values for columns of records with identical identifiers are merged. Note: In order for
    			this feature to work, the SELECT statement must have an ORDER BY clause on the primary key field. <br>Default: false]]>
    		</description>
    		<value>false</value>
    	</parameter>
    	<parameter name="JDBCAttributeColumn" type="string">
    		<description>
    			<![CDATA[For normalized tables, the column that contain attribute names. 
    			This setting must not be used with JDBCMultiValueIncludeFields and JDBCMultiValueSkipFields.
    			Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCValueColumn" type="string">
    		<description>
    			<![CDATA[For normalized tables, the column that contain attribute values.
    			This setting must not be used with JDBCMultiValueIncludeFields and JDBCMultiValueSkipFields.
    			Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCMultiValueIncludeFields" type="string">
    		<description><![CDATA[Semi colon separated list of column names that should be concatenated in the output. 
    		If this field is set, it'll override any settings to JDBCMultiValueSkipFields.
    		This setting must not be used with JDBCAttributeColumn and JDBCValueColumn.]]></description>
    		<value></value>
    	</parameter>
    	<parameter name="JDBCMultiValueSkipFields" type="string">
    		<description>
    			<![CDATA[Semi colon separated list of column names that have the same values in each row, and that should not be repeated in the output.
    			This setting must not be used with JDBCAttributeColumn and JDBCValueColumn.
    			Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
      </group>
    
      <group name="Output" expand="no">
    	<description>
    	<![CDATA[The parameter in the "Output" group is defining the operation mode of the connector: Add/Update]]>
    	</description>
    	<parameter name="OperationMode" type="string">
    		<description>
    			<![CDATA[Must be one of add, or update. 
    			Update should only be used if you want to do a partial update on already existing documents, i.e. during an incremental crawl.  
    			Default: (add)]]>
    		</description>
    		<valid-values>add;update</valid-values>
    		<value>add</value>
    	</parameter>  
      </group>
    
      <group name="FASTSearchSubmit" expand="no">
    	<description>
    	<![CDATA[The parameters in the "FASTSearchSubmit" group describe properties of the communication with FAST Search Server 2010 for SharePoint when submitting data.  ]]>
    	</description>
    	<parameter name="Collection" type="string" >
    		<description>
    			<![CDATA[MANDATORY. Name of collection to feed to. <br>Default: (none)]]>
    		</description>
    		<value>sp</value>
    	</parameter>
    	<parameter name="ActuallyPublish" type="boolean">
    		<description>
    			<![CDATA[If true, actually submit the documents to FAST Search Server 2010 for SharePoint. <br>Default: true ]]>
    		</description>
    		<value>true</value>
    	</parameter>
      </group>
    
      <group name="XMLExport" expand="no">
    	<description>
    	<![CDATA[The parameters in the "XMLExport" group are defining values regarding export to XML formatted file(s)  ]]>
    	</description>
    	<parameter name="ExportToXML" type="boolean">
    		<description>
    			<![CDATA[If true, export all documents as XML. <br>Default: false ]]>
    		</description>
    		<value>false</value>
    	</parameter>
    	<parameter name="BinaryFields" type="string">
    		<description>
    			<![CDATA[Semi colon separated list of field names of binary fields. Will be BASE64 encoded in the XML. <br>Default: data]]>
    		</description>
    		<value>data</value>
    	</parameter>  
      </group>
    
      <group name="ChangeDetection" expand="no">
    	<description>
    	<![CDATA[The parameters in the "ChangeDetection" group specify how to detect changes  in the database since last feeding from the database(s), using row checksums  ]]>
    	</description>
    	<parameter name="Enabled" type="boolean">
    		<description>
    			<![CDATA[If true, changedetection will be enabled. <br>Default: false ]]>
    		</description>
    		<value>true</value>
    	</parameter>
    	<parameter name="ChangeDBPurge" type="boolean">
    		<description>
    			<![CDATA[If true, will delete all entries in the checksum database on startup. <br>Default: false ]]>
    		</description>
    		<value>false</value>
    	</parameter>
    	<parameter name="ChangeDBAbortPercent" type="integer">
    		<description>
    			<![CDATA[When using checksum based change detection, this value defines the maximum percentage of accepted deleted records in the source.  If a higher percentage of deletions is observed during a run, this is considered an error situation, and no changes are performed to the FAST Search Server 2010 for SharePoint collection. <br>Default: 10]]>
    		</description>
    		<value>10</value>
    	</parameter>
    	<parameter name="ChangeDBIncludeFields" type="string">
    		<description>
    			<![CDATA[A semi colon separated list of field names that are to be included in the checksum computations. <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>
    	<parameter name="ChangeDBExcludeFields" type="string">
    		<description>
    			<![CDATA[A semi colon separated list of field names that are to be excluded from the checksum computations. If present, these take precedence over the include field list. As an example, if one knows that only the metadata of a document may change, not the binary content, one can exclude the data element from the computations.  <br>Default: (none)]]>
    		</description>
    		<value></value>
    	</parameter>  
      </group>
    
      <group name="ConnectorExecution" expand="no">
    	<description>
    	<![CDATA[The parameters in the "ConnectorExecution" group specify general values used during execution of the connector  ]]>
    	</description>
    	<parameter name="QueueSize" type="integer">
    		<description>
    			<![CDATA[Queue size for internal queue of documents. <br>Default: 200, Range: 1 - 10000]]>
    		</description>
    		<range>1-10000</range>
    		<value>200</value>
    	</parameter>
    	<parameter name="EnableStatustracker" type="boolean">
    		<description>
    			<![CDATA[The status tracker logs the item crawl status to the database. Set to false to disable status tracking.]]>
    		</description>
    		<value>true</value>
    	</parameter>
        <parameter name="WorkFolder" type="string">
           	<description>
                <![CDATA[The work folder for all file output, including log files and XML files]]>
            </description>
            <value>../var/jdbcconnector</value>
        </parameter>
      </group>
      <group name="General">
    		<parameter name="Name" type="string">
    			<description>
    				<![CDATA[Configuration name. This specifies the prefix for the propset, and must only consist of the characters 0-9 and a-f. Maximum 32 characters. The parameter is optional.]]>
    			</description>
    			<value></value>
    		</parameter>
      </group>
    </FastConfig>
    

     

     

     

    jdbcconnector.bat start -f D:\FASTSearch\etc\ubmJDBCConfig.xml

     

     

     

    Copyright (c) Microsoft Corporation.  All rights reserved.

    17:21:46,444 INFO  [JDBCConnector] Starting the connector!

    17:21:46,494 INFO  [JDBCConnector] Validating config.......

    17:21:48,067 INFO  [JDBCConnector] Testing connections to external systems

    17:21:48,163 INFO  [JDBCConnector] Checking if connections to source and target

    work....

    17:21:50,277 INFO  [JDBCAdapter] Opened JDBC Connection

    17:21:50,279 INFO  [JDBCConnector] Connection made to source system

    17:21:50,285 INFO  [CCTKDocumentFeeder] Publisher :Initializing: com.fastsearch.

    esp.cctk.publishers.CCTKDocumentFeeder

    17:21:50,955 INFO  [CCTKDocumentFeeder] Using Content Distributor(s): sh-search.

    semiconductor.com:13390

    17:21:51,910 INFO  [CCTKDocumentFeeder] Initiated CCTK Document Feeder:

    17:21:51,911 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/ActuallyPublish:true

    17:21:51,914 INFO  [CCTKDocumentFeeder] XMLExport/ExportToXML:false

    17:21:51,914 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/Collection:sp

    17:21:51,914 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxActiveDocuments(MB):

    10

    17:21:51,915 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxBatchSize(KB):1000

    17:21:51,915 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/BatchSubmissionTimeoutS

    ec:30

    17:21:51,915 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/TimeOut:60

    17:21:51,916 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxDocsInBatch:100

    17:21:51,916 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxRetries:3

    17:21:51,916 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/ContentDistributors:sh-

    search.semiconductor.com:13390

    17:21:51,917 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/CallbackType:Secured

    17:21:51,917 INFO  [JDBCConnector] Connection made to target system

    17:21:51,917 INFO  [JDBCConnector] Initializing.......

    17:21:51,942 INFO  [JDBCConnector] Starting crawl.......

    17:21:51,944 INFO  [JDBCConnector] Will start up publishers......

    17:21:51,944 INFO  [CCTKDocumentFeeder] Publisher :Initializing: com.fastsearch.

    esp.cctk.publishers.CCTKDocumentFeeder

    17:21:51,981 INFO  [CCTKDocumentFeeder] Using Content Distributor(s): sh-search.

    semiconductor.com:13390

    17:21:52,150 INFO  [CCTKDocumentFeeder] Initiated CCTK Document Feeder:

    17:21:52,151 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/ActuallyPublish:true

    17:21:52,152 INFO  [CCTKDocumentFeeder] XMLExport/ExportToXML:false

    17:21:52,152 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/Collection:sp

    17:21:52,153 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxActiveDocuments(MB):

    5

    17:21:52,154 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxBatchSize(KB):1000

    17:21:52,154 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/BatchSubmissionTimeoutS

    ec:30

    17:21:52,155 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/TimeOut:60

    17:21:52,156 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxDocsInBatch:100

    17:21:52,157 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/MaxRetries:3

    17:21:52,158 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/ContentDistributors:sh-

    search.semiconductor.com:13390

    17:21:52,159 INFO  [CCTKDocumentFeeder] FASTSearchSubmit/CallbackType:Secured

    17:21:52,159 INFO  [JDBCConnector] Starting publisher thread 0

    17:21:52,193 INFO  [JDBCAdapter] Opened JDBC Connection

    17:21:52,222 INFO  [JDBCAdapter] Running PRESQL

    17:21:52,224 INFO  [JDBCAdapter] Finished running PRESQL

    17:21:52,225 INFO  [JDBCAdapter] Running SQL: SELECT [Manufacturer],[SI Number],

    [Part Number],'http://sh-sharepoint:8002/sales/devicedetail.aspx?SI_Number=' + C

    onvert(varchar(10),[SI Number])  AS ubmurl

          ,[Size]

          ,[Device Descr.]

          ,[Date Code]

          ,[Technologies]

          ,[Package Type]

          ,[Pkg Top Marking]

          ,[Package Size]

          ,[Intact Packages]

          ,[Die Size]

          ,[Supplier] FROM device

    17:21:52,276 INFO  [JDBCAdapter] The time taken to execute SQL is : 0m 0s

    17:21:52,299 INFO  [JDBCAdapter] Running postSQL

    17:21:52,300 INFO  [JDBCAdapter] Finished running postSQL

    17:21:52,317 INFO  [JDBCAdapter] Closed JDBC connection

    17:21:52,528 INFO  [CCTKDocumentFeeder] Publisher 0: Last document in queue has

    been read

    17:21:52,529 INFO  [CCTKDocumentFeeder] Publisher 0: Shutting down: com.fastsear

    ch.esp.cctk.publishers.CCTKDocumentFeeder

    17:21:52,540 INFO  [CCTKDocumentFeeder] Waiting for FAST Search feeder to comple

    te.....

    17:21:52,547 INFO  [CCTKDocumentFeeder] Feeding to FAST Search completed.....

    17:21:52,826 INFO  [CCTKDocumentFeeder] At shutdown, the following statistics ha

    ve been collected:

    17:21:52,828 INFO  [CCTKDocumentFeeder] Total number of extracted documents...:8

     

    17:21:52,829 INFO  [CCTKDocumentFeeder] Rate of extracted documents...........:8

    .879024

    17:21:52,830 INFO  [CCTKDocumentFeeder] Total number of published documents...:8

     

    17:21:52,831 INFO  [CCTKDocumentFeeder] Rate of published documents...........:8

    .859358

    17:21:52,832 INFO  [CCTKDocumentFeeder] Change detection statistics for this rou

    nd:

    17:21:52,833 INFO  [CCTKDocumentFeeder] Number of added documents: 0

    17:21:52,834 INFO  [CCTKDocumentFeeder] Number of updated documents: 0

    17:21:52,835 INFO  [CCTKDocumentFeeder] Number of removed documents: 0

    17:21:52,835 INFO  [CCTKDocumentFeeder] Total number of successful callbacks..:0

    17:21:52,836 INFO  [CCTKDocumentFeeder] Total number of failed callbacks......:0

     

     

    I am really sure there is no "sumsang" in database now.

     

    Sorry ,it is too long.


    In God We Trust

    • Edited by Fiyoung Monday, February 6, 2012 9:31 AM e
    Monday, February 6, 2012 9:24 AM
  • Hi,

    Try turning off change detection in the config as per below and see if it will re-feed all items. You can also try to set ChangeDBPurge to true to reset your checksum database.

    <group name="ChangeDetection" expand="no">
    	<parameter name="Enabled" type="boolean">
    		<span style="background-color:#ffff00"><value>false</value></span>
    	</parameter>
    .
    .
    .
    </group>
    


    Regards,
    Mikael Svenson 

     


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/
    Monday, February 6, 2012 12:41 PM
  • hi Mikael

        en...,i tried it as your method,but i also find the "sumsang" result which i don't want.

        So appreciate for your help.


    In God We Trust

    Tuesday, February 7, 2012 2:43 AM
  • Hi,

    Did the log from the indexing indicate any deletes/inserts? Also, are you able to get new items into your index (made searchable)?

    Regards,
    Mikael Svenson


    Search Enthusiast - SharePoint MVP/WCF4/ASP.Net4
    http://techmikael.blogspot.com/

    Tuesday, February 7, 2012 12:05 PM
  • Have you tried to clear search content collection?

    Login to FAST server, run PowerShell, run Get-FASTSearchContentCollection, note name of yours, then run Clear-FASTSearchContentCollection.

    It usually works for me

    Monday, February 20, 2012 12:57 PM
  • hi CoderHere,

        Clear-FASTSearchContentCollection  is very useful.

       

        Is there any possible that it will remove the content want but not all of the contents.

     

        Now,it remove all of contents and i have to recrawl source again. 

        

        


    In God We Trust

    Tuesday, February 21, 2012 3:25 AM
  • Hm... May be I do not understand the goal.

    If it is onetime situation, then clear of content collection is the way to go. You reset errorneous value and recrawl database - since database has only correct values - there should not be any problem.

    If it can happen from time to time - then correction of record in database should result in FAST index update on next run of jdbcconnector, and again there should not be any problem.

    If you delete errorneous record from DB instead of update, then it should result in FAST index update if ChangeDetection is enabled, and again there should not be any problem.

    Or I got it wrong?

    • Marked as answer by Fiyoung Wednesday, February 29, 2012 6:05 AM
    Tuesday, February 21, 2012 12:09 PM
  • It's a good choice

    Thank you very much.


    In God We Trust

    Wednesday, February 22, 2012 2:31 AM