none
logtransformer error: total_hits.csf does not contain entry RRS feed

  • Question

  • Getting the following error every hour in the logtransformer.log (the entry number is different - higher - each time):

    [2011-09-29 11:15:00,571] ERROR com.fastsearch.esp.logtransformer.scheduler.LqlJob: The lql job had problems running  (FastLogTransformer_Worker-0)
    java.lang.IllegalArgumentException: The file E:\esp\components\logtransformer\data\datastore\querylog\crate0\total_hits.csf does not contain entry: 1559513
     at com.fastsearch.esp.logtransformer.datastore.io.ColumnSegmentFileReaderImpl.getEntry(ColumnSegmentFileReaderImpl.java:150)
     at com.fastsearch.esp.logtransformer.datastore.processor.visitor.AstEvaluatorVisitor.visit(AstEvaluatorVisitor.java:494)
     at com.fastsearch.esp.logtransformer.tql.ast.SimpleSelectField.acceptVisitor(SimpleSelectField.java:136)
     at com.fastsearch.esp.logtransformer.datastore.processor.visitor.AstEvaluatorVisitor.visit(AstEvaluatorVisitor.java:461)
     at com.fastsearch.esp.logtransformer.tql.ast.criterion.SimpleCriterion.acceptVisitor(SimpleCriterion.java:86)
     at com.fastsearch.esp.logtransformer.datastore.processor.visitor.AstEvaluatorVisitor.visit(AstEvaluatorVisitor.java:372)
     at com.fastsearch.esp.logtransformer.tql.ast.criterion.JunctionCriterion.acceptVisitor(JunctionCriterion.java:95)
     at com.fastsearch.esp.logtransformer.datastore.processor.visitor.AstEvaluatorVisitor.visit(AstEvaluatorVisitor.java:382)
     at com.fastsearch.esp.logtransformer.tql.ast.criterion.JunctionCriterion.acceptVisitor(JunctionCriterion.java:95)
     at com.fastsearch.esp.logtransformer.datastore.io.EntriesIterator.evaluateConstraints(EntriesIterator.java:228)
     at com.fastsearch.esp.logtransformer.datastore.io.EntriesIterator.findNextEntry(EntriesIterator.java:173)
     at com.fastsearch.esp.logtransformer.datastore.io.EntriesIterator.next(EntriesIterator.java:123)
     at com.fastsearch.esp.logtransformer.datastore.io.CrateEntryIterator.next(CrateEntryIterator.java:56)
     at com.fastsearch.esp.logtransformer.lql.processor.impl.LqlSelectStmtProcessorImpl.map(LqlSelectStmtProcessorImpl.java:111)
     at com.fastsearch.esp.logtransformer.lql.processor.impl.LqlMasterImpl.executeLql(LqlMasterImpl.java:193)
     at com.fastsearch.esp.logtransformer.lql.processor.impl.LqlMasterImpl.executeLql(LqlMasterImpl.java:109)
     at com.fastsearch.esp.logtransformer.scheduler.LqlJob.execute(LqlJob.java:134)
     at org.quartz.core.JobRunShell.run(JobRunShell.java:195)
     at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520)
    [2011-09-29 11:15:00,571] DEBUG com.fastsearch.esp.admin.lql.QueryStatisticsLqlFilter: Lql statement was not executed successfully  (FastLogTransformer_Worker-0)

    Any advice?

    Thursday, September 29, 2011 5:21 PM

Answers

  • Hi Joel,

    What version of ESP and service pack are you currently at?  At earlier versions of ESP, this could be addressed by limiting the scheduling library to use a single thread, effectively forcing jobs to be run sequentially, so that the lql jobs complete. This can be configured in %FASTSEARCH%\components\logtransformer\etc\quartz.properties, with the below line:

    org.quartz.threadPool.threadCount = 1

     

    Does your quartz.properties file already have this configured?  If not, I would suggest configuring this, and restarting logtransformer.  Let us know your findings.

     

    Thanks!

    Rob Vazzana | Sr Support Escalation Engineer | US Customer Service & Support

    Customer Service & Support                          Microsoft | Services

    Monday, October 3, 2011 7:13 PM
    Moderator