Package org.codelibs.fess.crawler

Class FessCrawlerThread

java.lang.Object
org.codelibs.fess.crawler.CrawlerThread
org.codelibs.fess.crawler.FessCrawlerThread
All Implemented Interfaces:
Runnable
public class FessCrawlerThread extends org.codelibs.fess.crawler.CrawlerThread

  • Field Summary

    Fields inherited from class org.codelibs.fess.crawler.CrawlerThread

    clientFactory, crawlerContainer, crawlerContext, dataService, logHelper, noWaitOnFolder, urlQueueService

  • Constructor Summary

    Constructors
    Constructor
    Description
    FessCrawlerThread()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected Set<org.codelibs.fess.crawler.entity.RequestData>
    getAnchorSet(Object obj)
     
    protected Set<org.codelibs.fess.crawler.entity.RequestData>
    getChildUrlSet(SearchEngineClient searchEngineClient, String id)
     
    protected boolean
    isContentUpdated(org.codelibs.fess.crawler.client.CrawlerClient client, org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue)
     
    protected void
    processResponse(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue, org.codelibs.fess.crawler.entity.ResponseData responseData)
     
    protected void
    storeChildUrl(String childUrl, String parentUrl, String metaData, int depth)
     
    protected void
    storeChildUrlsToQueue(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue, Set<org.codelibs.fess.crawler.entity.RequestData> childUrlSet)
     

    Methods inherited from class org.codelibs.fess.crawler.CrawlerThread

    addSitemapsFromRobotsTxt, finishCrawling, getClient, isContinue, isNoWaitOnFolder, isValid, log, run, setNoWaitOnFolder, startCrawling, storeChildUrls

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • FessCrawlerThread

      public FessCrawlerThread()

  • Method Details

    • isContentUpdated

      protected boolean isContentUpdated(org.codelibs.fess.crawler.client.CrawlerClient client, org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue)
      Overrides:
      isContentUpdated in class org.codelibs.fess.crawler.CrawlerThread

    • storeChildUrlsToQueue

      protected void storeChildUrlsToQueue(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue, Set<org.codelibs.fess.crawler.entity.RequestData> childUrlSet)

    • getAnchorSet

      protected Set<org.codelibs.fess.crawler.entity.RequestData> getAnchorSet(Object obj)

    • getChildUrlSet

      protected Set<org.codelibs.fess.crawler.entity.RequestData> getChildUrlSet(SearchEngineClient searchEngineClient, String id)

    • processResponse

      protected void processResponse(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue, org.codelibs.fess.crawler.entity.ResponseData responseData)
      Overrides:
      processResponse in class org.codelibs.fess.crawler.CrawlerThread

    • storeChildUrl

      protected void storeChildUrl(String childUrl, String parentUrl, String metaData, int depth)
      Overrides:
      storeChildUrl in class org.codelibs.fess.crawler.CrawlerThread