Package org.codelibs.fess.crawler

Class FessCrawlerThread

  • All Implemented Interfaces:
    java.lang.Runnable
    public class FessCrawlerThread
extends org.codelibs.fess.crawler.CrawlerThread

    • Field Summary

      • Fields inherited from class org.codelibs.fess.crawler.CrawlerThread

        clientFactory, crawlerContainer, crawlerContext, dataService, logHelper, noWaitOnFolder, urlQueueService

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected java.util.Set<org.codelibs.fess.crawler.entity.RequestData> getAnchorSet​(java.lang.Object obj)  
      protected java.util.Set<org.codelibs.fess.crawler.entity.RequestData> getChildUrlSet​(FessEsClient fessEsClient, java.lang.String id)  
      protected boolean isContentUpdated​(org.codelibs.fess.crawler.client.CrawlerClient client, org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue)  
      protected void processResponse​(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue, org.codelibs.fess.crawler.entity.ResponseData responseData)  
      protected void storeChildUrl​(java.lang.String childUrl, java.lang.String parentUrl, java.lang.String metaData, int depth)  
      protected void storeChildUrlsToQueue​(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue, java.util.Set<org.codelibs.fess.crawler.entity.RequestData> childUrlSet)  

      • Methods inherited from class org.codelibs.fess.crawler.CrawlerThread

        addSitemapsFromRobotsTxt, finishCrawling, getClient, isContinue, isNoWaitOnFolder, isValid, log, run, setNoWaitOnFolder, startCrawling, storeChildUrls

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • FessCrawlerThread

        public FessCrawlerThread()

    • Method Detail

      • isContentUpdated

        protected boolean isContentUpdated​(org.codelibs.fess.crawler.client.CrawlerClient client,
                                   org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue)
        Overrides:
        isContentUpdated in class org.codelibs.fess.crawler.CrawlerThread

      • storeChildUrlsToQueue

        protected void storeChildUrlsToQueue​(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue,
                                     java.util.Set<org.codelibs.fess.crawler.entity.RequestData> childUrlSet)

      • getAnchorSet

        protected java.util.Set<org.codelibs.fess.crawler.entity.RequestData> getAnchorSet​(java.lang.Object obj)

      • getChildUrlSet

        protected java.util.Set<org.codelibs.fess.crawler.entity.RequestData> getChildUrlSet​(FessEsClient fessEsClient,
                                                                                     java.lang.String id)

      • processResponse

        protected void processResponse​(org.codelibs.fess.crawler.entity.UrlQueue<?> urlQueue,
                               org.codelibs.fess.crawler.entity.ResponseData responseData)
        Overrides:
        processResponse in class org.codelibs.fess.crawler.CrawlerThread

      • storeChildUrl

        protected void storeChildUrl​(java.lang.String childUrl,
                             java.lang.String parentUrl,
                             java.lang.String metaData,
                             int depth)
        Overrides:
        storeChildUrl in class org.codelibs.fess.crawler.CrawlerThread