Package org.codelibs.fess.helper

Class CrawlingInfoHelper

java.lang.Object
org.codelibs.fess.helper.CrawlingInfoHelper
public class CrawlingInfoHelper extends Object
Helper class for managing crawling information and statistics. Provides functionality to track crawling sessions, manage document expiration, and handle crawling information storage and retrieval.

  • Field Details

    • FACET_COUNT_KEY

      public static final String FACET_COUNT_KEY
      Key used for facet count aggregations.
      See Also:

    • infoMap

      protected Map<String,String> infoMap
      Map containing crawling information parameters.

    • documentExpires

      protected Long documentExpires
      Document expiration time in milliseconds.

    • maxSessionIdsInList

      protected int maxSessionIdsInList
      Maximum number of session IDs to include in lists.

  • Constructor Details

    • CrawlingInfoHelper

      public CrawlingInfoHelper()
      Creates a new instance of CrawlingInfoHelper.

  • Method Details

    • getCrawlingInfoService

      protected CrawlingInfoService getCrawlingInfoService()
      Retrieves the CrawlingInfoService component instance.
      Returns:
      the CrawlingInfoService component for managing crawling information

    • getCanonicalSessionId

      public String getCanonicalSessionId(String sessionId)
      Extracts the canonical session ID by removing any suffix after the first hyphen. If the session ID contains a hyphen, returns the portion before the first hyphen. Otherwise, returns the original session ID.
      Parameters:
      sessionId - the session ID to process
      Returns:
      the canonical session ID (portion before first hyphen, or original if no hyphen)

    • store

      public void store(String sessionId, boolean create)
      Stores crawling information and parameters for the specified session. Creates a new crawling info record if none exists or if create flag is true. Also stores any accumulated information parameters and clears the info map.
      Parameters:
      sessionId - the session ID for the crawling information
      create - if true, creates a new crawling info regardless of existing records
      Throws:
      FessSystemException - if unable to store the crawling session

    • putToInfoMap

      public void putToInfoMap(String key, String value)
      Adds a key-value pair to the information map. Initializes the info map as a synchronized LinkedHashMap if it doesn't exist.
      Parameters:
      key - the parameter key to store
      value - the parameter value to store

    • updateParams

      public void updateParams(String sessionId, String name, int dayForCleanup)
      Updates crawling information parameters for the specified session. Sets the name and expiration time based on the provided parameters.
      Parameters:
      sessionId - the session ID to update
      name - the name to set for the crawling session (uses system name if blank)
      dayForCleanup - number of days until cleanup (sets expiration if >= 0)
      Throws:
      FessSystemException - if unable to store the updated crawling session

    • getDocumentExpires

      public Date getDocumentExpires(org.codelibs.fess.opensearch.config.exentity.CrawlingConfig config)
      Calculates the document expiration date based on crawling configuration. If the config has a timeToLive value, calculates expiration from current time. Otherwise, returns the stored document expiration time.
      Parameters:
      config - the crawling configuration containing time-to-live settings
      Returns:
      the document expiration date, or null if no expiration is set

    • getExpiredTime

      protected long getExpiredTime(int days)
      Calculates the expiration time in milliseconds from the current time.
      Parameters:
      days - the number of days from now when the item should expire
      Returns:
      the expiration time in milliseconds since epoch

    • getInfoMap

      public Map<String,String> getInfoMap(String sessionId)
      Retrieves all crawling information parameters for the specified session as a map.
      Parameters:
      sessionId - the session ID to retrieve parameters for
      Returns:
      a map containing all key-value parameter pairs for the session

    • generateId

      public String generateId(Map<String,Object> dataMap)
      Generates a unique document ID from the provided data map. Constructs an ID string from URL, roles, and virtual hosts, then generates a hash.
      Parameters:
      dataMap - the document data map containing URL, roles, and virtual host information
      Returns:
      a unique hashed ID string for the document

    • getSessionIdList

      public List<Map<String,String>> getSessionIdList(SearchEngineClient searchEngineClient)
      Retrieves a list of session IDs with their document counts from the search engine. Uses aggregation to get session segments and their corresponding document counts.
      Parameters:
      searchEngineClient - the search engine client to perform the query
      Returns:
      a list of maps containing session IDs and their document counts

    • generateId

      protected String generateId(String urlId)
      Generates a hashed ID from the provided URL ID string. Encodes special characters using URL encoding or Base64 encoding as needed, then applies a message digest algorithm to create a unique hash.
      Parameters:
      urlId - the URL ID string to generate a hash for
      Returns:
      a hashed ID string generated from the input URL ID

    • setMaxSessionIdsInList

      public void setMaxSessionIdsInList(int maxSessionIdsInList)
      Sets the maximum number of session IDs to include in session ID lists. This controls the size limit for aggregation results when retrieving session lists.
      Parameters:
      maxSessionIdsInList - the maximum number of session IDs to include in lists