Package org.codelibs.fess.crawler.transformer

Interface FessTransformer

All Known Implementing Classes:
AbstractFessFileTransformer, FessFileTransformer, FessStandardTransformer, FessXpathTransformer
public interface FessTransformer
Interface for transforming and processing crawled documents in Fess. Provides utility methods for URL processing, site extraction, data mapping, and field configuration handling during the document transformation process.

  • Field Details

    • parentEncodingMap

      static final Map<String,String> parentEncodingMap
      Synchronized LRU cache for storing parent URL encodings. Maps session+parent URL keys to their corresponding character encodings.

  • Method Details

    • getFessConfig

      org.codelibs.fess.mylasta.direction.FessConfig getFessConfig()
      Gets the Fess configuration instance.
      Returns:
      the Fess configuration object

    • getLogger

      org.apache.logging.log4j.Logger getLogger()
      Gets the logger instance for this transformer.
      Returns:
      the logger instance

    • getHost

      default String getHost(String u)
      Extracts the host name from a URL string. Removes protocol and path components to return just the hostname.
      Parameters:
      u - the URL string to extract host from
      Returns:
      the host name, or empty string if URL is blank, or unknown hostname if parsing fails

    • getSite

      default String getSite(String u, String encoding)
      Extracts and processes the site path from a URL with proper encoding handling. Removes protocol, query parameters, and applies URL decoding based on encoding settings.
      Parameters:
      u - the URL string to process
      encoding - the character encoding to use for URL decoding
      Returns:
      the processed site path, abbreviated if necessary

    • putResultDataBody

      default void putResultDataBody(Map<String,Object> dataMap, String key, Object value)
      Puts data into the result data map, handling value appending if configured. If data appending is enabled and the key already exists, values are combined into arrays.
      Parameters:
      dataMap - the data map to modify
      key - the key to store the value under
      value - the value to store

    • putResultDataWithTemplate

      default void putResultDataWithTemplate(Map<String,Object> dataMap, String key, Object value, String template, String scriptType)
      Puts data into the result data map after processing it through a template script. The template is evaluated using the specified script engine with the value and context.
      Parameters:
      dataMap - the data map to modify
      key - the key to store the processed value under
      value - the original value to process
      template - the template script to evaluate
      scriptType - the type of script engine to use

    • evaluateValue

      default Object evaluateValue(String scriptType, String template, Map<String,Object> paramMap)
      Evaluates a template script using the specified script engine and parameters.
      Parameters:
      scriptType - the type of script engine to use
      template - the template script to evaluate
      paramMap - the parameters to pass to the script
      Returns:
      the result of script evaluation, or empty string if template is empty

    • getMaxSiteLength

      default int getMaxSiteLength()
      Gets the maximum allowed length for site strings from configuration.
      Returns:
      the maximum site length as configured

    • abbreviateSite

      default String abbreviateSite(String value)
      Abbreviates a site string to the maximum allowed length if configured.
      Parameters:
      value - the site string to abbreviate
      Returns:
      the abbreviated string, or original if no length limit is set

    • getFileName

      default String getFileName(String url, String encoding)
      Extracts the filename from a URL, handling various protocols and URL decoding. Processes HTTP, HTTPS, file, SMB, and FTP URLs appropriately.
      Parameters:
      url - the URL to extract filename from
      encoding - the character encoding (currently unused in this method)
      Returns:
      the extracted filename, or empty string if none found

    • decodeUrlAsName

      default String decodeUrlAsName(String url, boolean escapePlus)
      Decodes a URL as a name using appropriate character encoding. Handles encoding detection from parent URLs and configuration settings.
      Parameters:
      url - the URL to decode
      escapePlus - whether to escape plus signs before decoding
      Returns:
      the decoded URL name, or original URL if decoding fails

    • getParentEncoding

      default String getParentEncoding(String parentUrl, String sessionId)
      Gets the character encoding for a parent URL from cache or data service. Caches encoding information to improve performance on subsequent requests.
      Parameters:
      parentUrl - the parent URL to get encoding for
      sessionId - the session ID for the crawling session
      Returns:
      the character encoding, or null if not found

    • processFieldConfigs

      default Map<String,Object> processFieldConfigs(Map<String,Object> dataMap, FieldConfigs fieldConfigs)
      Processes field configurations to handle field overwriting rules. Creates a new data map with fields processed according to their configuration.
      Parameters:
      dataMap - the original data map to process
      fieldConfigs - the field configurations to apply
      Returns:
      a new data map with configurations applied