Package org.codelibs.fess.helper

Class ProtocolHelper

java.lang.Object
org.codelibs.fess.helper.ProtocolHelper
public class ProtocolHelper extends Object
Helper class for managing and validating URL protocols in Fess crawling system. This class handles the initialization and validation of web and file protocols used by the crawler to determine which URLs can be crawled.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected String[]
    fileProtocols
    Array of supported file protocols with colon suffix (e.g., "file:", "ftp:")
    protected String[]
    webProtocols
    Array of supported web protocols with colon suffix (e.g., "http:", "https:")

  • Constructor Summary

    Constructors
    Constructor
    Description
    ProtocolHelper()
    Default constructor for ProtocolHelper.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    addFileProtocol(String protocol)
    Adds a new file protocol to the supported protocols list.
    void
    addWebProtocol(String protocol)
    Adds a new web protocol to the supported protocols list.
    String[]
    getFileProtocols()
    Returns the array of supported file protocols.
    String[]
    getWebProtocols()
    Returns the array of supported web protocols.
    boolean
    hasKnownProtocol(String path)
    Checks if the given path has a known protocol prefix that should not be converted.
    void
    init()
    Initializes the protocol helper by loading configured protocols from FessConfig and scanning for available protocol handlers in the classpath.
    boolean
    isFilePathProtocol(String url)
    Checks if the given URL is a file path protocol that requires directory and permission handling.
    boolean
    isFileSystemPath(String url)
    Checks if the given URL represents a file system path for content serving.
    boolean
    isValidFileProtocol(String url)
    Checks if the given URL uses a valid file protocol.
    boolean
    isValidWebProtocol(String url)
    Checks if the given URL uses a valid web protocol.
    protected void
    loadProtocols(String basePackage)
    Loads protocol handlers from the specified base package by scanning for Handler classes in subpackages and registering them as web or file protocols based on their PROTOCOL_TYPE field.
    boolean
    shouldSkipUrlDecode(String url)
    Checks if the given URL should skip URL decoding when extracting file names.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • webProtocols

      protected String[] webProtocols
      Array of supported web protocols with colon suffix (e.g., "http:", "https:")

    • fileProtocols

      protected String[] fileProtocols
      Array of supported file protocols with colon suffix (e.g., "file:", "ftp:")

  • Constructor Details

    • ProtocolHelper

      public ProtocolHelper()
      Default constructor for ProtocolHelper. Initializes the helper with empty protocol arrays that will be populated during init().

  • Method Details

    • init

      @PostConstruct public void init()
      Initializes the protocol helper by loading configured protocols from FessConfig and scanning for available protocol handlers in the classpath. This method is called automatically after bean construction.

    • loadProtocols

      protected void loadProtocols(String basePackage)
      Loads protocol handlers from the specified base package by scanning for Handler classes in subpackages and registering them as web or file protocols based on their PROTOCOL_TYPE field.
      Parameters:
      basePackage - the base package to scan for protocol handlers

    • getWebProtocols

      public String[] getWebProtocols()
      Returns the array of supported web protocols.
      Returns:
      array of web protocol strings with colon suffix (e.g., "http:", "https:")

    • getFileProtocols

      public String[] getFileProtocols()
      Returns the array of supported file protocols.
      Returns:
      array of file protocol strings with colon suffix (e.g., "file:", "ftp:")

    • isValidWebProtocol

      public boolean isValidWebProtocol(String url)
      Checks if the given URL uses a valid web protocol.
      Parameters:
      url - the URL to validate
      Returns:
      true if the URL starts with a supported web protocol, false otherwise

    • isValidFileProtocol

      public boolean isValidFileProtocol(String url)
      Checks if the given URL uses a valid file protocol.
      Parameters:
      url - the URL to validate
      Returns:
      true if the URL starts with a supported file protocol, false otherwise

    • addWebProtocol

      public void addWebProtocol(String protocol)
      Adds a new web protocol to the supported protocols list. If the protocol already exists, it will not be added again.
      Parameters:
      protocol - the protocol name to add (without colon suffix)

    • addFileProtocol

      public void addFileProtocol(String protocol)
      Adds a new file protocol to the supported protocols list. If the protocol already exists, it will not be added again.
      Parameters:
      protocol - the protocol name to add (without colon suffix)

    • isFilePathProtocol

      public boolean isFilePathProtocol(String url)
      Checks if the given URL is a file path protocol that requires directory and permission handling. Used for incremental crawling directory detection and file permission processing.
      Parameters:
      url - the URL to check
      Returns:
      true if the URL uses a file path protocol (smb, smb1, file, ftp, s3, gcs)

    • isFileSystemPath

      public boolean isFileSystemPath(String url)
      Checks if the given URL represents a file system path for content serving. Used to determine if special handling is needed for file system URLs.
      Parameters:
      url - the URL to check
      Returns:
      true if the URL is a file system path (file, smb, smb1, ftp, storage, s3, gcs)

    • shouldSkipUrlDecode

      public boolean shouldSkipUrlDecode(String url)
      Checks if the given URL should skip URL decoding when extracting file names. Some protocols (like SMB, FTP, S3, GCS) should preserve the original URL encoding.
      Parameters:
      url - the URL to check
      Returns:
      true if URL decoding should be skipped for this protocol

    • hasKnownProtocol

      public boolean hasKnownProtocol(String path)
      Checks if the given path has a known protocol prefix that should not be converted. Used to determine if path conversion is needed in the wizard.
      Parameters:
      path - the path to check
      Returns:
      true if the path has a known protocol prefix