Package org.codelibs.fess.util

Class GsaConfigParser

java.lang.Object
org.xml.sax.helpers.DefaultHandler
org.codelibs.fess.util.GsaConfigParser
All Implemented Interfaces:
ContentHandler, DTDHandler, EntityResolver, ErrorHandler
public class GsaConfigParser extends DefaultHandler
Parser for Google Search Appliance (GSA) configuration files. This SAX-based parser reads GSA XML configuration files and converts them into Fess configuration objects including web crawling configurations, file crawling configurations, and label types for access control.

The parser handles the following GSA configuration elements:

  • Collections with good/bad URL patterns
  • Global parameters including start URLs and filtering rules
  • User agent settings
  • URL pattern matching with regular expressions and contains filters

  • Field Details

    • REGEXP

      public static final String REGEXP
      Prefix for regular expression patterns.
      See Also:

    • REGEXP_CASE

      public static final String REGEXP_CASE
      Prefix for case-sensitive regular expression patterns.
      See Also:

    • REGEXP_IGNORE_CASE

      public static final String REGEXP_IGNORE_CASE
      Prefix for case-insensitive regular expression patterns.
      See Also:

    • CONTAINS

      public static final String CONTAINS
      Prefix for contains-based string matching patterns.
      See Also:

    • COLLECTIONS

      protected static final String COLLECTIONS
      XML element name for collections container.
      See Also:

    • COLLECTION

      protected static final String COLLECTION
      XML element name for individual collection.
      See Also:

    • GLOBALPARAMS

      protected static final String GLOBALPARAMS
      XML element name for global parameters container.
      See Also:

    • START_URLS

      protected static final String START_URLS
      XML element name for start URLs configuration.
      See Also:

    • GOOD_URLS

      protected static final String GOOD_URLS
      XML element name for good (included) URLs configuration.
      See Also:

    • BAD_URLS

      protected static final String BAD_URLS
      XML element name for bad (excluded) URLs configuration.
      See Also:

    • webProtocols

      protected String[] webProtocols
      Array of supported web protocols for URL classification.

    • fileProtocols

      protected String[] fileProtocols
      Array of supported file protocols for URL classification.

    • tagQueue

      protected LinkedList<String> tagQueue
      Queue to track the current XML element hierarchy during parsing.

    • labelList

      protected List<org.codelibs.fess.opensearch.config.exentity.LabelType> labelList
      List to store parsed label types for access control.

    • labelType

      protected org.codelibs.fess.opensearch.config.exentity.LabelType labelType
      Current label type being processed during parsing.

    • globalParams

      protected Map<String,String> globalParams
      Map to store global configuration parameters.

    • webConfig

      protected org.codelibs.fess.opensearch.config.exentity.WebConfig webConfig
      Generated web crawling configuration from parsed GSA config.

    • fileConfig

      protected org.codelibs.fess.opensearch.config.exentity.FileConfig fileConfig
      Generated file crawling configuration from parsed GSA config.

    • textBuf

      protected StringBuilder textBuf
      Buffer to accumulate character data between XML tags.

    • userAgent

      protected String userAgent
      User agent string to be used for web crawling.

  • Constructor Details

    • GsaConfigParser

      public GsaConfigParser()
      Default constructor for GsaConfigParser.

  • Method Details

    • parse

      public void parse(InputSource is)
      Parses a GSA configuration XML file from the given input source. This method configures a secure SAX parser and processes the XML content to extract configuration information for web and file crawling.
      Parameters:
      is - the input source containing the GSA configuration XML
      Throws:
      GsaConfigException - if parsing fails due to XML format issues or other errors

    • startDocument

      public void startDocument() throws SAXException
      SAX event handler called at the beginning of document parsing. Initializes internal data structures for processing the GSA configuration.
      Specified by:
      startDocument in interface ContentHandler
      Overrides:
      startDocument in class DefaultHandler
      Throws:
      SAXException - if a SAX error occurs during initialization

    • endDocument

      public void endDocument() throws SAXException
      SAX event handler called at the end of document parsing. Cleans up internal data structures used during parsing.
      Specified by:
      endDocument in interface ContentHandler
      Overrides:
      endDocument in class DefaultHandler
      Throws:
      SAXException - if a SAX error occurs during cleanup

    • startElement

      public void startElement(String uri, String localName, String qName, Attributes attributes) throws SAXException
      SAX event handler called when an XML start element is encountered. Processes collection definitions and tracks the element hierarchy.
      Specified by:
      startElement in interface ContentHandler
      Overrides:
      startElement in class DefaultHandler
      Parameters:
      uri - the namespace URI, or empty string if none
      localName - the local name without prefix, or empty string if namespace processing is not performed
      qName - the qualified name with prefix, or empty string if qualified names are not available
      attributes - the attributes attached to the element
      Throws:
      SAXException - if a SAX error occurs or if the XML format is invalid

    • endElement

      public void endElement(String uri, String localName, String qName) throws SAXException
      SAX event handler called when an XML end element is encountered. Processes the accumulated text content and creates appropriate configuration objects based on the element type (good_urls, bad_urls, start_urls, etc.).
      Specified by:
      endElement in interface ContentHandler
      Overrides:
      endElement in class DefaultHandler
      Parameters:
      uri - the namespace URI, or empty string if none
      localName - the local name without prefix, or empty string if namespace processing is not performed
      qName - the qualified name with prefix, or empty string if qualified names are not available
      Throws:
      SAXException - if a SAX error occurs during processing

    • characters

      public void characters(char[] ch, int start, int length) throws SAXException
      SAX event handler called to process character data between XML elements. Accumulates text content in a buffer for later processing when the element ends.
      Specified by:
      characters in interface ContentHandler
      Overrides:
      characters in class DefaultHandler
      Parameters:
      ch - the characters from the XML document
      start - the start position in the character array
      length - the number of characters to use from the character array
      Throws:
      SAXException - if a SAX error occurs during character processing

    • parseFilterPaths

      protected String parseFilterPaths(String text, boolean web, boolean file)
      Parses and filters URL patterns from text based on protocol types. Processes each line of the input text, filtering URLs based on web and file protocol support.
      Parameters:
      text - the raw text containing URL patterns, one per line
      web - true if web protocol URLs should be included
      file - true if file protocol URLs should be included
      Returns:
      a newline-separated string of filtered URL patterns

    • getFilterPath

      protected String getFilterPath(String s)
      Converts a GSA URL pattern into a regular expression pattern suitable for Fess. Handles various GSA pattern formats including regexp, contains, and URL-based patterns.
      Parameters:
      s - the input GSA pattern string
      Returns:
      a regular expression pattern string, or empty string for comments/invalid patterns

    • escape

      protected String escape(String s)
      Escapes special regex characters in a string to create a literal pattern. Handles anchor characters (^ and $) specially to preserve their regex meaning.
      Parameters:
      s - the string to escape
      Returns:
      an escaped regex pattern, or empty string for comments

    • unescape

      protected String unescape(String s)
      Unescapes double backslashes in regex patterns. Converts escaped backslashes (\\) back to single backslashes (\).
      Parameters:
      s - the string to unescape
      Returns:
      the unescaped string

    • appendFileterPath

      protected String appendFileterPath(StringBuilder buf, String v)
      Appends a filter path pattern to a string buffer with appropriate wildcards. Handles various pattern formats including anchored patterns and quoted patterns.
      Parameters:
      buf - the string buffer to append to
      v - the pattern value to append
      Returns:
      the complete pattern string from the buffer

    • setWebProtocols

      public void setWebProtocols(String[] webProtocols)
      Sets the array of web protocols to recognize for URL classification.
      Parameters:
      webProtocols - array of protocol prefixes (e.g., "http:", "https:")

    • setFileProtocols

      public void setFileProtocols(String[] fileProtocols)
      Sets the array of file protocols to recognize for URL classification.
      Parameters:
      fileProtocols - array of protocol prefixes (e.g., "file:", "smb:", "ftp:")

    • toString

      public String toString()
      Returns a string representation of this parser's current state. Includes information about parsed label types and configuration objects.
      Overrides:
      toString in class Object
      Returns:
      a string representation of the parser state

    • getWebConfig

      public org.dbflute.optional.OptionalEntity<org.codelibs.fess.opensearch.config.exentity.WebConfig> getWebConfig()
      Gets the web crawling configuration generated from the parsed GSA config.
      Returns:
      an optional containing the web configuration, or empty if no web URLs were found

    • getFileConfig

      public org.dbflute.optional.OptionalEntity<org.codelibs.fess.opensearch.config.exentity.FileConfig> getFileConfig()
      Gets the file crawling configuration generated from the parsed GSA config.
      Returns:
      an optional containing the file configuration, or empty if no file URLs were found

    • getLabelTypes

      public org.codelibs.fess.opensearch.config.exentity.LabelType[] getLabelTypes()
      Gets all label types (collections) parsed from the GSA configuration. Each label type represents a collection with its own URL filtering rules.
      Returns:
      an array of label types representing the parsed collections