Package org.codelibs.fess.helper

Class IndexingHelper

java.lang.Object
org.codelibs.fess.helper.IndexingHelper
public class IndexingHelper extends Object
Helper class for indexing operations in the Fess search engine. This class provides functionality for sending documents to the search engine, managing document lifecycle operations (create, update, delete), and handling thumbnail processing during indexing.

The IndexingHelper manages bulk operations, handles retries on failures, and provides various query-based operations for document management. It also integrates with the thumbnail generation system and handles the cleanup of old documents during updates.

  • Field Details

    • maxRetryCount

      protected int maxRetryCount
      Maximum number of retry attempts for failed operations

    • defaultRowSize

      protected int defaultRowSize
      Default number of rows to process in a single batch

    • requestInterval

      protected long requestInterval
      Interval between requests in milliseconds

  • Constructor Details

    • IndexingHelper

      public IndexingHelper()
      Default constructor for indexing helper. Creates a new instance with default values.

  • Method Details

    • sendDocuments

      public void sendDocuments(SearchEngineClient searchEngineClient, DocList docList)
      Sends a list of documents to the search engine for indexing. This method handles thumbnail processing, deletes old documents with the same URL, and performs bulk indexing operations with proper error handling.
      Parameters:
      searchEngineClient - the search engine client to use for indexing
      docList - the list of documents to be indexed
      Throws:
      SearchEngineClientException - if the bulk indexing operation fails

    • deleteOldDocuments

      protected long deleteOldDocuments(SearchEngineClient searchEngineClient, DocList docList)
      Deletes old documents that have the same URL but different document IDs as the documents in the provided list. This prevents duplicate documents from accumulating in the index.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      docList - the list of new documents to check against
      Returns:
      the number of old documents that were deleted

    • updateDocument

      public boolean updateDocument(SearchEngineClient searchEngineClient, String id, String field, Object value)
      Updates a specific field of a document in the search index.
      Parameters:
      searchEngineClient - the search engine client to use for the update
      id - the document ID to update
      field - the field name to update
      value - the new value for the field
      Returns:
      true if the update was successful, false otherwise

    • deleteDocument

      public boolean deleteDocument(SearchEngineClient searchEngineClient, String id)
      Deletes a document from the search index by its ID.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      id - the document ID to delete
      Returns:
      true if the deletion was successful, false otherwise

    • deleteDocumentByUrl

      public long deleteDocumentByUrl(SearchEngineClient searchEngineClient, String url)
      Deletes all documents that match the specified URL.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      url - the URL to match for document deletion
      Returns:
      the number of documents that were deleted

    • deleteDocumentsByDocId

      public long deleteDocumentsByDocId(SearchEngineClient searchEngineClient, List<String> docIdList)
      Deletes all documents that match the specified document IDs.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      docIdList - the list of document IDs to delete
      Returns:
      the number of documents that were deleted

    • deleteDocumentByQuery

      public long deleteDocumentByQuery(SearchEngineClient searchEngineClient, org.opensearch.index.query.QueryBuilder queryBuilder)
      Deletes all documents that match the specified query from the default update index.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      queryBuilder - the query to match documents for deletion
      Returns:
      the number of documents that were deleted

    • deleteDocumentByQuery

      protected long deleteDocumentByQuery(SearchEngineClient searchEngineClient, String index, org.opensearch.index.query.QueryBuilder queryBuilder)
      Deletes all documents that match the specified query from the specified index.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      index - the index name to delete documents from
      queryBuilder - the query to match documents for deletion
      Returns:
      the number of documents that were deleted

    • getDocument

      public Map<String,Object> getDocument(SearchEngineClient searchEngineClient, String id, String[] fields)
      Retrieves a document from the search index by its ID.
      Parameters:
      searchEngineClient - the search engine client to use for retrieval
      id - the document ID to retrieve
      fields - the fields to include in the response (null for all fields)
      Returns:
      the document as a map of field names to values, or null if not found

    • getDocumentListByPrefixId

      public List<Map<String,Object>> getDocumentListByPrefixId(SearchEngineClient searchEngineClient, String id, String[] fields)
      Retrieves a list of documents whose IDs start with the specified prefix.
      Parameters:
      searchEngineClient - the search engine client to use for retrieval
      id - the ID prefix to match documents
      fields - the fields to include in the response (null for all fields)
      Returns:
      a list of documents that match the prefix

    • deleteChildDocument

      public long deleteChildDocument(SearchEngineClient searchEngineClient, String id)
      Deletes all child documents that belong to the specified parent document.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      id - the parent document ID whose children should be deleted
      Returns:
      the number of child documents that were deleted

    • getChildDocumentList

      public List<Map<String,Object>> getChildDocumentList(SearchEngineClient searchEngineClient, String id, String[] fields)
      Retrieves all child documents that belong to the specified parent document.
      Parameters:
      searchEngineClient - the search engine client to use for retrieval
      id - the parent document ID whose children should be retrieved
      fields - the fields to include in the response (null for all fields)
      Returns:
      a list of child documents

    • getDocumentListByQuery

      protected List<Map<String,Object>> getDocumentListByQuery(SearchEngineClient searchEngineClient, org.opensearch.index.query.QueryBuilder queryBuilder, String[] fields)
      Retrieves a list of documents that match the specified query. This method handles large result sets by using scroll search when necessary and enforces maximum document size limits.
      Parameters:
      searchEngineClient - the search engine client to use for retrieval
      queryBuilder - the query to match documents
      fields - the fields to include in the response (null for all fields)
      Returns:
      a list of documents that match the query

    • getDocumentSizeByQuery

      protected long getDocumentSizeByQuery(SearchEngineClient searchEngineClient, org.opensearch.index.query.QueryBuilder queryBuilder, org.codelibs.fess.mylasta.direction.FessConfig fessConfig)
      Gets the total number of documents that match the specified query.
      Parameters:
      searchEngineClient - the search engine client to use for the count
      queryBuilder - the query to count documents for
      fessConfig - the Fess configuration
      Returns:
      the number of documents that match the query

    • deleteBySessionId

      public long deleteBySessionId(String sessionId)
      Deletes all documents associated with the specified session ID.
      Parameters:
      sessionId - the session ID to delete documents for
      Returns:
      the number of documents that were deleted

    • deleteBySessionId

      public long deleteBySessionId(SearchEngineClient searchEngineClient, String index, String sessionId)
      Deletes all documents associated with the specified session ID from the given index.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      index - the index name to delete documents from
      sessionId - the session ID to delete documents for
      Returns:
      the number of documents that were deleted

    • deleteByConfigId

      public long deleteByConfigId(String configId)
      Deletes all documents associated with the specified configuration ID.
      Parameters:
      configId - the configuration ID to delete documents for
      Returns:
      the number of documents that were deleted

    • deleteByConfigId

      public long deleteByConfigId(SearchEngineClient searchEngineClient, String index, String configId)
      Deletes all documents associated with the specified configuration ID from the given index.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      index - the index name to delete documents from
      configId - the configuration ID to delete documents for
      Returns:
      the number of documents that were deleted

    • deleteByVirtualHost

      public long deleteByVirtualHost(String virtualHost)
      Deletes all documents associated with the specified virtual host.
      Parameters:
      virtualHost - the virtual host to delete documents for
      Returns:
      the number of documents that were deleted

    • deleteByVirtualHost

      public long deleteByVirtualHost(SearchEngineClient searchEngineClient, String index, String virtualHost)
      Deletes all documents associated with the specified virtual host from the given index.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      index - the index name to delete documents from
      virtualHost - the virtual host to delete documents for
      Returns:
      the number of documents that were deleted

    • deleteByQueryBuilder

      protected long deleteByQueryBuilder(SearchEngineClient searchEngineClient, String index, org.opensearch.index.query.QueryBuilder queryBuilder)
      Deletes documents using the specified query builder and refreshes the index. This method first refreshes the index, then performs the deletion, and logs the result.
      Parameters:
      searchEngineClient - the search engine client to use for deletion
      index - the index name to delete documents from
      queryBuilder - the query to match documents for deletion
      Returns:
      the number of documents that were deleted

    • refreshIndex

      protected int refreshIndex(SearchEngineClient searchEngineClient, String index)
      Refreshes the specified index to make recent changes visible for search. This operation ensures that all pending writes are persisted and searchable.
      Parameters:
      searchEngineClient - the search engine client to use for refresh
      index - the index name to refresh
      Returns:
      the refresh status code

    • calculateDocumentSize

      public long calculateDocumentSize(Map<String,Object> dataMap)
      Calculates the memory size of a document data map. This is useful for monitoring memory usage during indexing operations.
      Parameters:
      dataMap - the document data as a map of field names to values
      Returns:
      the estimated memory size in bytes

    • setMaxRetryCount

      public void setMaxRetryCount(int maxRetryCount)
      Sets the maximum number of retry attempts for failed operations.
      Parameters:
      maxRetryCount - the maximum retry count

    • setDefaultRowSize

      public void setDefaultRowSize(int defaultRowSize)
      Sets the default number of rows to process in a single batch.
      Parameters:
      defaultRowSize - the default row size

    • setRequestInterval

      public void setRequestInterval(long requestInterval)
      Sets the interval between requests in milliseconds.
      Parameters:
      requestInterval - the request interval in milliseconds