Package org.codelibs.fess.llm

Class AbstractLlmClient

java.lang.Object
org.codelibs.fess.llm.AbstractLlmClient
All Implemented Interfaces:
LlmClient
Direct Known Subclasses:
GeminiLlmClient, OllamaLlmClient, OpenAiLlmClient
public abstract class AbstractLlmClient extends Object implements LlmClient
Abstract base class for LLM client implementations. Provides shared infrastructure (HTTP client, availability checking) and default implementations of RAG workflow methods with injectable prompt templates. Subclasses implement provider-specific chat/streamChat and checkAvailabilityNow.

  • Field Details

    • objectMapper

      protected static final com.fasterxml.jackson.databind.ObjectMapper objectMapper
      Shared ObjectMapper instance for JSON processing.

    • httpClient

      protected org.apache.hc.client5.http.impl.classic.CloseableHttpClient httpClient
      The HTTP client used for API communication.

    • cachedAvailability

      protected volatile Boolean cachedAvailability
      Cached availability status of the LLM provider.

    • availabilityCheckTask

      protected org.codelibs.core.timer.TimeoutTask availabilityCheckTask
      The scheduled task for periodic availability checks.

    • intentDetectionPrompt

      protected String intentDetectionPrompt
      Custom prompt template for intent detection. Use ${userMessage} as placeholder.

    • unclearIntentSystemPrompt

      protected String unclearIntentSystemPrompt
      Custom system prompt for unclear intent responses.

    • noResultsSystemPrompt

      protected String noResultsSystemPrompt
      Custom system prompt for no-results responses.

    • documentNotFoundSystemPrompt

      protected String documentNotFoundSystemPrompt
      Custom system prompt for document-not-found responses. Use ${documentUrl} as placeholder.

    • summarySystemPromptSuffix

      protected String summarySystemPromptSuffix
      Custom system prompt suffix for summary generation.

    • evaluationPrompt

      protected String evaluationPrompt
      Custom prompt template for relevance evaluation. Uses {{maxRelevantDocs}}, {{userMessage}}, {{query}}, {{searchResults}} placeholders.

    • answerGenerationSystemPrompt

      protected String answerGenerationSystemPrompt
      Custom system prompt template for answer generation. Uses {{systemPrompt}}, {{context}} placeholders.

    • summarySystemPrompt

      protected String summarySystemPrompt
      Custom system prompt template for summary generation. Uses {{systemPrompt}}, {{documentContent}} placeholders.

    • faqAnswerSystemPrompt

      protected String faqAnswerSystemPrompt
      Custom system prompt template for FAQ answer generation. Uses {{systemPrompt}}, {{context}} placeholders.

    • directAnswerSystemPrompt

      protected String directAnswerSystemPrompt
      Custom system prompt for direct answer generation (without document search).

  • Constructor Details

    • AbstractLlmClient

      public AbstractLlmClient()
      Default constructor.

  • Method Details

    • init

      public void init()
      Initializes the HTTP client and starts availability checking. Should be called from subclass init() methods.

    • destroy

      public void destroy()
      Cleans up resources.

    • startAvailabilityCheck

      protected void startAvailabilityCheck()
      Starts periodic availability checking if RAG chat is enabled.

    • updateAvailability

      protected void updateAvailability()
      Updates the cached availability state.

    • isAvailable

      public boolean isAvailable()
      Description copied from interface: LlmClient
      Checks if this LLM client is available and properly configured.
      Specified by:
      isAvailable in interface LlmClient
      Returns:
      true if the client is available, false otherwise

    • getHttpClient

      public org.apache.hc.client5.http.impl.classic.CloseableHttpClient getHttpClient()
      Gets the HTTP client, initializing it if necessary.
      Returns:
      the HTTP client

    • checkAvailabilityNow

      protected abstract boolean checkAvailabilityNow()
      Performs the actual availability check against the LLM provider.
      Returns:
      true if the provider is available

    • getTimeout

      protected abstract int getTimeout()
      Gets the request timeout in milliseconds.
      Returns:
      the timeout in milliseconds

    • getModel

      protected abstract String getModel()
      Gets the model name.
      Returns:
      the model name

    • getAvailabilityCheckInterval

      protected int getAvailabilityCheckInterval()
      Gets the availability check interval in seconds.
      Returns:
      the interval in seconds

    • isRagChatEnabled

      protected boolean isRagChatEnabled()
      Checks if RAG chat feature is enabled.
      Returns:
      true if RAG chat is enabled

    • getLlmType

      protected String getLlmType()
      Gets the configured LLM type.
      Returns:
      the LLM type from configuration

    • getTemperature

      protected double getTemperature()
      Gets the temperature parameter.
      Returns:
      the temperature

    • getMaxTokens

      protected int getMaxTokens()
      Gets the maximum tokens for the response.
      Returns:
      the maximum tokens

    • getIntentDetectionMaxTokens

      protected int getIntentDetectionMaxTokens()
      Gets the maximum tokens for intent detection.
      Returns:
      the maximum tokens for intent detection

    • getEvaluationMaxTokens

      protected int getEvaluationMaxTokens()
      Gets the maximum tokens for result evaluation.
      Returns:
      the maximum tokens for result evaluation

    • setIntentDetectionPrompt

      public void setIntentDetectionPrompt(String intentDetectionPrompt)
      Sets the custom prompt template for intent detection.
      Parameters:
      intentDetectionPrompt - the prompt template with {{userMessage}} placeholder

    • setUnclearIntentSystemPrompt

      public void setUnclearIntentSystemPrompt(String unclearIntentSystemPrompt)
      Sets the custom system prompt for unclear intent responses.
      Parameters:
      unclearIntentSystemPrompt - the system prompt

    • setNoResultsSystemPrompt

      public void setNoResultsSystemPrompt(String noResultsSystemPrompt)
      Sets the custom system prompt for no-results responses.
      Parameters:
      noResultsSystemPrompt - the system prompt

    • setDocumentNotFoundSystemPrompt

      public void setDocumentNotFoundSystemPrompt(String documentNotFoundSystemPrompt)
      Sets the custom system prompt for document-not-found responses.
      Parameters:
      documentNotFoundSystemPrompt - the system prompt with {{documentUrl}} placeholder

    • setSummarySystemPromptSuffix

      public void setSummarySystemPromptSuffix(String summarySystemPromptSuffix)
      Sets the custom system prompt suffix for summary generation.
      Parameters:
      summarySystemPromptSuffix - the prompt suffix

    • setEvaluationPrompt

      public void setEvaluationPrompt(String evaluationPrompt)
      Sets the custom evaluation prompt for relevance checking.
      Parameters:
      evaluationPrompt - the evaluation prompt template

    • setAnswerGenerationSystemPrompt

      public void setAnswerGenerationSystemPrompt(String answerGenerationSystemPrompt)
      Sets the custom system prompt for answer generation.
      Parameters:
      answerGenerationSystemPrompt - the answer generation system prompt template

    • setSummarySystemPrompt

      public void setSummarySystemPrompt(String summarySystemPrompt)
      Sets the custom system prompt for summary generation.
      Parameters:
      summarySystemPrompt - the summary system prompt template

    • setFaqAnswerSystemPrompt

      public void setFaqAnswerSystemPrompt(String faqAnswerSystemPrompt)
      Sets the custom system prompt for FAQ answer generation.
      Parameters:
      faqAnswerSystemPrompt - the FAQ answer system prompt template

    • setDirectAnswerSystemPrompt

      public void setDirectAnswerSystemPrompt(String directAnswerSystemPrompt)
      Sets the custom system prompt for direct answer generation.
      Parameters:
      directAnswerSystemPrompt - the direct answer system prompt

    • getUserLocale

      protected Locale getUserLocale()
      Gets the user's locale from the current request context.
      Returns:
      the user's locale, or default locale if not in request context

    • getLanguageInstruction

      protected String getLanguageInstruction()
      Gets the language instruction based on the user's locale.
      Returns:
      the language instruction string, or empty string if locale is English

    • resolveLanguageInstruction

      protected String resolveLanguageInstruction(String prompt)
      Resolves the {{languageInstruction}} placeholder in a prompt.
      Parameters:
      prompt - the prompt template
      Returns:
      the prompt with language instruction resolved

    • detectIntent

      public IntentDetectionResult detectIntent(String userMessage)
      Description copied from interface: LlmClient
      Detects the intent of a user message.
      Specified by:
      detectIntent in interface LlmClient
      Parameters:
      userMessage - the user's message
      Returns:
      the detected intent with extracted keywords

    • evaluateResults

      public RelevanceEvaluationResult evaluateResults(String userMessage, String query, List<Map<String,Object>> searchResults)
      Description copied from interface: LlmClient
      Evaluates search results for relevance to the user's question.
      Specified by:
      evaluateResults in interface LlmClient
      Parameters:
      userMessage - the original user message
      query - the search query used
      searchResults - the search results to evaluate
      Returns:
      evaluation result with relevant document IDs

    • generateAnswer

      public LlmChatResponse generateAnswer(String userMessage, List<Map<String,Object>> documents, List<LlmMessage> history)
      Description copied from interface: LlmClient
      Generates an answer using document content (synchronous version for non-enhanced flow).
      Specified by:
      generateAnswer in interface LlmClient
      Parameters:
      userMessage - the user's message
      documents - the documents with content
      history - the conversation history
      Returns:
      the chat response

    • streamGenerateAnswer

      public void streamGenerateAnswer(String userMessage, List<Map<String,Object>> documents, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates an answer using document content (streaming version for enhanced flow).
      Specified by:
      streamGenerateAnswer in interface LlmClient
      Parameters:
      userMessage - the user's message
      documents - the documents with content
      history - the conversation history
      callback - the streaming callback

    • generateUnclearIntentResponse

      public void generateUnclearIntentResponse(String userMessage, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates a response asking user for clarification when intent is unclear.
      Specified by:
      generateUnclearIntentResponse in interface LlmClient
      Parameters:
      userMessage - the user's message
      history - the conversation history
      callback - the streaming callback

    • generateNoResultsResponse

      public void generateNoResultsResponse(String userMessage, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates a response when no relevant documents are found.
      Specified by:
      generateNoResultsResponse in interface LlmClient
      Parameters:
      userMessage - the user's message
      history - the conversation history
      callback - the streaming callback

    • generateDocumentNotFoundResponse

      public void generateDocumentNotFoundResponse(String userMessage, String documentUrl, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates a response when the specified document URL is not found.
      Specified by:
      generateDocumentNotFoundResponse in interface LlmClient
      Parameters:
      userMessage - the user's message
      documentUrl - the URL that was not found
      history - the conversation history
      callback - the streaming callback

    • generateSummaryResponse

      public void generateSummaryResponse(String userMessage, List<Map<String,Object>> documents, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates a summary of the specified documents.
      Specified by:
      generateSummaryResponse in interface LlmClient
      Parameters:
      userMessage - the user's message
      documents - the documents to summarize
      history - the conversation history
      callback - the streaming callback

    • generateFaqAnswerResponse

      public void generateFaqAnswerResponse(String userMessage, List<Map<String,Object>> documents, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates an FAQ answer using document content (streaming). Uses a prompt optimized for direct, concise FAQ-style answers.
      Specified by:
      generateFaqAnswerResponse in interface LlmClient
      Parameters:
      userMessage - the user's message
      documents - the documents with content
      history - the conversation history
      callback - the streaming callback

    • generateDirectAnswer

      public void generateDirectAnswer(String userMessage, List<LlmMessage> history, LlmStreamCallback callback)
      Description copied from interface: LlmClient
      Generates a direct answer without document search.
      Specified by:
      generateDirectAnswer in interface LlmClient
      Parameters:
      userMessage - the user's message
      history - the conversation history
      callback - the streaming callback

    • buildIntentDetectionPrompt

      protected String buildIntentDetectionPrompt(String userMessage)
      Builds the intent detection prompt.
      Parameters:
      userMessage - the user's message
      Returns:
      the constructed prompt string

    • buildEvaluationPrompt

      protected String buildEvaluationPrompt(String userMessage, String query, List<Map<String,Object>> searchResults)
      Builds the evaluation prompt for relevance checking.
      Parameters:
      userMessage - the user's message
      query - the search query
      searchResults - the search results to evaluate
      Returns:
      the evaluation prompt

    • buildContext

      protected String buildContext(List<Map<String,Object>> documents, org.codelibs.fess.mylasta.direction.FessConfig fessConfig)
      Builds context from document content for the LLM prompt.
      Parameters:
      documents - the search result documents
      fessConfig - the Fess configuration
      Returns:
      the context string

    • buildStreamingRequest

      protected LlmChatRequest buildStreamingRequest(org.codelibs.fess.mylasta.direction.FessConfig fessConfig, String userMessage, String context, List<LlmMessage> history)
      Builds a streaming LLM chat request with conversation history.
      Parameters:
      fessConfig - the Fess configuration
      userMessage - the user's message
      context - the context from search results
      history - the conversation history
      Returns:
      the LLM chat request

    • parseIntentResponse

      protected IntentDetectionResult parseIntentResponse(String response)
      Parses the LLM response and extracts intent detection result.
      Parameters:
      response - the JSON response from LLM
      Returns:
      the parsed intent detection result

    • parseEvaluationResponse

      protected RelevanceEvaluationResult parseEvaluationResponse(String response, List<Map<String,Object>> searchResults)
      Parses the evaluation response from LLM.
      Parameters:
      response - the LLM response
      searchResults - the search results
      Returns:
      the parsed evaluation result

    • stripCodeFences

      protected String stripCodeFences(String response)
      Strips code fence markers from JSON response.
      Parameters:
      response - the response that may contain code fences
      Returns:
      the response with code fences removed

    • extractJsonString

      protected String extractJsonString(String json, String key)
      Extracts a string value from JSON response using Jackson parser.
      Parameters:
      json - the JSON response
      key - the key to extract
      Returns:
      the extracted string value

    • extractJsonStringFallback

      protected String extractJsonStringFallback(String json, String key)
      Fallback regex-based extraction for string values.
      Parameters:
      json - the JSON response
      key - the key to extract
      Returns:
      the extracted string value

    • extractJsonBoolean

      protected boolean extractJsonBoolean(String json, String key)
      Extracts a boolean value from JSON response.
      Parameters:
      json - the JSON response
      key - the key to extract
      Returns:
      the extracted boolean value

    • extractJsonArray

      protected List<String> extractJsonArray(String json, String key)
      Extracts a string array from JSON response.
      Parameters:
      json - the JSON response
      key - the key to extract
      Returns:
      the extracted string array

    • extractJsonIntArray

      protected List<Integer> extractJsonIntArray(String json, String key)
      Extracts an integer array from JSON response.
      Parameters:
      json - the JSON response
      key - the key to extract
      Returns:
      the extracted integer array

    • getStringValue

      protected String getStringValue(Map<String,Object> map, String key)
      Gets a string value from a map.
      Parameters:
      map - the map to get the value from
      key - the key to look up
      Returns:
      the string value, or an empty string if not found

    • addHistory

      protected void addHistory(LlmChatRequest request, List<LlmMessage> history)
      Adds conversation history to the request.
      Parameters:
      request - the LLM chat request
      history - the conversation history