Salesforce Connector

Overview

The Salesforce Connector provides functionality to retrieve data from Salesforce objects (standard objects, custom objects) and register them in the Fess index.

This feature requires the fess-ds-salesforce plugin.

Supported Objects

Standard Objects: Account, Contact, Lead, Opportunity, Case, Solution, etc.
Custom Objects: User-created custom objects
Knowledge Articles: Salesforce Knowledge

Prerequisites

Plugin installation is required
A Salesforce Connected App must be created
OAuth authentication must be configured
Read access to objects is required

Plugin Installation

Install from the admin console under “System” -> “Plugins”.

Or, see Plugin for details.

Configuration

Configure in the admin console under “Crawler” -> “Data Store” -> “Create New”.

Basic Settings

Item	Example
Name	Salesforce CRM
Handler Name	SalesforceDataStore
Enabled	On

Parameter Configuration

OAuth Token authentication (recommended):

base_url=https://login.salesforce.com
auth_type=oauth_token
username=admin@example.com
client_id=3MVG9...
private_key=-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQE...\n-----END PRIVATE KEY-----\n
number_of_threads=1
ignoreError=true
custom=FessObj,CustomProduct
FessObj.title=Name
FessObj.contents=Name,Description__c
CustomProduct.title=Product_Name__c
CustomProduct.contents=Product_Name__c,Product_Description__c

OAuth Password authentication:

base_url=https://login.salesforce.com
auth_type=oauth_password
username=admin@example.com
client_id=3MVG9...
client_secret=1234567890ABCDEF
security_token=AbCdEfGhIjKlMnOpQrSt
number_of_threads=1
ignoreError=true

Parameter List

Parameter	Required	Description
`base_url`	Yes	Salesforce URL (Production: `https://login.salesforce.com`, Sandbox: `https://test.salesforce.com`)
`auth_type`	Yes	Authentication type (`oauth_token` or `oauth_password`)
`username`	Yes	Salesforce username
`client_id`	Yes	Connected App Consumer Key
`private_key`	For oauth_token	Private key (PEM format, newlines as `\n`)
`client_secret`	For oauth_password	Connected App Consumer Secret
`security_token`	For oauth_password	User’s security token
`number_of_threads`	No	Number of parallel processing threads (default: 1)
`ignoreError`	No	Continue processing on error (default: true)
`custom`	No	Custom object names (comma-separated)
`<object>.title`	No	Field name to use for title
`<object>.contents`	No	Field names to use for content (comma-separated)

Script Configuration

title="[" + object.type + "] " + object.title
digest=object.description
content=object.content
created=object.created
timestamp=object.last_modified
url=object.url

Available Fields

Field	Description
`object.type`	Object type (e.g., Case, User, Solution)
`object.title`	Object name
`object.description`	Object description
`object.content`	Object text content
`object.id`	Object ID
`object.content_length`	Content length
`object.created`	Creation date/time
`object.last_modified`	Last modified date/time
`object.url`	Object URL
`object.thumbnail`	Thumbnail URL

Salesforce Connected App Configuration

1. Create Connected App

In Salesforce Setup:

Open “App Manager”
Click “New Connected App”
Enter basic information:
- Connected App Name: Fess Crawler
- API Name: Fess_Crawler
- Contact Email: your-email@example.com
Check “Enable API (Enable OAuth Settings)”

2. Configure OAuth Token Authentication (Recommended)

In OAuth settings:

Check “Use digital signatures”
Upload certificate (created using steps below)
Selected OAuth Scopes:
- Full access (full)
- Perform requests on your behalf at any time (refresh_token, offline_access)
Click “Save”
Copy Consumer Key

Creating the certificate:

# Generate private key
openssl genrsa -out private_key.pem 2048

# Generate certificate
openssl req -new -x509 -key private_key.pem -out certificate.crt -days 365

# Verify private key
cat private_key.pem

Upload the certificate (certificate.crt) to Salesforce and set the private key (private_key.pem) contents in the parameter.

3. Configure OAuth Password Authentication

In OAuth settings:

Callback URL: https://localhost (not used but required)
Selected OAuth Scopes:
- Full access (full)
- Perform requests on your behalf at any time (refresh_token, offline_access)
Click “Save”
Copy Consumer Key and Consumer Secret

Obtaining security token:

Open personal settings in Salesforce
Click “Reset My Security Token”
Copy the token sent via email

4. Authorize Connected App

In “Manage” -> “Manage Connected Apps”:

Select the created connected app
Click “Edit”
Change “Permitted Users” to “Admin approved users are pre-authorized”
Assign profiles or permission sets

Custom Object Configuration

Crawling Custom Objects

Specify custom object names in the custom parameter:

custom=FessObj,CustomProduct,ProjectTask

Field mapping for each object:

FessObj.title=Name
FessObj.contents=Name,Description__c,Notes__c

CustomProduct.title=Product_Name__c
CustomProduct.contents=Product_Name__c,Product_Description__c,Specifications__c

ProjectTask.title=Task_Name__c
ProjectTask.contents=Task_Name__c,Task_Description__c

Field Mapping Rules

<object_name>.title - Field to use for title (single field)
<object_name>.contents - Fields to use for content (comma-separated for multiple)

Usage Examples

Crawl Standard Objects

Parameters:

base_url=https://login.salesforce.com
auth_type=oauth_token
username=admin@example.com
client_id=3MVG9A2kN3Bn17hvOLkjEo7GFdC...
private_key=-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQE...\n-----END PRIVATE KEY-----\n
number_of_threads=1
ignoreError=true

Script:

title="[" + object.type + "] " + object.title
content=object.content
digest=object.description
created=object.created
timestamp=object.last_modified
url=object.url

Crawl Custom Objects

Parameters:

base_url=https://login.salesforce.com
auth_type=oauth_token
username=admin@example.com
client_id=3MVG9A2kN3Bn17hvOLkjEo7GFdC...
private_key=-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQE...\n-----END PRIVATE KEY-----\n
number_of_threads=2
ignoreError=true
custom=Product__c,Contract__c
Product__c.title=Name
Product__c.contents=Name,Description__c,Category__c
Contract__c.title=Contract_Name__c
Contract__c.contents=Contract_Name__c,Terms__c,Notes__c

Script:

title="[" + object.type + "] " + object.title
content=object.content
created=object.created
timestamp=object.last_modified
url=object.url

Crawl Sandbox Environment

Parameters:

base_url=https://test.salesforce.com
auth_type=oauth_password
username=admin@example.com.sandbox
client_id=3MVG9A2kN3Bn17hvOLkjEo7GFdC...
client_secret=1234567890ABCDEF1234567890ABCDEF
security_token=AbCdEfGhIjKlMnOpQrStUvWxYz
number_of_threads=1
ignoreError=true

Script:

title="[SANDBOX] [" + object.type + "] " + object.title
content=object.content
timestamp=object.last_modified
url=object.url

Troubleshooting

Authentication Error

Symptom: Authentication failed or invalid_grant

Check:

For OAuth Token authentication:
- Verify Consumer Key is correct
- Verify private key is copied correctly (newlines should be \n)
- Verify certificate is uploaded to Salesforce
- Verify username is correct
For OAuth Password authentication:
- Verify Consumer Key and Consumer Secret are correct
- Verify security token is correct
- Verify password and security token are not concatenated (set separately)
Common:
- Verify base_url is correct (production or sandbox environment)
- Verify connected app is authorized

Cannot Retrieve Objects

Symptom: Crawl succeeds but 0 objects found

Check:

Verify user has read permissions on objects
For custom objects, verify object name is correct (API Name)
Verify field mapping is correct
Check logs for error messages

Custom Object Names

Verify custom object API Name:

Open “Object Manager” in Salesforce Setup
Select the custom object
Copy the “API Name” (usually ends with __c)

Example:

Label: Product
API Name: Product__c (use this)

Verify Field Names

Verify custom field API Name:

Open “Fields & Relationships” for the object
Select the custom field
Copy the “Field Name” (usually ends with __c)

Example:

Field Label: Product Description
Field Name: Product_Description__c (use this)

API Rate Limiting

Symptom: REQUEST_LIMIT_EXCEEDED

Resolution:

Reduce number_of_threads (set to 1)
Increase crawl interval
Check Salesforce API usage
Purchase additional API limits if needed

Large Amount of Data

Symptom: Crawl takes too long or times out

Resolution:

Split objects into multiple data stores
Adjust number_of_threads (around 2-4)
Distribute crawl schedules
Map only necessary fields

Private Key Format Error

Symptom: Invalid private key format

Resolution:

Verify private key newlines are correctly formatted as \n:

# Correct format
private_key=-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQE...\n-----END PRIVATE KEY-----\n

# Incorrect format (contains actual newlines)
private_key=-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQE...
-----END PRIVATE KEY-----

Reference Information

Data Store Connector Overview - Data Store Connector Overview
Database Connector - Database Connector
Data Store Crawling - Data Store Configuration Guide
Salesforce REST API
Salesforce OAuth 2.0 JWT Bearer Flow
Salesforce Connected Apps

Menu

Overview

Getting Started

Documentation

Tutorial

Development

Others

Archives

Salesforce Connector

Overview

Supported Objects

Prerequisites

Plugin Installation

Configuration

Basic Settings

Parameter Configuration

Parameter List

Script Configuration

Available Fields

Salesforce Connected App Configuration

1. Create Connected App

2. Configure OAuth Token Authentication (Recommended)

3. Configure OAuth Password Authentication

4. Authorize Connected App

Custom Object Configuration

Crawling Custom Objects

Field Mapping Rules

Usage Examples

Crawl Standard Objects

Crawl Custom Objects

Crawl Sandbox Environment

Troubleshooting

Authentication Error

Cannot Retrieve Objects

Custom Object Names

Verify Field Names

API Rate Limiting

Large Amount of Data

Private Key Format Error

Reference Information