Understanding Solr configuration in Hybris


In Hybris, we have to configure the Solr in impex files

Lets see the impex files used for the same in detail

1)solr.impex
/import/coredata/stores/hybris/solr.impex
2)solr_en.impex
/import/coredata/stores/hybris/solr_en.impex
3)solrtrigger.impex
import/coredata/stores/hybris/solrtrigger.impex

1)solr.impex

This file contains all the configuration of Solr including Solr server,indexing types,facets etc.
Note: All the variables in the below impex files which starts with $ are called as macros and should be defined at the beginning of the impex.

Solr server configuration

We need to insert the data to SolrServerConfig table with solr server configuration name and mode of the server to indicate whether it’s embedded or standalone server.
Below impex line will do this

1
2
INSERT_UPDATE SolrServerConfig;name[unique=true];mode(code);embeddedMaster
;$serverConfigName;embedded;true
INSERT_UPDATE SolrServerConfig;name[unique=true];mode(code);embeddedMaster
;$serverConfigName;embedded;true

We have configured solr server to be embedded.

If we want to make solr server as standalone then we have to insert below data to SolrServerConfig table

1
2
INSERT_UPDATE SolrServerConfig;name[unique=true];mode(code);embeddedMaster
;$serverConfigName;standalone;false;
INSERT_UPDATE SolrServerConfig;name[unique=true];mode(code);embeddedMaster
;$serverConfigName;standalone;false;

For standalone solr server configuration , we must insert data to SolrEndpointUrl table as well to indicate the actual url where solr is running.

1
2
INSERT_UPDATE SolrEndpointUrl;solrServerConfig(name)[unique=true];url[unique=true];master[unique=true,default=false]
;$serverConfigName;http://localhost:8983/solr;true
INSERT_UPDATE SolrEndpointUrl;solrServerConfig(name)[unique=true];url[unique=true];master[unique=true,default=false]
;$serverConfigName;http://localhost:8983/solr;true


solr indexer configuration

we need to insert the data to SolrIndexConfig table with index configuration name,batch size of indexing,number of threads to run and the index mode

1
2
INSERT_UPDATE SolrIndexConfig;name[unique=true];batchSize;numberOfThreads;indexMode(code);
;$indexConfigName;100;1;TWO_PHASE;
INSERT_UPDATE SolrIndexConfig;name[unique=true];batchSize;numberOfThreads;indexMode(code);
;$indexConfigName;100;1;TWO_PHASE;

We have defined the batch size to be 100
1 thread for processing index and
TWO_PHASE as the indexing strategy.

Check Solr overview in Hybris article for the details of TWO_PHASE indexing mode.

Solr search configuration

we need to insert the data to SolrSearchConfig table with page size and description

1
2
INSERT_UPDATE SolrSearchConfig;description[unique=true];pageSize
;$searchConfigName;20
INSERT_UPDATE SolrSearchConfig;description[unique=true];pageSize
;$searchConfigName;20

We have given the page size as 20, so 20 pages of results will be retrieved by Solr.

Define the Solr indexed Types

We need to define the hybris item types to be indexed in the SolrIndexedType table

1
2
INSERT_UPDATE SolrIndexedType;identifier[unique=true];type(code);variant;sorts(&sortRefID)
;$solrIndexedType;Product;false;sortRef1,sortRef2,sortRef3,sortRef4,sortRef5,sortRef6
INSERT_UPDATE SolrIndexedType;identifier[unique=true];type(code);variant;sorts(&sortRefID)
;$solrIndexedType;Product;false;sortRef1,sortRef2,sortRef3,sortRef4,sortRef5,sortRef6

We have defined only one hybris item type to be indexed which is Product Type and defined some of the sorting references which we will define in the subsequent impex below.

Solr Facet Search Config

We need to insert the data to SolrFacetSearchConfig table to define index name prefix,supporting languages,currencies and also link the solrServerConfig, solrSearchConfig, solrIndexConfig, solrIndexedTypes defined above.

We also need to define the fallback language and catalog version

1
2
INSERT_UPDATE SolrFacetSearchConfig;name[unique=true];description;indexNamePrefix;languages(isocode);currencies(isocode);solrServerConfig(name);solrSearchConfig(description);solrIndexConfig(name);solrIndexedTypes(identifier);enabledLanguageFallbackMechanism;$catalogVersions
;$facetSearchConfigName;$facetSearchConfigDescription;$searchIndexNamePrefix;$indexLanguages;$indexCurrencies;$serverConfigName;$searchConfigName;$indexConfigName;$solrIndexedType;true;$productCatalog:Online,$productCatalog:Online
INSERT_UPDATE SolrFacetSearchConfig;name[unique=true];description;indexNamePrefix;languages(isocode);currencies(isocode);solrServerConfig(name);solrSearchConfig(description);solrIndexConfig(name);solrIndexedTypes(identifier);enabledLanguageFallbackMechanism;$catalogVersions
;$facetSearchConfigName;$facetSearchConfigDescription;$searchIndexNamePrefix;$indexLanguages;$indexCurrencies;$serverConfigName;$searchConfigName;$indexConfigName;$solrIndexedType;true;$productCatalog:Online,$productCatalog:Online


Link the solr to the site

We need to link the solr configuration to the site by providing solrFacetSearchConfiguration to the Base site as below

1
2
UPDATE BaseSite;uid[unique=true];solrFacetSearchConfiguration(name)
;$indexBaseSite;$facetSearchConfigName
UPDATE BaseSite;uid[unique=true];solrFacetSearchConfiguration(name)
;$indexBaseSite;$facetSearchConfigName


Defining the range value set

We need to insert the data to SolrValueRangeSet table for defining the Solr value range references which we will define in the next impex.

1
2
INSERT_UPDATE SolrValueRangeSet;name[unique=true];qualifier;type;solrValueRanges(&rangeValueRefID)
;priceRangeGBP;GBP;double;rangeRefGBP1,rangeRefGBP2,rangeRefGBP3,rangeRefGBP4,rangeRefGBP5
INSERT_UPDATE SolrValueRangeSet;name[unique=true];qualifier;type;solrValueRanges(&rangeValueRefID)
;priceRangeGBP;GBP;double;rangeRefGBP1,rangeRefGBP2,rangeRefGBP3,rangeRefGBP4,rangeRefGBP5

We have defined 5 ranges for priceRangeGBP and we will define the values for the same in the below impex

We need to insert the data for SolrValueRange table to define the actual range values for each range reference as below

1
2
3
4
5
6
INSERT_UPDATE SolrValueRange;&rangeValueRefID;solrValueRangeSet(name)[unique=true];name[unique=true];from;to
;rangeRefGBP1 ;priceRangeGBP;£0-£19.99;   0; 19.99
;rangeRefGBP2 ;priceRangeGBP;£20-£49.99;  20; 49.99
;rangeRefGBP3 ;priceRangeGBP;£50-£99.99;  50; 99.99
;rangeRefGBP4 ;priceRangeGBP;£100-£199.99; 100;199.99
;rangeRefGBP5 ;priceRangeGBP;£200-£299.99; 200;299.99
INSERT_UPDATE SolrValueRange;&rangeValueRefID;solrValueRangeSet(name)[unique=true];name[unique=true];from;to
;rangeRefGBP1 ;priceRangeGBP;£0-£19.99;   0; 19.99
;rangeRefGBP2 ;priceRangeGBP;£20-£49.99;  20; 49.99
;rangeRefGBP3 ;priceRangeGBP;£50-£99.99;  50; 99.99
;rangeRefGBP4 ;priceRangeGBP;£100-£199.99; 100;199.99
;rangeRefGBP5 ;priceRangeGBP;£200-£299.99; 200;299.99

we will use priceRangeGBP to provide range for the indexed property while defining indexed property.

Defining the non-facet indexed properties

We need to insert the data into SolrIndexedProperty table for defining the properties of item types to be indexed.

We need to define different features for each property on how it should be indexed, whether it has to be multivalued,used for autocomplete etc.

1
2
3
INSERT_UPDATE SolrIndexedProperty;solrIndexedType(identifier)[unique=true];name[unique=true];type(code);sortableType(code);currency[default=false];localized[default=false];multiValue[default=false];useForSpellchecking[default=false];useForAutocomplete[default=false];fieldValueProvider;valueProviderParameter
;$solrIndexedType; name                   ;text   ;sortabletext;    ;true;    ;true;true;
;$solrIndexedType; priceValue             ;double ;            ;true;    ;    ;    ;    ;productPriceValueProvider;
INSERT_UPDATE SolrIndexedProperty;solrIndexedType(identifier)[unique=true];name[unique=true];type(code);sortableType(code);currency[default=false];localized[default=false];multiValue[default=false];useForSpellchecking[default=false];useForAutocomplete[default=false];fieldValueProvider;valueProviderParameter
;$solrIndexedType; name                   ;text   ;sortabletext;    ;true;    ;true;true;
;$solrIndexedType; priceValue             ;double ;            ;true;    ;    ;    ;    ;productPriceValueProvider;

We also need to define the Value provider for some of the attributes whose value cannot be understood by Solr directly.
We will see how value providers works in a separate article.

Defining the Facet indexed properties

Facet properties are generally displayed at the sidebar of the site where we can select those facet fields to get the list of products accordingly.

1
2
INSERT_UPDATE SolrIndexedProperty;solrIndexedType(identifier)[unique=true];name[unique=true];type(code);sortableType(code);currency[default=false];localized[default=false];multiValue[default=false];facet[default=true];facetType(code);facetSort(code);priority;visible;useForSpellchecking[default=false];useForAutocomplete[default=false];fieldValueProvider;facetDisplayNameProvider;customFacetSortProvider;topValuesProvider;rangeSets(name)
;$solrIndexedType; price            ;double ; ;true ;    ;    ; ;MultiSelectOr ;Alpha ; 4000;true;    ;    ;productPriceValueProvider              ;                                             ;                             ;defaultTopValuesProvider ;priceRangeGBP
INSERT_UPDATE SolrIndexedProperty;solrIndexedType(identifier)[unique=true];name[unique=true];type(code);sortableType(code);currency[default=false];localized[default=false];multiValue[default=false];facet[default=true];facetType(code);facetSort(code);priority;visible;useForSpellchecking[default=false];useForAutocomplete[default=false];fieldValueProvider;facetDisplayNameProvider;customFacetSortProvider;topValuesProvider;rangeSets(name)
;$solrIndexedType; price            ;double ; ;true ;    ;    ; ;MultiSelectOr ;Alpha ; 4000;true;    ;    ;productPriceValueProvider              ;                                             ;                             ;defaultTopValuesProvider ;priceRangeGBP

We have defined price field as facet field and it can be selected to filter the products based on that field.

we have defined facet[default=true] to indcate that the field is facet type

Define the indexed queries

We need to define the indexed queries for full index and update index so that solr picks up the corresponding source data from hybris for indexing.

1
2
3
INSERT_UPDATE SolrIndexerQuery;solrIndexedType(identifier)[unique=true];identifier[unique=true];type(code);injectCurrentDate[default=true];injectCurrentTime[default=true];injectLastIndexTime[default=true];query;user(uid)
;$solrIndexedType;$solrIndexedType-fullQuery;full;;;false;"SELECT {PK} FROM {Product}";anonymous
;$solrIndexedType;$solrIndexedType-updateQuery;update;;;;"SELECT {p:PK} FROM {Product AS p} WHERE ({p:modifiedtime} >= ?lastIndexTime";anonymous
INSERT_UPDATE SolrIndexerQuery;solrIndexedType(identifier)[unique=true];identifier[unique=true];type(code);injectCurrentDate[default=true];injectCurrentTime[default=true];injectLastIndexTime[default=true];query;user(uid)
;$solrIndexedType;$solrIndexedType-fullQuery;full;;;false;"SELECT {PK} FROM {Product}";anonymous
;$solrIndexedType;$solrIndexedType-updateQuery;update;;;;"SELECT {p:PK} FROM {Product AS p} WHERE ({p:modifiedtime} >= ?lastIndexTime";anonymous

We have defined the 2 queries that Solr uses for full indexing and update indexing.

Full indexing query gets all the data without any condition

Update indexing query gets the data which has been modified from the last indexed time, so it won’t fetch entire data for indexing and hence it’s faster too.

Defining the Sort

We need to insert the sort references into SolrSort table which we will be using while inserting values for SolrIndexedType to define the sorts for the indexed item type

1
2
3
4
5
6
7
INSERT_UPDATE SolrSort;&sortRefID;indexedType(identifier)[unique=true];code[unique=true];useBoost
;sortRef1;$solrIndexedType;relevance;true
;sortRef2;$solrIndexedType;topRated;false
;sortRef3;$solrIndexedType;name-asc;false
;sortRef4;$solrIndexedType;name-desc;false
;sortRef5;$solrIndexedType;price-asc;false
;sortRef6;$solrIndexedType;price-desc;false
INSERT_UPDATE SolrSort;&sortRefID;indexedType(identifier)[unique=true];code[unique=true];useBoost
;sortRef1;$solrIndexedType;relevance;true
;sortRef2;$solrIndexedType;topRated;false
;sortRef3;$solrIndexedType;name-asc;false
;sortRef4;$solrIndexedType;name-desc;false
;sortRef5;$solrIndexedType;price-asc;false
;sortRef6;$solrIndexedType;price-desc;false

we have given these references while inserting data into SolrIndexedType table above.

Define the Solr sort fields

We need to insert the sorting fields into SolrSortField table which will be linked to the sort reference codes defined above

1
2
3
4
5
6
7
8
9
INSERT_UPDATE SolrSortField;sort(indexedType(identifier),code)[unique=true];fieldName[unique=true];ascending[unique=true]
;$solrIndexedType:relevance;inStockFlag;false
;$solrIndexedType:relevance;score;false
;$solrIndexedType:topRated;inStockFlag;false
;$solrIndexedType:topRated;reviewAvgRating;false
;$solrIndexedType:name-asc;name;true
;$solrIndexedType:name-desc;name;false
;$solrIndexedType:price-asc;priceValue;true
;$solrIndexedType:price-desc;priceValue;false
INSERT_UPDATE SolrSortField;sort(indexedType(identifier),code)[unique=true];fieldName[unique=true];ascending[unique=true]
;$solrIndexedType:relevance;inStockFlag;false
;$solrIndexedType:relevance;score;false
;$solrIndexedType:topRated;inStockFlag;false
;$solrIndexedType:topRated;reviewAvgRating;false
;$solrIndexedType:name-asc;name;true
;$solrIndexedType:name-desc;name;false
;$solrIndexedType:price-asc;priceValue;true
;$solrIndexedType:price-desc;priceValue;false



We have defined 5 sort fields as below

inStockFlag field can be sorted based on relevance and Top rating in descending order.

score field can be sorted based on relevance only in descending order

reviewAvgRating field can be sorted based on Top rating only in descending order

name field can be sorted based on both ascending & descending order

price field can be sorted based on both ascending & descending order


2)solr_en.impex


import/coredata/stores/apparel-uk/solr_en.impex

This impex file is used for localization of solr indexed fields in English language as below

1
2
3
4
UPDATE SolrIndexedProperty;solrIndexedType(identifier)[unique=true];name[unique=true];displayName[lang=$lang]
 ;$solrIndexedType;allPromotions;"Promotions"
UPDATE SolrSort;indexedType(identifier)[unique=true];code[unique=true];name[lang=$lang]
 ;$solrIndexedType;name-asc;"Name (ascending)"
UPDATE SolrIndexedProperty;solrIndexedType(identifier)[unique=true];name[unique=true];displayName[lang=$lang]
 ;$solrIndexedType;allPromotions;"Promotions"
UPDATE SolrSort;indexedType(identifier)[unique=true];code[unique=true];name[lang=$lang]
 ;$solrIndexedType;name-asc;"Name (ascending)"

Similarly for other languages we will have corresponding solr localization files in the appropriate localization folders.


3)solrtrigger.impex


/import/coredata/stores/apparel-uk/solrtrigger.impex

This impex file is used to schedule the cron jobs for Solr indexing.

1
2
3
4
5
6
INSERT_UPDATE Trigger;
cronJob(code)[unique=true];second;minute;hour;day;month;year;relative;active;maxAcceptableDelay
# Run the full index cronJob at 3:05 AM every day
;full-index-cronJob;0;5;3;-1;-1;-1;false;false;-1
# Run the update index cronJob every 1 minutes
;update-index-cronJob;0;1;-1;-1;-1;-1;true;false;-1
INSERT_UPDATE Trigger;
cronJob(code)[unique=true];second;minute;hour;day;month;year;relative;active;maxAcceptableDelay
# Run the full index cronJob at 3:05 AM every day
;full-index-cronJob;0;5;3;-1;-1;-1;false;false;-1
# Run the update index cronJob every 1 minutes
;update-index-cronJob;0;1;-1;-1;-1;-1;true;false;-1

We have defined cron job scheduling time through Trigger table for both full and update indexing.

We can observe that full indexing job is scheduled only once a day and update indexing is scheduled to run every 1 minute.

we already know the reason why update index runs more frequently, if not please check Solr overview in Hybris article

For practicing in B2C Apparel store, check the below impex file
hybris\bin\ext-data\apparelstore\resources\apparelstore\import\coredata\stores\apparel-uk\solr.impex

About the Author

Karibasappa G C (KB)
Founder of javainsimpleway.com
I love Java and open source technologies and very much passionate about software development.
I like to share my knowledge with others especially on technology 🙂
I have given all the examples as simple as possible to understand for the beginners.
All the code posted on my blog is developed,compiled and tested in my development environment.
If you find any mistakes or bugs, Please drop an email to kb.knowledge.sharing@gmail.com

Connect with me on Facebook for more updates

Share this article on