Populating and retrieving data from Solr

Solr is one of the integral component of WCS and it is a very frequent requirement to update and read the data in Solr. In this write up I will explain it with some examples.

Populating data into Solr

There are multiple ways we can populate solr indexes. The source can be a cvs file, JSON feed, a url (like a web service) that will return the data in the way we need. Here we will have cvs file as the input.

The headings of the input file must be same as the solr fields. Referring to my earlier post, we will use the same data. So the headings will be searchterm and recipes. The file must have data as specified in this post http://exploringwebspherecommerce.blogspot.com.au/2016/02/creating-and-populating-new-core-in.html

To populate the indexes, we can write a java command and can use the solrj libraries. The base example would be something like below. It assumes the data to be populated is properly updated in the file c:/Ranjith/searchRecepies.csv

SolrServer solrServer= new HttpSolrServer("url");//solr url needs to be passed here

NamedList<String> params = new NamedList<String>();

//Sets the file from which data need to be streamed 
updateParams.add(stream.file","c:/Ranjith/searchRecepies.csv"); 

updateParams.add("stream.contentType", "text/csv;charset=utf-8");

//set to false ignored to handle transaction rollbacks. Would need a specific commit query issued at //the end.
updateParams.add("commit", "false");
SolrQuery updateQuery = new SolrQuery();
SolrParams solrParams = updateQuery.toSolrParams(params);
updateQuery.add(solrParams);
QueryRequest solrRequest = new QueryRequest(updateQuery);
solrRequest.setPath("/update");
QueryResponse solrResponse;
solrResponse = solrRequest.process(server);

We must have a separate method which fires a query with just parameter "commit" as true and must call  it at the end so that it is committed. Similarly we can have a different method for rollback as well  so that we can rollback the changes in case of an exception or so.

Retrieving data from Solr

To retrieve data from solr we can use the below sample code. 

SolrServer solrServer= new HttpSolrServer("url");
SolrQuery qry = new SolrQuery();
qry.setQuery("chicken"); //Set the search term here
//You can set all other parameters in Solr query as needed like sort, filter query, fields, facets  etc
QueryResponse response = solrServer.query(query);
List<SolrDocument> data = response.getResults();

Now data object will have the response form Solr. You can manipulate it as required and pass it back to UI.

In case if we want all the GET request to go through a set of logic, it is possible to add an handler in solrconfig.xml and set the requestHandler in the query as well.