Cross transactional cache invalidation across WCS and Search applications

Introduction
Cache invalidation is one of the key and tedious part of software development. It is really important that we purge the invalid data out of the cache at the right time so that the customers always see the updated content. After FEP7 IBM has segregated the websphere commerce application into two. WCS and Search. We use DynaCacheInvalidation command to invalidate the cache.

Caching
It is now possible to cache the data in Search as well as WCS dynacache as they are placed in different servers. We can make changes to the cachespec.xml in these applications and define the caching strategy. As both the caches are used we should make sure that whenever the data is updated the cache entries must be cleared in WCS and search in the right order.

Cache invalidation
In WCS,  OOB uses DynacacheinvalidationCmd for clearing the cache entries. As this is a scheduler command, it runs inside the WCS JVM and will clear the cache entries from WCS dynacache. For better performance we can cache the data at search as well.

For Search, OOB uses a filter. Every search request will go through this filter and it will invalidate the relevant cache entires as per the dependency id and insert time in CAHCHEIVL table. The name of the filter is RestInvalidationFilter. The filter will look at the value of "CrossTransactionCache/invalidationJobInterval"  in wc-component.xml under com.ibm.commerce.foundation folder. If the value is 30, it means the filter will trigger the cache invalidation every 30 seconds.

Why Cross transactional invalidations are required betweens WCS and Search? 
For example the jsps can be cached in WCS dynacache and search responses can be cached in Search dynacache. In this scenario we need both the caches to be cleared in the right fashion. Let us take a product page(say product.jsp) for example. To show the product page we make a /detailsByProductId search call to get the product data. The page is cached in commerce and the search response is cached in Search JVM. If any of the product data changes we need to invalidate both caches. The search cache has to be invalidated before the WCS cache because in case if commerce cache entry is cleared (where as search is not yet cleared) and it makes a call to search for getting the details, the search will respond with old data.

How to trigger invalidation between apps
There are many approaches to achieve it. One of it is to customise the DynaCacheInvalidationCmd and to make a search rest call in it.

1. Create a new rest in service in search server and implement it in the same way as what the filter currently does. The changes where it looks for component configuration and all can be overridden. So the rest can call the OOB LocalJVMDynaCacheInvalidationHelper (as the OOB rest filter does). It will automatically remember the last invalidation time and will check for the cacheivl records after that.

2. Change the DynaCaheInvalidationCmd implementation. Extend the OOB command and make sure we call the above rest call before  we trigger the WCS invalidations. That will make sure that the search cache is invalidated prior to WCS cache.

3. Extend the wc-component.xml and override the CrossTransactionCache/invalidationJobInterval value to -1 so that the filter will not invalidate on its own. Alternatively the filter can be removed from web.xml as well.

Clustered environment: In a clustered environments there were issues with cache getting replicated across cells. One of the way to fix it is to schedule two different instances of DynaCacheInvalidation job on one server in each cell using the JVM property com.ibm.commerce.scheduler.SchedulerHostName

Creating custom rest service using databeans

Introduction
Custom rest services will be required in multiple scenarios. WCS provides three ways to create new REST services.

• Using BOD mapping framework
• Using controller command framework
• Using databean mapping framework

Let us create a new rest service using a databean. The url of the service is https://localhost/wcs/resources/store/{storeId}/myPath/customDetails

Steps
1. Create a custom handler to service your request.
A new java class must be created (say MyCustomHandler) and it should extend the class com.ibm.commerce.rest.classic.core.AbstractConfigBasedClassicHandler.

In the handler set the path and create a method to service the request

Path("store/{storeId}/myPath")
@Encoded
@Description("This class provides RESTful services to get some custom details")
public class MyCustomHandler extends AbstractConfigBasedClassicHandler{
@POST
@Path("customDetails")
@Description("Gets custom details")
public Response findCustomDetails(
@PathParam("storeId")
@ParameterDescription(description = "storeId in parameter", valueProviderClass = StoreIdProvider.class, required = true) String storeId,
@QueryParam("responseFormat") @ParameterDescription(description = "responseFormat description", valueProviderClass = ResponseType.class) String responseFormat) {
//params is a HashMap containing the parameters to be set for bean
Map params = new HashMap<String, String>();
params.put("storeId", storeId);

//call executeConfigBasedBeanWithContext method to execute the databean and to get the response
//calling with profile name Custom_Profile_1
Response result =
executeConfigBasedBeanWithContext("com.mycompany.beans.MyCustomDataBean", "Custom_Profile_1", responseFormat,params);

}
}

2. Add the handler to resource file
Navigate to Rest\WebContent\WEB-INF\config\resources-ext.properties and add the fully qualified name of the handler in the file. If the file is not there, create it

3. Create the data bean mapping xml 
Navigate to Rest\WebContent\WEB-INF\config\beanMapping-ext
Create an xml file with complete name of Databean. com.mycompany.beans.MyCustomDataBean.xml
Configure the input-output profiles in this xmls. We can have multiple profiles in the same xml. A simple example is given below. It will have the input/output parameter name and the corresponding method in data bean used for setting/getting them. The profile name used on the handler will choose what profile will be called.

<?xml version="1.0" encoding="UTF-8"?>
<bean>
<profiles>
<profile name="Custom_Profile_1">
<inputs>
<input inputName="storeId" methodName="setStoreId" />
</inputs>

<outputs>
<output methodName="getDetails" outputName="Details" />
</outputs>
</profile>
                       <!--Add more profiles if needed   -->
</profiles>
</bean>

4. Create the databean
Create the data bean. Use the same steps that we use for creating databeans. The changes specific for rest service bean are

• It must implement com.ibm.commerce.security.Delegator
• It must override the getDelegate() method. We can just return null in that method.
• It must have all methods specified in the mapping xml

Restart and republish is required for reflecting the changes