Distributed Hibernate Search with Apache Tomcat 6, ActiveMQ and Spring

Today I would like to share my experience with configuring Hibernate Search in master/slave(s) deployment using Apache ActiveMQ, Spring and running all this stuff inside Apache Tomcat 6 container.

How it works:
- Hibernate Search supports distributed configuration using JMS back-end and master / slave(s) index
- master server exposes index over network share (NFS,...)
- slave(s) on regular base replicate this master copy to own local copies

Used version:
- Apache Tomcat 6.0.20
- Hibernate Search 3.1.1 GA
- Apache ActiveMQ 5.3.0
- Spring 2.5.6
- XBean-Spring 3.6

Master Index Configuration
Master configuration is a little bit complicated. Here are configuration-specific properties:

${local.index.dir} - directory to store master index
${master.index.dir} - directory to copy master index to, it's shared network location for replication with slave(s)

First of all, for simplification, let's run ActiveMQ broker on the same host. For this purpose we could use simple embedded broker configuration placed into WEB-INF/activemq.xml:

<beans
 xmlns="http://www.springframework.org/schema/beans"
 xmlns:amq="http://activemq.apache.org/schema/core"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.springframework.org/schema/beans
 http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
 http://activemq.apache.org/schema/core
 http://activemq.apache.org/schema/core/activemq-core.xsd">

 <amq:broker brokerName="HibernateSearchBroker">
     <amq:managementContext>
         <amq:managementContext createConnector="false"/>
     </amq:managementContext>

     <amq:transportConnectors>
         
     </amq:transportConnectors>
  </amq:broker>

  <amq:queue name="queue/hibernatesearch" physicalName="hibernateSearchQueue" />
</beans>

Then we need to configure JNDI resources (JMS Connection Factory and Queue) through web application META-INF/context.xml file (Tomcat-specific):

...
<!-- ActiveMQ ConnectionFactory -->
<Resource
name="jms/ConnectionFactory"
auth="Container"
type="org.apache.activemq.ActiveMQConnectionFactory"
description="JMS Connection Factory"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
brokerURL="tcp://0.0.0.0:61616?trace=true"
brokerName="HibernateSearchBroker" />

<!-- ActiveMQ HibernateSearch queue -->
<Resource
name="queue/hibernatesearch"
auth="Container" type="org.apache.activemq.command.ActiveMQQueue"
description="Hibernate search queue"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
physicalName="hibernateSearchQueue" />
...

Next step is configuration of the Hibernate Search itself through Hibernate configuration file (hibernate.cfg.xml):

<property name="hibernate.search.default.directory_provider">org.hibernate.search.store.FSMasterDirectoryProvider</property>
<property name="hibernate.search.default.indexBase">${local.index.dir}</property>
<property name="hibernate.search.default.sourceBase">${master.index.dir}</property>
<property name="hibernate.search.default.refresh">60</property>

One important difference between master and slave codebase: master implementation must include subclass of AbstractJMSHibernateSearchController as message listener. For example,

import javax.jms.MessageListener;

import org.hibernate.Session;
import org.hibernate.search.backend.impl.jms.AbstractJMSHibernateSearchController;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class JMSHibernateSearchController
 extends AbstractJMSHibernateSearchController
     implements MessageListener {

 @Override
 protected void cleanSessionIfNeeded(Session session) {
    // clean session here ...     
 }

 @Override
 protected Session getSession() {
    // return new session here ...     
 }
}

Finally, let's wrap it up inside Spring configuration file applicationContext.xml:

<bean id="broker" class="org.apache.activemq.xbean.BrokerFactoryBean">
 <property name="config" value="WEB-INF/activemq.xml" />
 <property name="start" value="true" />
</bean>  

<bean name="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
 <property name="jndiName" value="java:comp/env/jms/ConnectionFactory" />
</bean>

<bean name="jmsHibernateSearchQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
 <property name="jndiName" value="java:comp/env/queue/hibernatesearch" />

<bean id="hibernateSearchController" class="<your implementation of AbstractJMSHibernateSearchController>" />

<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer" depends-on="broker">
 <property name="connectionFactory" ref="jmsConnectionFactory"/>
 <property name="destination" ref="jmsHibernateSearchQueue"/>
 <property name="messageListener" ref="hibernateSearchController" />
</bean>

With those configurations in place Hibernate Search master is ready to run.


Slave Index Configuration
Slave(s) configuration is much simple. Here are configuration-specific properties:

${server} - server which runs ActiveMQ broker
${local.index.dir} - directory to store local index (master copy)
${master.index.share} - mounted network share with master index

First of all, we need to configure JNDI resources (JMS Connection Factory and Queue) through web application META-INF/context.xml file (Tomcat-specific):

...
<!-- ActiveMQ ConnectionFactory -->
<Resource
name="jms/ConnectionFactory"
auth="Container"
type="org.apache.activemq.ActiveMQConnectionFactory"
description="JMS Connection Factory"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
brokerURL="tcp://${server}:61616?trace=true"
brokerName="HibernateSearchBroker" />

<!-- ActiveMQ HibernateSearch queue -->
<Resource
name="queue/hibernatesearch"
auth="Container" type="org.apache.activemq.command.ActiveMQQueue"
description="Hibernate search queue"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
physicalName="hibernateSearchQueue" />
...

Then we have to configure Hibernate Search itself through Hibernate configuration file (hibernate.cfg.xml):

<property name="hibernate.search.default.directory_provider">org.hibernate.search.store.FSSlaveDirectoryProvider</property>
<property name="hibernate.search.default.indexBase">${local.index.dir}</property>
<property name="hibernate.search.default.sourceBase">${master.index.share}</property>
<property name="hibernate.search.default.refresh">60</property>
<property name="hibernate.search.worker.backend">jms</property>
<property name="hibernate.search.worker.jms.connection_factory">java:comp/env/jms/ConnectionFactory</property>
<property name="hibernate.search.worker.jms.queue">java:comp/env/queue/hibernatesearch</property>
<property name="hibernate.search.worker.jndi.java.naming.factory.initial">org.apache.activemq.jndi.ActiveMQInitialContextFactory</property>

And ... that's it!

Few additional words about testing all this stuff with JUnit. Only problem is JNDI which could be mocked up with Spring JNDI templates. For example:

<bean name="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiName" value="java:comp/env/jms/ConnectionFactory" />
    <property name="jndiTemplate">
        <bean class="org.springframework.mock.jndi.ExpectedLookupTemplate">
            <constructor-arg index="0" value="java:comp/env/jms/ConnectionFactory" />
            <constructor-arg index="1">
                <bean class="org.apache.activemq.ActiveMQConnectionFactory">
                    <property name="brokerURL">
                        <value>tcp://0.0.0.0:61616</value>
                    </property>
                </bean>
            </constructor-arg>
        </bean>
    </property>
</bean>

<bean name="jmsHibernateSearchQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
    <property name="jndiName" value="java:comp/env/queue/hibernatesearch" />
    <property name="jndiTemplate">
       <bean class="org.springframework.mock.jndi.ExpectedLookupTemplate">
           <constructor-arg index="0" value="java:comp/env/queue/hibernatesearch" />
           <constructor-arg index="1">
               <bean id="jmsHibernateSearchQueue"   class="org.apache.activemq.command.ActiveMQQueue">
                   <constructor-arg value="queue/hibernateSearchQueue"/>
               </bean>               
           </constructor-arg>
       </bean>
    </property>
</bean>
...

Hibernate Search + Apache Lucene (continued)

Let's continue very useful and interesting topic about Hibernate Search and Apache Lucene. In my last post we discovered new annotations which allow persistent entities to be "searchable". In this post I am going to show how to query such entities with various search criteria.

Once entities are annotated, they could be queried with regular Apache Lucene query. Simple flow looks like this:

Session session = sessionFactory.getNewSession();
FullTextSession fullTextSession = Search.getFullTextSession( session );

try {  
fullTextSession.beginTransaction();

// query - regular Lucene query, f.e. "author=King"
// entities - list of entity classes, like Entity1.class, Entity2.class, ...

FullTextQuery q = fullTextSession
 .createFullTextQuery( query, entities );

// Get a collection of matched entities
Collection< Object > results = q.list();

fullTextSession.getTransaction().commit();
} catch (Exception ex ) {
fullTextSession.getTransaction().rollback();
throw ex;
} finally {
session.close();
}

That's how Hibernate Search makes it simple. The only difference from Hibernate's model is FullTextSession usage. Here are few simple details about how Hibernate Search works on top of Apache Lucene:

• for each entity class Apache Lucene creates own index


• Hibernate Search adds a few additional properties (like persistent entity Id and Type) to associate Apache Lucene document and Hibernate entity


• for each query Hibernate Search uses query rewrite to ensure only indexed properties of a particular entity are present in query


• entity's (re)indexing occurs when database transaction commits (but Hibernate Search allows you manually (re)index entity)

Hibernate Search also allows you to use very powerful features like filters (with parameters) and sorting. Filters are extremely useful in case of security rules or search with search results implementation.

...
FullTextQuery q = fullTextSession
 .createFullTextQuery( query, entities );
     .setSort( new Sort(
             new SortField[] {
                 new SortField( "name", false )
             }
         )
     );

// Enable filter with name 'filterName' and parameter 'value' set to value
Object value = ...;
q.enableFullTextFilter( "filterName" )
 .setParameter( "value", value );

// Get a collection of matched entities
Collection< Object > results = q.list();
...

It's very simple to integrate and use Hibernate Search into existing applications. It most cases I found integration seamless.

There are a bunch of interesting issues that might occur:

• how to get all indexed properties for entity?


• how to reindex whole existing database?


• how to deploy master/slave(s) configuration?

For those I am going to create dedicated post because an implementation is a little bit tricky (but in general not complicated at all).

Hibernate Search + Apache Lucene

It worthwhile to say that every Java developer knows about Apache Lucene project. No doubts, the best open source search engine! But today it's not about Apache Lucene, it's about Hibernate Search project, just another excellent library from Hibernate team.

So, what if you need some search capabilities in your application? For sure, it depends on requirements, but let's consider Java application which uses database as back-end and Hibernate as ORM framework. I would suggest a few ways to do search in this case:
1) Implement own search framework based on full text search capabilities of your database server (vendor-locked solution).
2) Implement own search framework with SQL/HQL queries. Works well for very simple scenarios. but is not enough in most cases.
3) Use Hibernate Search (and Apache Lucene). As powerful as Apache Lucene and as flexible as Hibernate framework. The most preferable solution.

From developer point of view, integration of the Hibernate Search is just a matter of additional annotations together with a few configuration lines in hibernate.cfg.xml. Let's start with last one.

<property name="hibernate.search.default.directory_provider">
org.hibernate.search.store.RAMDirectoryProvider
</property>

That tells Hibernate Search to store all indexes in RAM (don't do that on production system!). And ... that's it!

Now let's look on typical persistent entity and ... how to make it searchable!

@Entity
@Indexed
public class User {
   private Long id;
   private String firstName;
   private String lastName;

   @Id
   @GeneratedValue( strategy = GenerationType.AUTO )
   @Column( name = "UserID" )
   @DocumentId
   public Long getId() {
       return id;
   }
   public void setId(Long id) {
       this.id = id;
   }

   @Basic
   @Column( name = "FirstName", length = 50 )
   @Field( index = Index.UN_TOKENIZED )
   public String getFirstName() {
       return firstName;
   }
  
   public void setFirstName( String firstName ) {
       this.firstName = firstName;
   }
 
   @Basic
   @Column( name = "LastName", length = 50 )
   @Field( index = Index.UN_TOKENIZED )
   public String getLastName() {
       return lastName;
   }
 
   public void setLastName( String lastName ) {
       this.lastName = lastName;
   }
}

A few annotations in bold (@Indexed, @Field, @DocumentId) make all the magic. Hibernate Search provides additional listeners which take care on entity indexing (Lucene documents creation/deletion). In next post I will explain how to search stored entities by specific criteria. For now, I would mention a great book Hibernate Search in Action which perfectly explains all details.