Monday, April 19, 2010

Using Maven 2 and Ant's XMLTask to modify XML files

When we are talking about software development, it's not only about writing a code (for sure, high-quality code). It's also about a bunch of supporting processes like automated building, testing, deployment, integration, ... In this blog I am trying to touch every aspect so this post starts a series of articles about building Java projects with Apache Maven 2. The Maven's web site has very good documentation so I will skip introductory part and concentrate on some practical issues which arrive quite often.

Suppose, you have XML configuration files and depending on build profile you have to modify some parameters (database server address, JMS endpoints, ...). How to do that with Apache Maven 2? Quite easy using ... Apache Ant integration for Apache Maven 2. Apache Ant has excellent and very powerful plug-in to work with XML files - XMLTask. Let us make use of it!

<profiles>
 <profile>      
  <id>testing</id>
   <build>
    <plugins>
     <plugin>
      <artifactId>maven-antrun-plugin</artifactId>
       <dependencies>
        <dependency>
         <groupId>com.oopsconsultancy</groupId>
         <artifactId>xmltask</artifactId>
         <version>1.14</version>
       </dependency>
      </dependencies>
      <executions>
       <execution>
        <phase>prepare-package</phase>
        <configuration>
         <tasks> 
           <echo message="Using testing configuration" />
            <taskdef name="xmltask"
             classname="com.oopsconsultancy.xmltask.ant.XmlTask"
             classpathref="maven.plugin.classpath"/>
            <xmltask 
             source="${project.basedir}/src/main/webapp/WEB-INF/web.xml" 
             dest="${project.build.directory}/web.xml" 
             preserveType="true">        
            <remove path="//*[@id='<some id here>']" />
           </xmltask>           
         </configuration>
        </executions>
       </execution>
     </plugin>
    </plugins>
   </build>
  </profile>
 </profiles>

What this simple fragment does: for testing builds, it will remove from web.xml all XML elements with id attribute <some id here>. Not very meaningful but gives the idea how it works. XMLTask could do mostly everything you need: insert/removed elements and XML fragments, insert/remove/modify attributes with values and properties, copy/cut/paste XML, and a lot more. I found it extremely useful.

Sunday, April 18, 2010

On the wave of RIA, Adobe Flex and Java

This post will be not very technical but I would like to share some of my experience related to Internet applications development.

It's quite a few years I have been involved into web applications development. I started from PHP, then moved to ASP.NET, then to JSF, then AJAX diluted all that stuff, and finally I moved to Adobe Flex. The trend is obvious: web applications must be as closed to desktop counterparts as possible. Adobe Flex is really cool, very coooool ... I didn't play with Microsoft Silverlight and JavaFX too much but it all about the same.

As more reach become web applications, more features are requested from them. For developers it's a whole new world to explore. My current project is built on top of Adobe Flex and Java. It worth-while to say that Adobe Flex and Java integrates very good via BlazeDS (opensource) or LCDS (commercial) bridges. SpringSource provides excellent support for Flex and BlazeDS development by means of Spring BlazeDS integration project.

What all this is about... Development of RIA on top of Java platform is a challenge which requires from developer to engage the whole new technology stack. It's something which couldn't be done using pure Java platform. JavaFX is coming, but too late. Will it be successful?

Nevertheless, I would like to encourage developers to consider Adobe Flex as part of your next web project. It's worthwhile the time you will spend on it.

Saturday, January 30, 2010

Testing servlets with Spring

So far I haven't had a need to test servlets within Spring framework environment. But the issue came up recently and I am going to share my experience with testing file upload servlet based on Apache FileUpload and Spring.

Let's start with a file upload servlet implementation. I will omit some unnecessary details and concentrate on two issues: get application context and retrieve/save file to disk.
public class FileUploadServlet extends HttpServlet { @Override public void doPost( HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException { ApplicationContext appContext = WebApplicationContextUtils .getRequiredWebApplicationContext( getServletContext() ); // Get some beans here from application context ... DiskFileItemFactory factory = new DiskFileItemFactory(); ServletFileUpload upload = new ServletFileUpload( factory ); try { Iterator< ? > iter = upload.parseRequest( request ).iterator(); while( iter.hasNext() ) { FileItem item = ( FileItem )iter.next(); if( !item.isFormField() ) { // store items here ... } } response.setStatus( HttpServletResponse.SC_OK ); } catch( Exception e ) { response.setStatus( HttpServletResponse.SC_INTERNAL_SERVER_ERROR ); } finally { response.flushBuffer(); } } }
Servlet is ready. Let's develop test case to verify it. There are basically three steps:
  • create mock request (and response)
  • create servlet instance and pass Spring application context to it
  • wrap file into request and call servlet's post()
The code fragment below shows how easy it could be done using Spring testing scaffolding (thanks Spring team again).
public class UploadServlerTestCase extends AbstractJUnit4SpringContextTests { private byte[] buffer; @Before public void setUp() throws Exception { // Load file content from resource final InputStream in = getClass().getResourceAsStream( "test.pdf" ); buffer = new byte[ in.available() ]; in.read( buffer ); in.close(); } @Test public void testFileUpload() { // create mock servlet config and pass Spring application context to it StaticWebApplicationContext ctx = new StaticWebApplicationContext(); ctx.setParent( applicationContext ); MockServletConfig sc = new MockServletConfig(); sc.getServletContext().setAttribute( WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE, ctx ); // create mock request (and response) MockHttpServletRequest request = new MockHttpServletRequest( "POST", "http://localhost/" ); MockHttpServletResponse response = new MockHttpServletResponse(); // wrap file into request final ByteArrayOutputStream out = new ByteArrayOutputStream(); try { out.write( String.format( "-----1234\r\n" + "Content-Disposition: form-data; name=\"%s\"; filename=\"%s\"\r\n" + "Content-Type: %s\r\n" + "\r\n", "textField", "test.pdf", "application/pdf" ).getBytes() ); out.write( buffer ); out.write( new String( "\r\n-----1234" ).getBytes() ); out.flush(); request.setContentType( "multipart/form-data; boundary=---1234" ); request.setContent( out.toByteArray() ); } finally { out.close(); } // create servlet instance and call post() FileUploadServlet servlet = new FileUploadServlet(); servlet.init( sc ); servlet.doPost( request, response ); // do some checks to ensure file has been stored ... } }
Test case is ready. Depending on your uploads management strategy (disk, database, Amazon S3, ...), test case should be extended to ensure that file has been stored by upload servlet at proper location.

Sunday, December 20, 2009

Hibernate Search + Apache Lucene (tricks)

In this post I would like to show a few Hibernate Search tricks which could be useful in some cases (at least, I use them quite often):

  • how to get all indexed properties for entity?
  • final Class< ? > entityClass = ...;

    final FullTextSession fullTextSession = Search.getFullTextSession(
    sessionFactory.getCurrentSession() );

    final SearchFactory searchFactory = fullTextSession.getSearchFactory();
    final ReaderProvider readerProvider = searchFactory.getReaderProvider();

    final Collection< String > names = new ArrayList< String >();
    IndexReader indexReader = null;

    try {
    indexReader = readerProvider.openReader( searchFactory.getDirectoryProviders( clazz ) );
    for( Object obj: indexReader.getFieldNames( FieldOption.INDEXED ) ) {
    if( obj instanceof String ) {
    String name = ( String )obj;
    names.add( name );
    }
    }
    } finally {
    if( indexReader != null ) {
    readerProvider.closeReader( indexReader );
    }
    }
  • how to reindex whole existing database?
  • final FullTextSession fullTextSession = Search.getFullTextSession(
    sessionFactory.getCurrentSession() );

    final Set< ? > entitites = new HashSet< ? >();
    // Get all indexed persistent entities from Hibernate session factory
    Iterator< ? > iterator = sessionFactory.getConfiguration().getClassMappings();
    while( iterator.hasNext() ) {
    Object obj = iterator.next();
    if( obj instanceof PersistentClass ) {
    PersistentClass persistentClass = ( PersistentClass )obj;
    try {
    Class< ? > clazz = Class.forName( persistentClass.getClassName() );
    if( clazz.getAnnotation( Indexed.class ) != null ) {
    entitites.add( clazz );
    }
    }
    } catch( ClassNotFoundException ex ) {
    ex.printStackTrace();
    }
    }

    for( Class< ? > entityClass: entitites ) {
    fullTextSession.purgeAll( entityClass );
    fullTextSession.flushToIndexes();

    for( Object entity: session.createCriteria( entityClass ).list() ) {
    fullTextSession.index( entity );
    }
    }

    fullTextSession.flushToIndexes();
    fullTextSession.getSearchFactory().optimize();

Monday, December 7, 2009

Distributed Hibernate Search with Apache Tomcat 6, ActiveMQ and Spring

Today I would like to share my experience with configuring Hibernate Search in master/slave(s) deployment using Apache ActiveMQ, Spring and running all this stuff inside Apache Tomcat 6 container.

How it works:
- Hibernate Search supports distributed configuration using JMS back-end and master / slave(s) index
- master server exposes index over network share (NFS,...)
- slave(s) on regular base replicate this master copy to own local copies

Used version:
- Apache Tomcat 6.0.20
- Hibernate Search 3.1.1 GA
- Apache ActiveMQ 5.3.0
- Spring 2.5.6
- XBean-Spring 3.6

Master Index Configuration
Master configuration is a little bit complicated. Here are configuration-specific properties:

${local.index.dir} - directory to store master index
${master.index.dir} - directory to copy master index to, it's shared network location for replication with slave(s)

First of all, for simplification, let's run ActiveMQ broker on the same host. For this purpose we could use simple embedded broker configuration placed into WEB-INF/activemq.xml:

<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core
http://activemq.apache.org/schema/core/activemq-core.xsd">

<amq:broker brokerName="HibernateSearchBroker">
<amq:managementContext>
<amq:managementContext createConnector="false"/>
</amq:managementContext>

<amq:transportConnectors>

</amq:transportConnectors>
</amq:broker>

<amq:queue name="queue/hibernatesearch" physicalName="hibernateSearchQueue" />
</beans>
Then we need to configure JNDI resources (JMS Connection Factory and Queue) through web application META-INF/context.xml file (Tomcat-specific):
...
<!-- ActiveMQ ConnectionFactory -->
<Resource
name="jms/ConnectionFactory"
auth="Container"
type="org.apache.activemq.ActiveMQConnectionFactory"
description="JMS Connection Factory"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
brokerURL="tcp://0.0.0.0:61616?trace=true"
brokerName="HibernateSearchBroker" />

<!-- ActiveMQ HibernateSearch queue -->
<Resource
name="queue/hibernatesearch"
auth="Container" type="org.apache.activemq.command.ActiveMQQueue"
description="Hibernate search queue"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
physicalName="hibernateSearchQueue" />
...
Next step is configuration of the Hibernate Search itself through Hibernate configuration file (hibernate.cfg.xml):
<property name="hibernate.search.default.directory_provider">org.hibernate.search.store.FSMasterDirectoryProvider</property>
<property name="hibernate.search.default.indexBase">${local.index.dir}</property>
<property name="hibernate.search.default.sourceBase">${master.index.dir}</property>
<property name="hibernate.search.default.refresh">60</property>
One important difference between master and slave codebase: master implementation must include subclass of AbstractJMSHibernateSearchController as message listener. For example,

import javax.jms.MessageListener;

import org.hibernate.Session;
import org.hibernate.search.backend.impl.jms.AbstractJMSHibernateSearchController;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

@Service
public class JMSHibernateSearchController
extends AbstractJMSHibernateSearchController
implements MessageListener {

@Override
protected void cleanSessionIfNeeded(Session session) {
// clean session here ...
}

@Override
protected Session getSession() {
// return new session here ...
}
}

Finally, let's wrap it up inside Spring configuration file applicationContext.xml:
<bean id="broker" class="org.apache.activemq.xbean.BrokerFactoryBean">
<property name="config" value="WEB-INF/activemq.xml" />
<property name="start" value="true" />
</bean>

<bean name="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jms/ConnectionFactory" />
</bean>

<bean name="jmsHibernateSearchQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/queue/hibernatesearch" />

<bean id="hibernateSearchController" class="<your implementation of AbstractJMSHibernateSearchController>" />

<bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer" depends-on="broker">
<property name="connectionFactory" ref="jmsConnectionFactory"/>
<property name="destination" ref="jmsHibernateSearchQueue"/>
<property name="messageListener" ref="hibernateSearchController" />
</bean>
With those configurations in place Hibernate Search master is ready to run.


Slave Index Configuration
Slave(s) configuration is much simple. Here are configuration-specific properties:

${server} - server which runs ActiveMQ broker
${local.index.dir} - directory to store local index (master copy)
${master.index.share} - mounted network share with master index

First of all, we need to configure JNDI resources (JMS Connection Factory and Queue) through web application META-INF/context.xml file (Tomcat-specific):
...
<!-- ActiveMQ ConnectionFactory -->
<Resource
name="jms/ConnectionFactory"
auth="Container"
type="org.apache.activemq.ActiveMQConnectionFactory"
description="JMS Connection Factory"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
brokerURL="tcp://${server}:61616?trace=true"
brokerName="HibernateSearchBroker" />

<!-- ActiveMQ HibernateSearch queue -->
<Resource
name="queue/hibernatesearch"
auth="Container" type="org.apache.activemq.command.ActiveMQQueue"
description="Hibernate search queue"
factory="org.apache.activemq.jndi.JNDIReferenceFactory"
physicalName="hibernateSearchQueue" />
...
Then we have to configure Hibernate Search itself through Hibernate configuration file (hibernate.cfg.xml):
<property name="hibernate.search.default.directory_provider">org.hibernate.search.store.FSSlaveDirectoryProvider</property>
<property name="hibernate.search.default.indexBase">${local.index.dir}</property>
<property name="hibernate.search.default.sourceBase">${master.index.share}</property>
<property name="hibernate.search.default.refresh">60</property>
<property name="hibernate.search.worker.backend">jms</property>
<property name="hibernate.search.worker.jms.connection_factory">java:comp/env/jms/ConnectionFactory</property>
<property name="hibernate.search.worker.jms.queue">java:comp/env/queue/hibernatesearch</property>
<property name="hibernate.search.worker.jndi.java.naming.factory.initial">org.apache.activemq.jndi.ActiveMQInitialContextFactory</property>
And ... that's it!

Few additional words about testing all this stuff with JUnit. Only problem is JNDI which could be mocked up with Spring JNDI templates. For example:

<bean name="jmsConnectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jms/ConnectionFactory" />
<property name="jndiTemplate">
<bean class="org.springframework.mock.jndi.ExpectedLookupTemplate">
<constructor-arg index="0" value="java:comp/env/jms/ConnectionFactory" />
<constructor-arg index="1">
<bean class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL">
<value>tcp://0.0.0.0:61616</value>
</property>
</bean>
</constructor-arg>
</bean>
</property>
</bean>

<bean name="jmsHibernateSearchQueue" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/queue/hibernatesearch" />
<property name="jndiTemplate">
<bean class="org.springframework.mock.jndi.ExpectedLookupTemplate">
<constructor-arg index="0" value="java:comp/env/queue/hibernatesearch" />
<constructor-arg index="1">
<bean id="jmsHibernateSearchQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="queue/hibernateSearchQueue"/>
</bean>
</constructor-arg>
</bean>
</property>
</bean>
...

Tuesday, November 17, 2009

Freemarker ... powerful templating in Java

In most applications there is a point when you as developer need to create a template (for e-mail, notification, message, ...). Basically, the very simple template is just a string with some parameters like "Hello, ${name}" which have to be substituted at run-time. In many cases simple replace(...) is enough, but what if you need some logic inside template (expressions, flow control, function calls, ... )? For those who need powerful templates in Java projects I would like to recommend Freemarker - Java Template Engine Library.

Freemarker, in fact, has very good documentation. It's easy to start using it without any specific knowledge. In this post I would like to share a few issues which I found very useful. Let's start with very basic e-mail message template.
Hello ${user.fistName} ${user.lastName}!
Welcome to the world of templates!
In this template we assume that some object (bean) user with public properties firstName and lastName must be passed to template engine in order to build e-mail message. Let's save this template to message.ftl file (.ftl is default file extension for Freemarker templates). To process this template we need few simple steps:
Configuration cfg = new Configuration();

// Specify the data source where the template files come from.
cfg.setDirectoryForTemplateLoading( new File("/where/you/store/templates"));
// Specify how templates will see the data-model
cfg.setObjectWrapper( ObjectWrapper.DEFAULT_WRAPPER );

final Map< String, Object > context = new HashMap< String, Object >();
final User user = new User( "First Name", "Last Name" );
context.put( "user", user );

final Template t = cfg.getTemplate( "message.ftl");
final Environment env = t.createProcessingEnvironment( context, writer );
env.process();
It's quite clear what the code does: create configuration, create context (simple map) and then process template (stored in file). When this code snippet finishes, writer will contain fully processed template. The tricky part here is such code:
// Specify how templates will see the data-model
cfg.setObjectWrapper( ObjectWrapper.DEFAULT_WRAPPER );
We will see what it means later but for know it's enough to say that it has influence on how Freemarker processes objects passed to template (via context).

We are done with e-mail message but what if you need to construct e-mail subject in template as well and return it back to callee? Freemarker allows to do that using environment. Let's modify the template a little bit:
<#assign subject="Congratulations ${user.fistName} ${user.lastName}!" />
Hello ${user.fistName} ${user.lastName}!
Welcome to the world of templates!
So in this template we create internal variable subject which we will use later in code. We need just a few additional steps:
...
env.process();

String subject = "";
final TemplateModel subjectModel = env.getVariable( "subject" );
if( subjectModel instanceof TemplateScalarModel ) {
subject = ( ( TemplateScalarModel )subjectModel ).getAsString();
}
Next interesting question is about object properties / functions / static functions which you could use inside template. Basically, with ObjectWrapper.DEFAULT_WRAPPER you are free to use any object property which comply with Java Beans specification. In case you need, for example, calls like getClass(), you need ObjectWrapper.BEANS_WRAPPER.
cfg.setObjectWrapper( ObjectWrapper.BEANS_WRAPPER );  
Using static functions / properties could be done by means of static models. Let say you have a class with static my.package.StaticUtil. To use static members of this class inside the template we need to use code like:
final BeansWrapper wrapper = BeansWrapper.getDefaultInstance();
final TemplateHashModel staticModels = wrapper.getStaticModels();

final TemplateHashModel staticUtil =( TemplateHashModel )staticModels
.get( "my.package.StaticUtil" );

context.put( "StaticUtil", staticUtil );
Later in template you can use constructions like ${StaticUtil.someStaticFunction()} to access static members.

That's it for now. One thing is worthwhile to say is that Freemarker has good integration with Eclipse via JBoss Tools.

Saturday, July 11, 2009

Hibernate Search + Apache Lucene (continued)

Let's continue very useful and interesting topic about Hibernate Search and Apache Lucene. In my last post we discovered new annotations which allow persistent entities to be "searchable". In this post I am going to show how to query such entities with various search criteria.

Once entities are annotated, they could be queried with regular Apache Lucene query. Simple flow looks like this:
Session session = sessionFactory.getNewSession();
FullTextSession fullTextSession = Search.getFullTextSession( session );

try {
fullTextSession.beginTransaction();

// query - regular Lucene query, f.e. "author=King"
// entities - list of entity classes, like Entity1.class, Entity2.class, ...

FullTextQuery q = fullTextSession
.createFullTextQuery( query, entities );

// Get a collection of matched entities
Collection< Object > results = q.list();

fullTextSession.getTransaction().commit();
} catch (Exception ex ) {
fullTextSession.getTransaction().rollback();
throw ex;
} finally {
session.close();
}
That's how Hibernate Search makes it simple. The only difference from Hibernate's model is FullTextSession usage. Here are few simple details about how Hibernate Search works on top of Apache Lucene:

  • for each entity class Apache Lucene creates own index

  • Hibernate Search adds a few additional properties (like persistent entity Id and Type) to associate Apache Lucene document and Hibernate entity

  • for each query Hibernate Search uses query rewrite to ensure only indexed properties of a particular entity are present in query

  • entity's (re)indexing occurs when database transaction commits (but Hibernate Search allows you manually (re)index entity)

Hibernate Search also allows you to use very powerful features like filters (with parameters) and sorting. Filters are extremely useful in case of security rules or search with search results implementation.
...
FullTextQuery q = fullTextSession
.createFullTextQuery( query, entities );
.setSort( new Sort(
new SortField[] {
new SortField( "name", false )
}
)
);

// Enable filter with name 'filterName' and parameter 'value' set to value
Object value = ...;
q.enableFullTextFilter( "filterName" )
.setParameter( "value", value );

// Get a collection of matched entities
Collection< Object > results = q.list();
...
It's very simple to integrate and use Hibernate Search into existing applications. It most cases I found integration seamless.

There are a bunch of interesting issues that might occur:

  • how to get all indexed properties for entity?

  • how to reindex whole existing database?

  • how to deploy master/slave(s) configuration?

For those I am going to create dedicated post because an implementation is a little bit tricky (but in general not complicated at all).