Testing persistence layer

As a supporter of TDD I prefer to test as much as it's necessary to be sure the code works properly (but I'm not a paranoid in that). With respect to this, testing persistence layer takes a very important place in my TDD practices.

By "testing persistence layer" I basically mean two things:
- testing that ORM mapping (Hibernate) is valid
- testing that application works well with rich test data sets

Using persistence layer tests allows to verify corner cases as well as application behavior in case of different database faults. Even more, development with testing in mind allows to design low coupled and well layered application architecture.

To start with, let's reuse two classes Employer and Employee from previous post and develop simple test cases for them. Thanks to Hibernate, we're completely decoupled from underlying database so we can create any suitable testing configuration. And thanks to HSQLDB project we're able to easily create in-memory database and relegate standalone database server deployment. For sure, we'll create test cases based on JUnit framework.

First, let's create abstract test case encapsulating all configuration in it. Basically, we have to configure Hibernate, add annotated classes, and create session factory.

import org.hibernate.Session;
import org.hibernate.dialect.HSQLDialect;
import org.hsqldb.jdbcDriver;
import org.hibernate.cfg.AnnotationConfiguration;
import org.hibernate.cfg.Environment;

public abstract class AbstractPersistentTestCase {
    private AnnotationConfiguration configuration;
    private SessionFactory sessionFactory;

    @Before
    protected void setUp() throws Exception {
        configuration = new AnnotationConfiguration();
        configuration.setProperty( Environment.DIALECT, 
            HSQLDialect.class.getName() );
        configuration.setProperty( Environment.DRIVER, 
            jdbcDriver.class.getName() );
        configuration.setProperty( Environment.URL, 
            "jdbc:hsqldb:mem:testdb" );
        configuration.setProperty( Environment.CURRENT_SESSION_CONTEXT_CLASS,
            "org.hibernate.context.ThreadLocalSessionContext" );
        configuration.setProperty( Environment.HBM2DDL_AUTO, 
            "create-drop" );
        configuration.setProperty( Environment.STATEMENT_BATCH_SIZE, 
            "0" );
        configuration.setProperty( Environment.SHOW_SQL, 
            "false" );
        configuration.setProperty( Environment.FORMAT_SQL, 
            "true" );
        
        configuration.addAnnotatedClass( Employee.class )
            .addAnnotatedClass( Employer.class );

        sessionFactory = configuration.buildSessionFactory();
    }

    @After
    public void tearDown() {
        SchemaExport schemaExport = new SchemaExport( configuration );
        schemaExport.drop( true, true );
    }

    protected Session getSession() {
        return sessionFactory.getCurrentSession(  );
    }
}

Then, let's develop a simple test case for persisting Employer and Employee classes. There's one important note here. Those test methods should be executed within transaction boundary (for example, with help of AspectJ). Otherwise, the HibernateException comes up.

public class PersistentTestCase extends AbstractPersistentTestCase {
    @Test
    public void testSaveEmployer() {
        Employer employer = new Employer();
        getSession().save( employer );

        Assert.assertNotNull( employer.getId() );
    }

    @Test
    public void testSaveEmployee() {
        Employer employer = new Employer();
        getSession().save( employer );

        Employee employee = new Employee();
        employee.setEmployer( employer );
        getSession().save( employee );

        Assert.assertNotNull( employee.getId() );
    }
}

Here I covered very simple scenario. Just to give the idea. Next step will be preparing test data sets with help of DbUnit. It allows to prepare data in XML format (among others) and upload this file into database. Let's prepare quite simple XML file 'dataset.xml':

    <?xml version="1.0" encoding="UTF-8"?>
    <dataset>
        <employer id="1" name="IBM" email="ibm@ibm.com" />
        <employer id="2" name="Microsoft" email="microsoft@microsoft.com" />
    </dataset>

And that's it! DbUnit will automatically map XML elements to tables and XML attributes to columns! Awesome! Let's develop the test case using DbUnit and newly created data set.

public class DatasetTestCase extends AbstractPersistentTestCase {
    @Before
    protected void setUp() throws Exception {
        super.setUp();
        uploadTestDataset();
    }

    private void uploadTestDataset() throws Exception
    {
        final Session session = getSession();
        Transaction t = session.beginTransaction(  );

        try {
            IDataSet dataSet = new FlatXmlDataSet( 
                new FileInputStream( 
                    new File( 
                        getClass().getResource( "/dataset.xml" ).toURI() 
                    )
                )  
            );

            IDatabaseConnection connection = 
                new DatabaseConnection( session.connection() );            
            DatabaseOperation.CLEAN_INSERT.execute( connection, 
                dataSet );

            t.commit(  );
        } finally {
            if ( t.isActive(  ) ) {
                t.rollback(  );
            }
        }
    }

    @Test
    public void testLoadEmployer() {
        Employer employer = ( Employer )getSession().load( 
            Employer.class, new Integer( 1 ) );

        Assert.assertEquals( "IBM", employer.getName() );
        Assert.assertEquals( "ibm@ibm.com", employer.getEmail() );
    }
}

This scenario is very simple as well. But it's just a foundation ... Developing comprehensive datasets and testing application business logic against them is great step in achieving high product quality. Those techniques allow to fully cover very complex flows with tests and detect errors early.

The only problem with that is ... maintenance. It's particularly true for projects in active development. Changes in business logic lead to test failures (in most cases). Supporting huge tests code base can me nightmare. So ... I'm always looking for balanced solution.

Using Criteria API in Hibernate

Hibernate is awesome! I'm pretty excited about Hibernate and I'll continue to use it as primary ORM solution for my future projects. Today I'd like to share typical usage scenarios of Hibernate Criteria API which provides extremely powerful capabilities to build strong typed queries.

Let's start with few simple classes (Employer and Employee) mapped to database tables. We'll use those in future examples.

@Entity
@Table( name = "employees" )
public class Employee {
   @Id
   @GeneratedValue( strategy = GenerationType.AUTO )
   @Column( name = "id" )
   private Integer id;

   @Column( name = "name" )
   private String name;

   @Column( name = "email" )
   private String email;

   @ManyToOne
   private Employer employer;

   public void setId( Integer id ) {
       this.id = id;
   }

   public Integer getId() {
       return this.id;
   }

   public void setName( String name ) {
       this.name = name;
   }

   public String getName() {
       return this.name;
   }

   public void setEmail( String email ) {
       this.email = email;
   }

   public String getEmail() {
       return this.email;
   }
};

@Entity
@Table( name = "employers" )
public class Employer {
   @Id
   @GeneratedValue( strategy = GenerationType.AUTO )
   @Column( name = "id" )
   private Integer id;

   @OneToMany( cascade = CascadeType.ALL, mappedBy = "employer" )
   private Set<Employee> employees = new HashSet<Employee>();

   public void setId( Integer id ) {
       this.id = id;
   }

   public Integer getId() {
       return this.id;
   }

   public Set<Employee> getEmployees() {
       return this.employees;
   }

   public void setEmployees( Set<Employee> employees ) {
       this.employees = employees;
   }
};

Let say we have to find Employees and Employers using different search criteria. For all examples I assume that helper method getCurrentSession() returns Hibernate's session and each code fragment is running within transaction.

1) Find all Employees with name "Tom" (the easiest one).

// Get current session
final Session session = getCurrentSession();

// Create Criteria for Employee class
final Criteria criteria = session.createCriteria( Employee.class );
// Add name = "Tom" restriction
criteria.add( Restrictions.eq( "name", "Tom" );

// Run query and get the results
List< Employee gt; employees = criteria.list();
// Do something here
...

2) Find all Employers which don't have any Employees.

// Get current session
final Session session = getCurrentSession();

// Create Criteria for Employer class
final Criteria criteria = session.createCriteria( Employer.class );
// Add empty Employees restriction
criteria.add( Restrictions.isEmpty( "employees" );

// Run query and get the results
List< Employer gt; employers = criteria.list();
// Do something here
...

3) Find all Employers which have Employees with name "Tom".

// Get current session
final Session session = getCurrentSession();

// Create Criteria for Employer class
final Criteria criteria = session.createCriteria( Employer.class );
// Create alias for Employees collection
criteria.createAlias( "employees", "employees" );
// Add name = "Tom" restriction
criteria.add( Restrictions.eq( "employees.name", "Tom" );

// Run query and get the results
List< Employer > employers = criteria.list();
// Do something here
...

4) For this example let me introduce another table "black_list" and BlackListEntry entity mapping.

@Entity
@Table( "black_list" )
public class BlackListEntry {
   @Id
   @GeneratedValue( strategy = GenerationType.TABLE )
   @Column( name = "email" )
   private String email;

   public void setEmail( String email ) {
       this.email = email;
   }

   public String getEmail() {
       return this.email;
   }
};

The purpose of this entity is pretty simple: a black list of e-mail addresses. Those should be exclude from any mailing lists. Let's find all Employees whose e-mails are in that black list.

// Get current session
final Session session = getCurrentSession();

// Create detached Criteria
DetachedCriteria detachedCriteria =
    DetachedCriteria.forClass( BlackListEntry.class );
// Create projection to select 'email' only
detachedCriteria.setProjection( Projections.property( "email" ) );

// Create Criteria for Employee class
final Criteria criteria = session.createCriteria( Employee.class );
// Add subquery restriction (all emails in black list)
criteria.add( Subqueries.propertyIn( "email", detachedCriteria ) );

// Run query and get the results
List< Employee > employees = criteria.list();
// Do something here
...

That's it. :-)

What I like in C# 3.0 and what I would like to see in Java

Nevertheless I'm working mostly with Java right now, Microsoft .NET platform and C# is also the stuff I'm very interested to know. I've been developing on Microsoft .NET since 2002 and I'm really excited how such efforts and investments Microsoft is putting to it. Sun seems to miss the train ...

Java 1.5 had been a great, innovative release either of the language and platform. Sun's answer to Microsoft's C# 2.0. But not enough ... Microsoft delivered C# 3.0 so far with great features inside: enhanced initialization (type instances & collections), anonymous types and delegates, LINQ, lambda-functions ( == closures) ... As a developer, I'm really excited about those things. Many of them I would like to have in Java as well.

1) Collection initialization

Collection < int > integers = new ArrayList< int >() { 1, 2, 3, 4 };
ArrayList < int > list = { 1, 2, 3, 4 };

2) Closures

This is a most wanted featured I guess. Any modern language has to support it. Groovy is very good complementary of Java with excellent language syntax tradeoffs.

db.eachRow( "SELECT * FROM uses" ) { user ->
// Do something here
}

3) Class initialization by property names

Employee employee = new Employee() {
FirstName = "Bob",
LastName = "Smith"
};

4) Enhanced generics (parametrized types) support
Even Java 1.5 brings something like C# generics (which both are similar to C++ templates), the Java's implementation is the worst. It's my point of view.

// I would like to have something like this. Of course, it means that some
// type T has to have default constructor. C# uses constraints for that
// (like new()) so why Java doesn't?
class A<> {
private T t = new T();
};

// Why it's impossible to get the class of generic type parameter?
// Sure, taking into account that it's impossible for int, double, ...
// there're reasons for that behavior. Again, why don't we use constraints
// for that (like class)?
class A {
<T>void func( T ) {
Class< ? > t = T.class;
}
};

5) Default type value

int t = default( int );
Integer t = default( Integer.class );

Hope, Sun is going to thieve something from C# in put it to the Java. Will see ...

Java and dynamic languages

Dynamic languages (like Ruby, Groovy) are quite popular in nowadays. As for me it's a big challenge for developers to learn and use at least one dynamic language in every day job. Of course, there should be a good reason to do that.

So today I would like to demonstrate some examples how I'm using Groovy to write tests for Java classes I've been developing. What's very interesting that it's possible to combine Java and Groovy code seamlessly within one project (== one jar) and everything works just fine.

Let's start with an example.

Assume, in our project we heavily use image processing class ImageUtils with static methods scaleHighQuality and scaleLowQuality, which perform image scaling. For testing purposes, we've included a bunch of images as resources to our jar. Let's develop test case witch goes through all image resources, scales each image (up or down) to 125x125 pixels with low- and high-quality scaling algorithms.

import org.junit.Test
import java.awt.image.BufferedImage
import javax.imageio.ImageIO

public class ImagesTestCase extends GroovyTestCase {
  private void scale( Closure c ) {
    URL location = new URL(
       getClass().getProtectionDomain().getCodeSource().getLocation(),
       ""
    )

    // Enumerate all image resources
    new File( location.toURI() ).list().each() { file ->
      if( file.toLowerCase() =~ /(png|gif|jpg)$/ ) {
        BufferedImage image = ImageIO.read( new URL( location, file ) )
        assertNotNull( "Input image is null", image )

        BufferedImage transformed = c( image );
        assertNotNull( "Transformed image is null", resized )

        assertTrue( transformed.getWidth() <= 125 )         
        assertTrue( transformed.getHeight() <= 125 )       
      }     
    }  
  }   

  @Test  public void testImagesLowQuality() {    
    scale() { image ->
      ImageUtils.scaleLowQuality( image, 125, 125 )
    }
  }

  @Test
  public void testImagesHighQuality() {
    scale() { image ->
      ImageUtils.scaleHighQuality( image, 125, 125 )
    }
  }
}

Why I'm using Groovy for that?
1) It's much faster to write a test code (simplified syntax)
2) Tests look more accurate and compact (powerful library)
3) Groovy has a bunch of modern features which make development even more pleasant (closures, regular expressions, ...)

I would say, "Developers, keep in touch with dynamic languages!"

Integration testing: building our own test sandbox (practice)

Previous post prepared some theoretical background for building our own test sandbox. In this post we'll flesh out the theory with practice. So, let's follow startup steps and confirm each one with code snippets.

1. Configure HSQLDB data source and JNDI

import org.enhydra.jdbc.standard.StandardDataSource;
...

StandardDataSource dataSource = new StandardDataSource();
dataSource.setDriverName( "org.hsqldb.jdbcDriver" );
dataSource.setUrl( "jdbc:hsqldb:mem:testdb" );
dataSource.setUser( "sa" );

In the snippet above I've used very useful XAPool project (to manipulate with JDBC data sources).

import javax.naming.Context;
import javax.naming.InitialContext;
import org.mortbay.jetty.plus.naming.NamingEntry;
import org.mortbay.jetty.plus.naming.Resource;
...

Context context = new InitialContext();
Context compCtx = ( Context )context.lookup( "java:comp" );
compCtx.createSubcontext( "env" );

// Configure JNDI
NamingEntry.setScope( NamingEntry.SCOPE_GLOBAL) ;

// This actually registers the resource in JNDI
new Resource( "jdbc/testdb", dataSource ).bindToENC();

2. Configure Hibernate

In fact, there is a trick here. I'm using AspectJ to intercept in run-time (weaver) all calls to Hibernate's class method Configuration.configure(). There is an aspect's code.

import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.hibernate.cfg.Configuration;
@Aspect
public class HibernateAspect {
@Around("execution( * org.hibernate.cfg.Configuration.configure() throws * )")
public Object reconfigure( ProceedingJoinPoint joinPoint ) throws Throwable {
Object r = ((Configuration) joinPoint.getThis()).configure( <path to configuration file> );
return r;
}
}

3. Configure Web Services for Jetty

import org.mortbay.jetty.Server;
import org.mortbay.jetty.servlet.ServletHolder;
import org.apache.axis2.transport.http.AxisServlet;
import org.mortbay.jetty.servlet.Context;
...

Server server = new Server(0);

ServletHolder axisServletholder = new ServletHolder( new AxisServlet() );
axisServletholder.setInitParameter( "axis2.xml.path", <path to axis2.xml> );
axisServletholder.setInitParameter( "axis2.repository.path", <path to WEB-INF folder> );

Context root = new Context( server, "/", Context.SESSIONS );
root.addServlet( axisServletholder, "/servlet/AxisServlet" );
root.addServlet( axisServletholder, "/services/*" );

4. Run Jetty

server.start();
int actualPort = server.getConnectors()[0].getLocalPort();

5. Export database schema (Hibernate)

import org.hibernate.tool.hbm2ddl.SchemaExport;
...

// Be sure, Hibernate uses JNDI data source name. In this test "jdbc/testdb".
// And database dialect is set to org.hibernate.dialect.HSQLDialect.
SchemaExport schemaExport = new SchemaExport( );
schemaExport.create( true, true );

6. Prepare test dataset (DbUnit)

import org.dbunit.database.IDatabaseConnection;
import org.dbunit.database.DatabaseConnection;
import org.dbunit.dataset.xml.FlatXmlDataSet;
import org.dbunit.dataset.IDataSet;
import org.dbunit.operation.DatabaseOperation;
import org.hibernate.Session;
import java.io.FileInputStream;
...

Session session = HibernateUtil.getCurrentSession();
IDataSet dataSet = new FlatXmlDataSet(new FileInputStream(<path to test dataset XML>));
IDatabaseConnection connection = new DatabaseConnection( session.connection() );
DatabaseOperation.CLEAN_INSERT.execute( connection, dataSet );

7. Run test(s) (Java or Groovy)

Let's summarize what we have at this point:
- Jetty is up and running
- Web Services are configured and deployed at: http://localhost:<actual port>/services/*
- Database schema is created and populated with test dataset

So we're ready to pull our web service methods. But for this purpose we need web service client. The first approach is to use raw SOAP requests. Second one is to generate web service (WSDL) stubs (Java). And last but not least is to use some scripting/dynamic language (like Groovy).

My choive was Groovy because:
- it's easy to consume web services with GroovyWS library
- it's easy to write test code
- it's excellent dynamic scripting language with seamless Java integration

Here is code sample:

import groovyx.net.ws.WSClient;

def proxy = new WSClient(
"http://localhost:<actual port>/services/TestService?wsdl", this.class.classLoader
);

def param = proxy.create( "testServiceNamespace.SomeType" )
param.someProperty =

def result = proxy.someTestMethod( param )
assert ( result = )

Looks really great and very promising. The only problem I've encountered that GroovyWS is started recently and is "green". Unfortunately, I've failed to test most web service methods with complex in/out parameters and return types (so I was reluctant to use WSDL generated stubs).

That's it. Our test sandbox is ready for use!

Integration testing: building our own test sandbox (theory)

To be honest, I aim for developing high-quality software. Unit tests, now BDD really helps with that. They allow us to test individual classes, modules, and event subsystems. But what about testing whole application with everything in place (integration testing)? Mostly, this is test teams responsibility. Before that, can developers be sure that application is not only built properly but foremost flows work properly as well? Sometimes it's not so obvious, but in general the answer is "yes".

Today I would like to share my own experience with building test sandbox for Axis 2 Web Services. Primarily, I tried to avoid any external dependencies (such as database, servers, etc.). Test sandbox should be fully reproducible and isolated.

Application architecture looks like in picture below.











So, our test sandbox should have:
- HTTP Server (and Servlet container) -> Jetty
- Database (in memory) -> HSQLDB

The flow to start up test sandbox looks like:
- Configure HSQLDB data source and JNDI
- Configure Hibernate
- Configure Web Services for Jetty
- Run Jetty
- Export database schema (Hibernate)
- Prepare test dataset (DbUnit)
- Run test(s) (Java or Groovy)

Also, using AspectJ weaver, we can reconfigure Hibernate with HSQLDB data source on the fly. It looks a little bit complicated but completely feasible. So, from theory to code ... (next post)

BDD in Java: practical example

Continuing last post about BDD, I would like to demonstrate an example with both xUnit and BDD tests. Because right now I'm developing mostly on Java, the JUnit and easyb are my choices for this small demo. So, let's start with our domain model.

Let's assume, we're modeling bank transfer subsystem. Needless to say, classes Bank and Account jibe well with our domain.

public class Account {
public void credit( final int amount ) throw NoEnoughMoneyException { ... }
public void debit( final int amount ) { ... }
public int getBalance() { ... }
}

public class Bank {
public void transfer( final Account from, final Account to, final int amount ) { ... }
}

These classes are so simple that I omitted all the code. Contracts only matter. The respective test case for money transfer looks like:

public class BankTestCase {
@Test( expected = NoEnoughMoneyException.class )
public void testTransfer() {
Bank bank = new Bank();
Account from = new Account( 100 ), to = new Account( 200 );
bank.transfer( from, to, 150 );
}
}

This test code is trying to provoke NoEnoughMoneyException by means of debit of the unallowed amount of money. Everything seems to be OK here. Now let's take a look how same test case looks in BDD:

scenario "Bank Transfer", {
given "an specific bank", {
bank = new Bank();
}

given "bank accounts", {
from = new Account( 100 );
to = new Account( 200 );
}

when "no enough money for transfer", {
transfer = {
bank.transfer( from, to, 150 )
}
}

then "exception should be thrown", {
ensureThrows( NoEnoughMoneyException ) {
transfer();
}
}
}


It's great, isn't it? You read it as a story.
And in fact output of this test scenario is real story:

Scenario Bank transfer
Given an specific bank
Given bank accounts
When no enough money for transfer
Then exception should be thrown

I like it. It's clear and friendly.