For hardcore server-side Java developer the only way to "speak" out to the world is by using APIs. Today's post is all about JAX-RS: writing and exposing RESTful services using Java.
But we won't do that using a traditional, heavyweight approach involving application server, WAR packaging and whatnot. Instead, we will use awesome Apache CXF framework and as always rely on Spring to wire all pieces together. And for sure we won't stop on that either as we need a web server to run our services on. Using fat or one jar concept we will embed Jetty server into our application and make our final JAR redistributable (all dependencies included) and runnable.
It's a lot of work so let's get started. As we stated above, we will use Apache CXF, Spring and Jetty as a building blocks so let's have them described in a POM file. The one additional dependency worth mentioning is excellent Jackson library for JSON processing.
It's a lot of stuff but should be pretty clear. Now, we are ready to develop our first JAX-RS services by starting with simple JAX-RS application.
package com.example.rs;
import javax.ws.rs.ApplicationPath;
import javax.ws.rs.core.Application;
@ApplicationPath( "api" )
public class JaxRsApiApplication extends Application {
}
As simple as it looks like, our application defines an /api to be the entry path for the JAX-RS services. The sample service will manage people represented by Person class.
package com.example.model;
public class Person {
private String email;
private String firstName;
private String lastName;
public Person() {
}
public Person( final String email ) {
this.email = email;
}
public String getEmail() {
return email;
}
public void setEmail( final String email ) {
this.email = email;
}
public String getFirstName() {
return firstName;
}
public String getLastName() {
return lastName;
}
public void setFirstName( final String firstName ) {
this.firstName = firstName;
}
public void setLastName( final String lastName ) {
this.lastName = lastName;
}
}
And following bare bones business service (for simplicity, no database or any other storage are included).
package com.example.services;
import java.util.ArrayList;
import java.util.Collection;
import org.springframework.stereotype.Service;
import com.example.model.Person;
@Service
public class PeopleService {
public Collection< Person > getPeople( int page, int pageSize ) {
Collection< Person > persons = new ArrayList< Person >( pageSize );
for( int index = 0; index < pageSize; ++index ) {
persons.add( new Person( String.format( "person+%d@at.com", ( pageSize * ( page - 1 ) + index + 1 ) ) ) );
}
return persons;
}
public Person addPerson( String email ) {
return new Person( email );
}
}
As you can see, we will generate a list of persons on the fly depending on the page requested. Standard Spring annotation @Service marks this class as a service bean. Our JAX-RS service PeopleRestService will use it for retrieving persons as the following code demonstrates.
package com.example.rs;
import java.util.Collection;
import javax.inject.Inject;
import javax.ws.rs.DefaultValue;
import javax.ws.rs.FormParam;
import javax.ws.rs.GET;
import javax.ws.rs.PUT;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.QueryParam;
import com.example.model.Person;
import com.example.services.PeopleService;
@Path( "/people" )
public class PeopleRestService {
@Inject private PeopleService peopleService;
@Produces( { "application/json" } )
@GET
public Collection< Person > getPeople( @QueryParam( "page") @DefaultValue( "1" ) final int page ) {
return peopleService.getPeople( page, 5 );
}
@Produces( { "application/json" } )
@PUT
public Person addPerson( @FormParam( "email" ) final String email ) {
return peopleService.addPerson( email );
}
}
Though simple, this class needs more explanations. First of all, we want to expose our RESTful service to /people endpoint. Combining it with /api (where our JAX-RS application resides), it gives as the /api/people as qualified path.
Now, whenever someone issues HTTP GET to this path, the method getPeople should be invoked. This method accepts optional parameter page (with default value 1) and returns list of persons as JSON. In turn, if someone issues HTTP PUT to the same path, the method addPerson should be invoked (with required parameter email) and return new person as a JSON.
Now let's take a look on Spring configuration, the core of our application.
package com.example.config;
import java.util.Arrays;
import javax.ws.rs.ext.RuntimeDelegate;
import org.apache.cxf.bus.spring.SpringBus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.codehaus.jackson.jaxrs.JacksonJsonProvider;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import com.example.rs.JaxRsApiApplication;
import com.example.rs.PeopleRestService;
import com.example.services.PeopleService;
@Configuration
public class AppConfig {
@Bean( destroyMethod = "shutdown" )
public SpringBus cxf() {
return new SpringBus();
}
@Bean
public Server jaxRsServer() {
JAXRSServerFactoryBean factory = RuntimeDelegate.getInstance().createEndpoint( jaxRsApiApplication(), JAXRSServerFactoryBean.class );
factory.setServiceBeans( Arrays.< Object >asList( peopleRestService() ) );
factory.setAddress( "/" + factory.getAddress() );
factory.setProviders( Arrays.< Object >asList( jsonProvider() ) );
return factory.create();
}
@Bean
public JaxRsApiApplication jaxRsApiApplication() {
return new JaxRsApiApplication();
}
@Bean
public PeopleRestService peopleRestService() {
return new PeopleRestService();
}
@Bean
public PeopleService peopleService() {
return new PeopleService();
}
@Bean
public JacksonJsonProvider jsonProvider() {
return new JacksonJsonProvider();
}
}
It doesn't look complicated but a lot happens under the hood. Let's dissect it into the peices. The two key component here are the factory JAXRSServerFactoryBean which does all heavy lifting for configuring our instance of JAX-RS server, and SpringBus instance which seamlessly glues Spring and Apache CXF together. All other components represent regular Spring beans.
What's not on a picture yet is embedding Jetty web server instance. Our main application class Starter does exactly that.
package com.example;
import org.apache.cxf.transport.servlet.CXFServlet;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.ServletContextHandler;
import org.eclipse.jetty.servlet.ServletHolder;
import org.springframework.web.context.ContextLoaderListener;
import org.springframework.web.context.support.AnnotationConfigWebApplicationContext;
import com.example.config.AppConfig;
public class Starter {
public static void main( final String[] args ) throws Exception {
Server server = new Server( 8080 );
// Register and map the dispatcher servlet
final ServletHolder servletHolder = new ServletHolder( new CXFServlet() );
final ServletContextHandler context = new ServletContextHandler();
context.setContextPath( "/" );
context.addServlet( servletHolder, "/rest/*" );
context.addEventListener( new ContextLoaderListener() );
context.setInitParameter( "contextClass", AnnotationConfigWebApplicationContext.class.getName() );
context.setInitParameter( "contextConfigLocation", AppConfig.class.getName() );
server.setHandler( context );
server.start();
server.join();
}
}
Looking through this code uncovers that we are running Jetty server instance on port 8080, we are configuring Apache CXF servlet to handle all request at /rest/* path (which together with our JAX-RS application and service gives us the /rest/api/people), we are adding Spring context listener parametrized with the configuration we have defined above and finally we are starting server up. What we have at this point is full-blown web server hosting our JAX-RS services. Let's see it in action. Firstly, let's package it as single, runnable and redistributable fat or one jar:
mvn clean package
Let's pick up the bits from the target folder and run them:
2013-01-19 11:43:08.636:INFO:oejs.Server:jetty-8.1.8.v20121106
2013-01-19 11:43:08.698:INFO:/:Initializing Spring root WebApplicationContext
Jan 19, 2013 11:43:08 AM org.springframework.web.context.ContextLoader initWebApplicationContext
INFO: Root WebApplicationContext: initialization started
Jan 19, 2013 11:43:08 AM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing Root WebApplicationContext: startup date [Sat Jan 19 11:43:08 EST 2013]; root of context hierarchy
Jan 19, 2013 11:43:08 AM org.springframework.context.annotation.ClassPathScanningCandidateComponentProvider registerDefaultFilters
INFO: JSR-330 'javax.inject.Named' annotation found and supported for component scanning
Jan 19, 2013 11:43:08 AM org.springframework.web.context.support.AnnotationConfigWebApplicationContext loadBeanDefinitions
INFO: Successfully resolved class for [com.example.config.AppConfig]
Jan 19, 2013 11:43:09 AM org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor
INFO: JSR-330 'javax.inject.Inject' annotation found and supported for autowiring
Jan 19, 2013 11:43:09 AM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@1f8166e5: defining beans [org.springframework.context.annotation.internal
ConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProces
sor,org.springframework.context.annotation.internalCommonAnnotationProcessor,appConfig,org.springframework.context.annotation.ConfigurationClassPostProcessor.importAwareProcessor,c
xf,jaxRsServer,jaxRsApiApplication,peopleRestService,peopleService,jsonProvider]; root of factory hierarchy
Jan 19, 2013 11:43:10 AM org.apache.cxf.endpoint.ServerImpl initDestination
INFO: Setting the server's publish address to be /api
Jan 19, 2013 11:43:10 AM org.springframework.web.context.ContextLoader initWebApplicationContext
INFO: Root WebApplicationContext: initialization completed in 2227 ms
2013-01-19 11:43:10.957:INFO:oejsh.ContextHandler:started o.e.j.s.ServletContextHandler{/,null}
2013-01-19 11:43:11.019:INFO:oejs.AbstractConnector:Started SelectChannelConnector@0.0.0.0:8080
Having our server up and running, let's issue some HTTP requests to it so to be sure everything works just as we expected:
Before ending the post, I would like to mention one great project, Dropwizard, which uses quite similar concepts but pushes it to the level of excellent, well-designed framework, thanks to Yammer guys for that.
Among plenty of useful classes which Java provides for concurrency support, there is one I would like to talk about: SynchronousQueue. In particular, I would like to walk through Producer / Consumer implementation using handy SynchronousQueue as an exchange mechanism.
It might not sound clear why to use this type of queue for producer / consumer communication unless we look under the hood of SynchronousQueue implementation. It turns out that it's not really a queue as we used to think about queues. The analogy would be just a collection containing at most one element.
Why it's useful? Well, there are several reasons. From producer's point of view, only one element (or message) could be stored into the queue. In order to proceed with the next element (or message), the producer should wait till consumer consumes the one currently in the queue. From consumer's point of view, it just polls the queue for next element (or message) available. Quite simple, but the great benefit is: producer cannot send messages faster than consumer can process them.
Here is one of the use cases I encountered recently: compare two database tables (possibly just huge) and detect if those contain different data or data is the same (copy). The SynchronousQueue is quite a handy tool for this problem: it allows to handle each table in own thread as well as compensate the possible timeouts / latency while reading from two different databases.
Let's start by defining our compare function which accepts source and destination data sources as well as a table name (to compare). I am using quite useful JdbcTemplate class from Spring framework as it extremely well abstract all the boring details dealing with connections and prepared statements.
public boolean compare( final DataSource source, final DataSource destination, final String table ) {
final JdbcTemplate from = new JdbcTemplate( source );
final JdbcTemplate to = new JdbcTemplate( destination );
}
Before doing any actual data comparison, it's a good idea to compare table's row count of the source and destination databases:
Now, at least knowing that table contains same number of rows in both databases, we can start with data comparison. The algorithm is very simple:
create a separate thread for source (producer) and destination (consumer) databases
producer thread reads single row from the table and puts it into the SynchronousQueue
consumer thread also reads single row from the table, then asks queue for the available row to compare (waits if necessary) and lastly compare two result sets
Using another great part Java concurrent utilities for thread pooling, let's define a thread pool with fixed amount of threads (2).
final ExecutorService executor = Executors.newFixedThreadPool( 2 );
final SynchronousQueue< List< ? > > resultSets = new SynchronousQueue< List< ? > >();
Following the described algorithm, the producer functionality could be represented as a single callable:
Callable< Void > producer = new Callable< Void >() {
@Override
public Void call() throws Exception {
from.query( "SELECT * FROM " + table,
new RowCallbackHandler() {
@Override
public void processRow(ResultSet rs) throws SQLException {
try {
List< ? > row = ...; // convert ResultSet to List
if( !resultSets.offer( row, 2, TimeUnit.MINUTES ) ) {
throw new SQLException( "Having more data but consumer has already completed" );
}
} catch( InterruptedException ex ) {
throw new SQLException( "Having more data but producer has been interrupted" );
}
}
}
);
return null;
}
};
The code is a bit verbose due to Java syntax but it doesn't do much actually. Every result set read from the table producer converts to a list (implementation has been omitted as it's a boilerplate) and puts in a queue (offer). If queue is not empty, producer is blocked waiting for consumer to finish his work. The consumer, respectively, could be represented as a following callable:
Callable< Void > consumer = new Callable< Void >() {
@Override
public Void call() throws Exception {
to.query( "SELECT * FROM " + table,
new RowCallbackHandler() {
@Override
public void processRow(ResultSet rs) throws SQLException {
try {
List< ? > source = resultSets.poll( 2, TimeUnit.MINUTES );
if( source == null ) {
throw new SQLException( "Having more data but producer has already completed" );
}
List< ? > destination = ...; // convert ResultSet to List
if( !source.equals( destination ) ) {
throw new SQLException( "Row data is not the same" );
}
} catch ( InterruptedException ex ) {
throw new SQLException( "Having more data but consumer has been interrupted" );
}
}
}
);
return null;
}
};
The consumer does a reverse operation on the queue: instead of putting data it pulls it (poll) from the queue. If queue is empty, consumer is blocked waiting for producer to publish next row. The part which is left is only submitting those callables for execution. Any exception returned by the Future's get method indicates that table doesn't contain the same data (or there are issue with getting data from database):
YAML is well-known format within Ruby community, quite widely used for a long time now. But we as Java developers mostly deal with property files and XMLs in case we need some configuration for our apps. How many times we needed to express complicated configuration by inventing our own XML schema or imposing property names convention?
Though JSON is becoming a popular format for web applications, using JSON files to describe the configuration is a bit cumbersome and, in my opinion, is not as expressive as YAML. Let's see what YAML can do for us to make our life easier.
For sure, let's start with the problem. In order for our application to function properly, we need to feed it following data somehow:
version and release date
database connection parameters
list of supported protocols
list of users with their passwords
This list of parameters sounds a bit weird, but the purpose is to demonstrate different data types in work: strings, numbers, dates, lists and maps. The Java model consists of two simple classes: Connection
package com.example.yaml;
public final class Connection {
private String url;
private int poolSize;
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url;
}
public int getPoolSize() {
return poolSize;
}
public void setPoolSize(int poolSize) {
this.poolSize = poolSize;
}
@Override
public String toString() {
return String.format( "'%s' with pool of %d", getUrl(), getPoolSize() );
}
}
and Configuration, both are typical Java POJOs, verbose because of property setters and getters (we get used to it, right?).
package com.example.yaml;
import static java.lang.String.format;
import java.util.Date;
import java.util.List;
import java.util.Map;
public final class Configuration {
private Date released;
private String version;
private Connection connection;
private List< String > protocols;
private Map< String, String > users;
public Date getReleased() {
return released;
}
public String getVersion() {
return version;
}
public void setReleased(Date released) {
this.released = released;
}
public void setVersion(String version) {
this.version = version;
}
public Connection getConnection() {
return connection;
}
public void setConnection(Connection connection) {
this.connection = connection;
}
public List< String > getProtocols() {
return protocols;
}
public void setProtocols(List< String > protocols) {
this.protocols = protocols;
}
public Map< String, String > getUsers() {
return users;
}
public void setUsers(Map< String, String > users) {
this.users = users;
}
@Override
public String toString() {
return new StringBuilder()
.append( format( "Version: %s\n", version ) )
.append( format( "Released: %s\n", released ) )
.append( format( "Connecting to database: %s\n", connection ) )
.append( format( "Supported protocols: %s\n", protocols ) )
.append( format( "Users: %s\n", users ) )
.toString();
}
}
Now, as model is quite clear, let us try to express it as the human being normally does it. Looking back to our list of required configuration, let's try to write it down one by one.
Please notice the usage of Java 1.7, the language extensions and additional libraries simplify a lot of regular tasks as we could see looking into YamlConfigRunner:
The code snippet here loads the configuration from file (args[ 0 ]), tries to parse it and fill up the Configuration class with meaningful data using JavaBeans conventions, converting to the declared types where possible. Running this class with sample.yml as an argument generates the following output:
Version: 1.0
Released: Thu Nov 29 19:00:00 EST 2012
Connecting to database: 'jdbc:mysql://localhost:3306/db' with pool of 5
Supported protocols: [http, https]
Users: {tom=passwd, bob=passwd}
Totally identical to the values we have configured!
Today's post will target an interesting but quite powerful concept: packing your application as single, runnable JAR file, also known as one or fat JAR.
We get used to large WAR archives which contain all dependencies packed together under some common folder structure. With JAR-like packaging the story is a bit different: in order to make your application runnable (via java -jar) all dependencies should be provided over classpath parameter or environment variable. Usually it means there would be some lib folder with all dependencies and some runnable script which will do the job to construct classpath and run JVM. Maven Assembly plugin is well know for making such kind of application distribution.
A slightly different approach would be to package all your application dependencies to the same JAR file and make it runnable without any additional parameters or scripting required. Sounds great but ... it won't work unless you add some magic: meet One-JAR project.
Let's briefly outline the problem: we are writing a stand-alone Spring application which should be runnable just by typing java -jar <our-app.jar>.
As always, let's start with our POM file, which will be pretty simple
Our sample application will bootstrap Spring context, get some bean instance and call a method on it. Our bean is called SimpleBean and looks like:
package com.example;
public class SimpleBean {
public void print() {
System.out.println( "Called from single JAR!" );
}
}
Falling in love with Spring Java configuration, let us define our context as annotated AppConfig POJO:
package com.example.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import com.example.SimpleBean;
@Configuration
public class AppConfig {
@Bean
public SimpleBean simpleBean() {
return new SimpleBean();
}
}
And finally, our application Starter with main():
package com.example;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import com.example.config.AppConfig;
public class Starter {
public static void main( final String[] args ) {
ApplicationContext context = new AnnotationConfigApplicationContext( AppConfig.class );
SimpleBean bean = context.getBean( SimpleBean.class );
bean.print();
}
}
Adding our main class to META-INF/MANIFEST.MF allows to leverage Java capabilities to run JAR file without explicitly specifying class with main() method. Maven JAR plugin can help us with that.
Trying to run java -jar spring-one-jar-0.0.1-SNAPSHOT.jar will print the exception to the console: java.lang.NoClassDefFoundError. The reason is pretty straightforward: even such a simple application as this one already required following libraries to be in classpath.
As the result, there will be another artifact available in the target folder, postfixed with one-jar: spring-one-jar-0.0.1-SNAPSHOT.one-jar.jar. Running this one with java -jar spring-one-jar-0.0.1-SNAPSHOT.one-jar.jar will print to the console:
Called from single JAR!
Fully runnable Java application as single, redistributable JAR file! The last comment: though our application looks pretty simple, One-JAR works perfectly for complex, large applications as well without any issues. Please, add it to your toolbox, it's really useful tool to have.
Continuing to discover the powerful set of Redis features, the one worth mentioning about is out of the box support of pub/sub messaging.
Pub/Sub messaging is essential part of many software architectures. Some software systems demand from messaging solution to provide high-performance, scalability, queues persistence and durability, fail-over support, transactions, and many more nice-to-have features, which in Java world mostly always leads to using one of JMS implementation providers. In my previous projects I have actively used Apache ActiveMQ (now moving towards Apache ActiveMQ Apollo). Though it's a great implementation, sometimes I just needed simple queuing support and Apache ActiveMQ just looked overcomplicated for that.
Alternatives? Please welcome Redis pub/sub! If you are already using Redis as key/value store, few additional lines of configuration will bring pub/sub messaging to your application in no time.
Spring Data Redis project abstracts very well Redis pub/sub API and provides the model so familiar to everyone who uses Spring capabilities to integrate with JMS.
As always, let's start with the POM configuration file. It's pretty small and simple, includes necessary Spring dependencies, Spring Data Redis and Jedis, great Java client for Redis.
Moving on to configuring Spring context, let's understand what we need to have in order for a publisher to publish some messages and for a consumer to consume them. Knowing the respective Spring abstractions for JMS will help a lot with that.
we need connection factory -> JedisConnectionFactory
we need a template for publisher to publish messages -> RedisTemplate
we need a message listener for consumer to consume messages -> RedisMessageListenerContainer
Using Spring Java configuration, let's describe our context:
package com.example.redis.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.ChannelTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.listener.adapter.MessageListenerAdapter;
import org.springframework.data.redis.serializer.GenericToStringSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.scheduling.annotation.EnableScheduling;
import com.example.redis.IRedisPublisher;
import com.example.redis.impl.RedisMessageListener;
import com.example.redis.impl.RedisPublisherImpl;
@Configuration
@EnableScheduling
public class AppConfig {
@Bean
JedisConnectionFactory jedisConnectionFactory() {
return new JedisConnectionFactory();
}
@Bean
RedisTemplate< String, Object > redisTemplate() {
final RedisTemplate< String, Object > template = new RedisTemplate< String, Object >();
template.setConnectionFactory( jedisConnectionFactory() );
template.setKeySerializer( new StringRedisSerializer() );
template.setHashValueSerializer( new GenericToStringSerializer< Object >( Object.class ) );
template.setValueSerializer( new GenericToStringSerializer< Object >( Object.class ) );
return template;
}
@Bean
MessageListenerAdapter messageListener() {
return new MessageListenerAdapter( new RedisMessageListener() );
}
@Bean
RedisMessageListenerContainer redisContainer() {
final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory( jedisConnectionFactory() );
container.addMessageListener( messageListener(), topic() );
return container;
}
@Bean
IRedisPublisher redisPublisher() {
return new RedisPublisherImpl( redisTemplate(), topic() );
}
@Bean
ChannelTopic topic() {
return new ChannelTopic( "pubsub:queue" );
}
}
Very easy and straightforward. The presence of @EnableScheduling annotation is not necessary and is required only for our publisher implementation: the publisher will publish a string message every 100 ms.
package com.example.redis.impl;
import java.util.concurrent.atomic.AtomicLong;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.listener.ChannelTopic;
import org.springframework.scheduling.annotation.Scheduled;
import com.example.redis.IRedisPublisher;
public class RedisPublisherImpl implements IRedisPublisher {
private final RedisTemplate< String, Object > template;
private final ChannelTopic topic;
private final AtomicLong counter = new AtomicLong( 0 );
public RedisPublisherImpl( final RedisTemplate< String, Object > template,
final ChannelTopic topic ) {
this.template = template;
this.topic = topic;
}
@Scheduled( fixedDelay = 100 )
public void publish() {
template.convertAndSend( topic.getTopic(), "Message " + counter.incrementAndGet() +
", " + Thread.currentThread().getName() );
}
}
And finally our message listener implementation (which just prints message on a console).
package com.example.redis.impl;
import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.connection.MessageListener;
public class RedisMessageListener implements MessageListener {
@Override
public void onMessage( final Message message, final byte[] pattern ) {
System.out.println( "Message received: " + message.toString() );
}
}
Awesome, just two small classes, one configuration to wire things together and we have full pub/sub messaging support in our application! Let's run the application as standalone ...
package com.example.redis;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import com.example.redis.config.AppConfig;
public class RedisPubSubStarter {
public static void main(String[] args) {
new AnnotationConfigApplicationContext( AppConfig.class );
}
}
Today's post is about BTrace which I am considering as a hidden gem for Java developer.
BTrace is a safe, dynamic tracing tool for the Java platform. BTrace can be used to dynamically trace a running Java program (similar to DTrace for OpenSolaris applications and OS).
Shortly, the tool allows to inject tracing points without restarting or reconfiguring your Java application while it's running. Moreover, though there are several ways to do that, the one I would like to discuss today is using JVisualVM tool from standard JDK bundle.
What is very cool, BTrace itself uses Java language to define injection trace points. The approach looks very familiar if you ever did aspect-oriented programming (AOP).
So let's get started with a problem: we have an application which uses one of the NoSQL databases (f.e., let it be MongoDB) and suddenly starts to experience significant performance slowdown. Developers suspect that application runs too many queries or updates but cannot say it with confidence. Here BTrace can help.
First thing first, let's run JVisualVM and install BTrace plugin:
JVisualVM should be restarted in order for plugin to appear. Now, while our application is up and running, let's right click on it in JVisualVM applications tree:
The following very intuitive BTrace editor (with simple toolbar) should appear:
This is a place where tracing instrumentation could be defined and dynamically injected into the running application. BTrace has a very rich model in order to define what exactly should be traced: methods, constructors, method returns, errors, .... Also it supports aggregations out of the box so it quite easy to collect a bunch of metrics while application is running. For our problem, we would like to see which methods related to MongoDB are being executed.
As my application uses Spring Data MongoDB, I am interested in which methods of any implementation of org.springframework.data.mongodb.core.MongoOperations interface are being called by application and how long every call takes. So I have defined a very simple BTrace script:
Let me explain briefly what I am doing here. Basically, I would like to know when any method of any implementation of org.springframework.data.mongodb.core.MongoOperations is called (onMongo marks that) and duration of the call (onMongoReturn marks that in turn). Thread-local variable method holds full qualified method name (with a class), while thanks to useful BTrace predefined annotation, duration parameter holds the method execution time (in nanoseconds). Though it's pure Java, BTrace allows only small subset of Java classes to be used. It's not a problem as com.sun.btrace.BTraceUtils class provides a lot of useful methods (f.e., strcat) to fill the gaps. Running this script produces following output:
** Compiling the BTrace script ...
*** Compiled
** Instrumenting 1 classes ...
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 25ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 3ms
Method org.springframework.data.mongodb.core.MongoTemplate::getDb executed in 22ms
Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 19ms
Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 1ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 3ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::getDb executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 1ms
Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 6ms
Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 1ms
Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 0ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 1ms
Method org.springframework.data.mongodb.core.MongoTemplate::getDb executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 1ms
Method org.springframework.data.mongodb.core.MongoTemplate::prepareCollection executed in 6ms
Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 1ms
Method org.springframework.data.mongodb.core.MongoTemplate::access$100 executed in 0ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 2ms
Method org.springframework.data.mongodb.core.MongoTemplate::maybeEmitEvent executed in 1ms
...
As you can see, output contains bunch of inner classes which could easily be eliminated by providing more precise method name templates (or maybe even tracing MongoDB driver instead).
I have just started to discover BTrace but I definitely see a great value for me as a developer from using this awesome tool. Thanks to BTrace guys!
As NoSQL solutions are getting more and more popular for many kind of problems, more often the modern projects consider to use some (or several) of NoSQLs instead (or side-by-side) of traditional RDBMS. I have already covered my experience with MongoDB in this, this and this posts. In this post I would like to switch gears a bit towards Redis, an advanced key-value store.
Aside from very rich key-value semantics, Redis also supports pub-sub messaging and transactions. In this post I am going just to touch the surface and demonstrate how simple it is to integrate Redis into your Spring application.
As always, we will start with Maven POM file for our project:
Spring Data Redis is the another project under Spring Data umbrella which provides seamless injection of Redis into your application. The are several Redis clients for Java and I have chosen the Jedis as it is stable and recommended by Redis team at the moment of writing this post.
We will start with simple configuration and introduce the necessary components first. Then as we move forward, the configuration will be extended a bit to demonstrated pub-sub capabilities. Thanks to Java config support, we will create the configuration class and have all our dependencies strongly typed, no XML anymore:
package com.example.redis.config;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.connection.jedis.JedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.serializer.GenericToStringSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
@Configuration
public class AppConfig {
@Bean
JedisConnectionFactory jedisConnectionFactory() {
return new JedisConnectionFactory();
}
@Bean
RedisTemplate< String, Object > redisTemplate() {
final RedisTemplate< String, Object > template = new RedisTemplate< String, Object >();
template.setConnectionFactory( jedisConnectionFactory() );
template.setKeySerializer( new StringRedisSerializer() );
template.setHashValueSerializer( new GenericToStringSerializer< Object >( Object.class ) );
template.setValueSerializer( new GenericToStringSerializer< Object >( Object.class ) );
return template;
}
}
That's basically everything we need assuming we have single Redis server up and running on localhost with default configuration. Let's consider several common uses cases: setting a key to some value, storing the object and, finally, pub-sub implementation. Storing and retrieving a key/value pair is very simple:
@Autowired private RedisTemplate< String, Object > template;
public Object getValue( final String key ) {
return template.opsForValue().get( key );
}
public void setValue( final String key, final String value ) {
template.opsForValue().set( key, value );
}
Optionally, the key could be set to expire (yet another useful feature of Redis), f.e. let our keys expire in 1 second:
public void setValue( final String key, final String value ) {
template.opsForValue().set( key, value );
template.expire( key, 1, TimeUnit.SECONDS );
}
Arbitrary objects could be saved into Redis as hashes (maps), f.e. let save instance of some class User
public class User {
private final Long id;
private String name;
private String email;
// Setters and getters are omitted for simplicity
}
public void setUser( final User user ) {
final String key = String.format( "user:%s", user.getId() );
final Map< String, Object > properties = new HashMap< String, Object >();
properties.put( "id", user.getId() );
properties.put( "name", user.getName() );
properties.put( "email", user.getEmail() );
template.opsForHash().putAll( key, properties);
}
Respectively, object could easily be inspected and retrieved using the id.
public User getUser( final Long id ) {
final String key = String.format( "user:%s", id );
final String name = ( String )template.opsForHash().get( key, "name" );
final String email = ( String )template.opsForHash().get( key, "email" );
return new User( id, name, email );
}
There are much, much more which could be done using Redis, I highly encourage to take a look on it. It surely is not a silver bullet but could solve many challenging problems very easy. Finally, let me show how to use a pub-sub messaging with Redis. Let's add a bit more configuration here (as part of AppConfig class):
@Bean
MessageListenerAdapter messageListener() {
return new MessageListenerAdapter( new RedisMessageListener() );
}
@Bean
RedisMessageListenerContainer redisContainer() {
final RedisMessageListenerContainer container = new RedisMessageListenerContainer();
container.setConnectionFactory( jedisConnectionFactory() );
container.addMessageListener( messageListener(), new ChannelTopic( "my-queue" ) );
return container;
}
The style of message listener definition should look very familiar to Spring users: generally, the same approach we follow to define JMS message listeners. The missed piece is our RedisMessageListener class definition:
package com.example.redis.impl;
import org.springframework.data.redis.connection.Message;
import org.springframework.data.redis.connection.MessageListener;
public class RedisMessageListener implements MessageListener {
@Override
public void onMessage(Message message, byte[] paramArrayOfByte) {
System.out.println( "Received by RedisMessageListener: " + message.toString() );
}
}
Now, when we have our message listener, let see how we could push some messages into the queue using Redis. As always, it's pretty simple:
@Autowired private RedisTemplate< String, Object > template;
public void publish( final String message ) {
template.execute(
new RedisCallback< Long >() {
@SuppressWarnings( "unchecked" )
@Override
public Long doInRedis( RedisConnection connection ) throws DataAccessException {
return connection.publish(
( ( RedisSerializer< String > )template.getKeySerializer() ).serialize( "queue" ),
( ( RedisSerializer< Object > )template.getValueSerializer() ).serialize( message ) );
}
}
);
}
That's basically it for very quick introduction but definitely enough to fall in love with Redis.
Seasoned software developer with a great passion to code. I am extensively working with JVM platform using Java, Groovy, Scala as well as other languages and technologies (Ruby, Grails, Play!, Akka, MySQL, PostreSQL, MongoDB, Redis, JUnit, ...)
Posts
