Showing posts with label web api. Show all posts
Showing posts with label web api. Show all posts

Wednesday, September 30, 2020

For gourmets and practioners: pick your flavour of the reactive stack with JAX-RS and Apache CXF

When JAX-RS 2.1 specification was released back in 2017, one of its true novelties was the introduction of the reactive API extensions. The industry has acknowledged the importance of the modern programming paradigms and specification essentially mandated the first-class support of the asynchronous and reactive programming for the Client API.

But what about the server side? It was not left out, the JAX-RS 2.1 asynchronous processing model has been enriched with Java 8's CompletionStage support, certainly a step in a right direction. Any existing REST web APIs built on top of the JAX-RS 2.1 implementation (like Apache CXF for example) could benefit from such enhancements right away.

import java.util.concurrent.CompletableFuture;
import java.util.concurrent.CompletionStage;

@Service
@Path("/people")
public class PeopleRestService {
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public CompletionStage<List<Person>> getPeople() {
        return CompletableFuture
        	.supplyAsync(() -> Arrays.asList(new Person("a@b.com", "Tom", "Knocker")));
    }
}

Udoubtedly, CompletionStage and CompletableFuture are powerful tools but not without own quirks and limitations. The Reactive Streams specification and a number of its implementations offer a considerably better glimpse on how asynchronous and reactive programming should look like on JVM. With that, the logical question pops up: could your JAX-RS web services and APIs take advantage of the modern reactive libraries? And if the answer is positive, what does it take?

If your bets are on Apache CXF, you are certainly well positioned. The latest Apache CXF 3.2.14 / 3.3.7 / 3.4.0 release trains bring a comprehesive support of RxJava3, RxJava2 and Project Reactor. Along this post we are going to see how easy it is to plug your favorite reactive library in and place it at the forefront of your REST web APIs and services.

Since the most applications and services on the JVM are built on top of excellent Spring framework and Spring Boot, we will be developing the reference implementations using those as a foundation. The Spring Boot starter which comes along with Apache CXF distribution is taking care of most of the boring wirings you would have needed to do otherwise.

<dependency>
	<groupId>org.apache.cxf</groupId>
	<artifactId>cxf-spring-boot-starter-jaxrs</artifactId>
	<version>3.4.0</version>
</dependency>

The Project Reactor is the number one choice as the reactive foundation for Spring-based applications and services, so let us just start from that.

<dependency>
	<groupId>org.apache.cxf</groupId>
	<artifactId>cxf-rt-rs-extension-reactor</artifactId>
	<version>3.4.0</version>
</dependency>

Great, believe it or not, we are mostly done here. In order to teach Apache CXF to understand Project Reactor types like Mono or/and Flux we need to tune the configuration just a bit using ReactorCustomizer instance.

import org.apache.cxf.Bus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.apache.cxf.jaxrs.reactor.server.ReactorCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;

@Configuration
public class AppConfig {
    @Bean
    public Server rsServer(Bus bus, PeopleRestService service) {
        final JAXRSServerFactoryBean bean = new JAXRSServerFactoryBean();
        bean.getProperties(true).put("useStreamingSubscriber", true);
        bean.setBus(bus);
        bean.setAddress("/");
        bean.setServiceBean(service);
        bean.setProvider(new JacksonJsonProvider());
        new ReactorCustomizer().customize(bean);
        return bean.create();
    }
}

With such customization in-place, our JAX-RS web services and APIs could freely utilize Project Reactor primitives in a streaming fashion, for example.

import reactor.core.publisher.Flux;

@Service
@Path("/people")
public class PeopleRestService {
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Flux<Person> getPeople() {
        return Flux.just(new Person("a@b.com", "Tom", "Knocker"));
    }
}

As you probably noticed, the implementation purposely does not do anything complicated. However, once the reactive types are put at work, you could unleash the full power of the library of your choice (and Project Reactor is really good at that).

Now, when you undestand the principle, it comes the turn of the RxJava3, the last generation of the pioneering reactive library for the JVM platform.

<dependency>
	<groupId>org.apache.cxf</groupId>
	<artifactId>cxf-rt-rs-extension-rx3</artifactId>
	<version>3.4.0</version>
</dependency>

The configuration tuning is mostly identical to the one we have seen with Project Reactor, the customizer instance, ReactiveIOCustomizer, is all that changes.

import org.apache.cxf.Bus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.apache.cxf.jaxrs.rx3.server.ReactiveIOCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;

@Configuration
public class AppConfig {
    @Bean
    public Server rsServer(Bus bus, PeopleRestService service) {
        final JAXRSServerFactoryBean bean = new JAXRSServerFactoryBean();
        bean.getProperties(true).put("useStreamingSubscriber", true);
        bean.setBus(bus);
        bean.setAddress("/");
        bean.setServiceBean(service);
        bean.setProvider(new JacksonJsonProvider());
        new ReactiveIOCustomizer().customize(bean);
        return bean.create();
    }
}

The list of supported types includes Flowable, Single and Observable, the equivalent implementation in terms of RxJava3 primitives may look like this.


import io.reactivex.rxjava3.core.Flowable;

@Service
@Path("/people")
public class PeopleRestService {
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Flowable<Person> getPeople() {
        return Flowable.just(new Person("a@b.com", "Tom", "Knocker"));
    }
}

Pretty simple, isn't it? If you stuck with an older generation, RxJava2, nothing to worry about, Apache CXF has you covered.

<dependency>
	<groupId>org.apache.cxf</groupId>
	<artifactId>cxf-rt-rs-extension-rx2</artifactId>
	<version>3.4.0</version>
</dependency>

The same configuration trick with applying the customizer (which may look annoying at this point to be fair) is all that is required.

import org.apache.cxf.Bus;
import org.apache.cxf.endpoint.Server;
import org.apache.cxf.jaxrs.JAXRSServerFactoryBean;
import org.apache.cxf.jaxrs.rx2.server.ReactiveIOCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider;

@Configuration
public class AppConfig {
    @Bean
    public Server rsServer(Bus bus, PeopleRestService service) {
        final JAXRSServerFactoryBean bean = new JAXRSServerFactoryBean();
        bean.getProperties(true).put("useStreamingSubscriber", true);
        bean.setBus(bus);
        bean.setAddress("/");
        bean.setServiceBean(service);
        bean.setProvider(new JacksonJsonProvider());
        new ReactiveIOCustomizer().customize(bean);
        return bean.create();
    }
}

And we are good to go, ready to use the familiar reactive types Observable, Flowable and Single.

import io.reactivex.Flowable;
import io.reactivex.Observable;

@Service
@Path("/people")
public class PeopleRestService {
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Observable<Person> getPeople() {
        return Flowable
            .just(new Person("a@b.com", "Tom", "Knocker"))
            .toObservable();
    }
}

Last but not least, if you happens to be using the first generation of RxJava, it is also available with Apache CXF but certainly not recommended for production (as it has EOLed a couple of years ago).

Reactive programming paradigm is steadily getting more and more traction. It is great to see that the ecosystem embraces that and frameworks like Apache CXF are not an exception. If you are looking for robust foundation to build reactive and/or asynchronous REST web APIs on JVM, Apache CXF is worth considering, please give it a try!

The complete source code is available on Github.

Saturday, May 11, 2019

When HTTP status code is not enough: tackling web APIs error reporting

One area of the RESTful web APIs design, quite frequently overlooked, is how to report errors and problems, either related to business or application. The proper usage of the HTTP status codes comes to mind first, and although quite handy, often it is not informative enough. Let us take 400 Bad Request for example. Yes, it clearly states that the request is problematic, but what exactly is wrong?

The RESTful architectural style does not dictate what should be done in this case and so everyone is inventing its own styles, conventions and specifications. It could be as simple as including error message into the response or as shortsighted as copy/pasting long stack traces (in case of Java or .NET, to name a few cultprits). There is no shortage of ideas but luckily, we have at least some guidance available in the form of RFC 7807: Problem Details for HTTP APIs. Despite the fact that it is not an official specification but a draft (still), it outlines the good common principles on the problem at hand and this is what we are going to talk about in this post.

In the nutshell, RFC 7807: Problem Details for HTTP APIs just proposes the error or problem representation (in JSON or XML formats) which may include at least the following details:

  • type - A URI reference that identifies the problem type
  • title - A short, human-readable summary of the problem type
  • status - The HTTP status code
  • detail - A human-readable explanation specific to this occurrence of the problem
  • instance - A URI reference that identifies the specific occurrence of the problem
More importantly, the problem type definitions may extend the problem details object with additional members, contributing to the ones above. As you see, it looks dead simple from the implementation perspective. Even better, thanks to Zalando, we already have the RFC 7807: Problem Details for HTTP APIs implementation for Java (and Spring Web in particular). So ... let us give it a try!

Our imaginary People Management web API is going to be built using the state of the art technology stack, Spring Boot and Apache CXF, the popular web services framework and JAX-RS 2.1 implementation. To keep it somewhat simple, there are only two endpoints which are exposed: registration and lookup by person identifier.

Sweeping aside the tons of issues and business constraints you may run into while developing the real-world services, even with this simple API a few things may go wrong. The first problem we age going to tackle is what if the person you are looking for is not registered yet? Looks like a fit for 404 Not Found, right? Indeed, let us start with our first problem, PersonNotFoundProblem!

public class PersonNotFoundProblem extends AbstractThrowableProblem {
    private static final long serialVersionUID = 7662154827584418806L;
    private static final URI TYPE = URI.create("http://localhost:21020/problems/person-not-found");
    
    public PersonNotFoundProblem(final String id, final URI instance) {
        super(TYPE, "Person is not found", Status.NOT_FOUND, 
            "Person with identifier '" + id + "' is not found", instance, 
                null, Map.of("id", id));
    }
}

It resembles a lot the typical Java exception, and it really is one, since AbstractThrowableProblem is the subclass of the RuntimeException. As such, we could throw it from our JAX-RS API.

@Produces({ MediaType.APPLICATION_JSON, "application/problem+json" })
@GET
@Path("{id}")
public Person findById(@PathParam("id") String id) {
    return service
        .findById(id)
        .orElseThrow(() -> new PersonNotFoundProblem(id, uriInfo.getRequestUri()));
}

If we run the server and just try to fetch the person providing any identifier, the problem detail response is going to be returned back (since the dataset is not pre-populated), for example:

$ curl "http://localhost:21020/api/people/1" -H  "Accept: */*" 

HTTP/1.1 404
Content-Type: application/problem+json

{
    "type" : "http://localhost:21020/problems/person-not-found",
    "title" : "Person is not found",
    "status" : 404,
    "detail" : "Person with identifier '1' is not found",
    "instance" : "http://localhost:21020/api/people/1",
    "id" : "1"
}

Please notice the usage of the application/problem+json media type along with additional property id being included into the response. Although there are many things which could be improved, it is arguably better than just naked 404 (or 500 caused by EntityNotFoundException). Plus, the documentation section behind this type of the problem (in our case, http://localhost:21020/problems/person-not-found) could be consulted in case further clarifications may be needed.

So designing the problems after exceptions is just one option. You may often (and for very valid reasons) restrain from coupling you business logic with unrelated details. In this case, it is perfectly valid to return the problem details as the response payload from the JAX-RS resource. As an example, the registration process may raise NonUniqueEmailException so our web API layer could transform it into appropriate problem detail.

@Consumes(MediaType.APPLICATION_JSON)
@Produces({ MediaType.APPLICATION_JSON, "application/problem+json" })
@POST
public Response register(@Valid final CreatePerson payload) {
    try {
        final Person person = service.register(payload.getEmail(), 
            payload.getFirstName(), payload.getLastName());
            
        return Response
            .created(uriInfo.getRequestUriBuilder().path(person.getId()).build())
            .entity(person)
            .build();

    } catch (final NonUniqueEmailException ex) {
        return Response
            .status(Response.Status.BAD_REQUEST)
            .type("application/problem+json")
            .entity(Problem
                .builder()
                .withType(URI.create("http://localhost:21020/problems/non-unique-email"))
                .withInstance(uriInfo.getRequestUri())
                .withStatus(Status.BAD_REQUEST)
                .withTitle("The email address is not unique")
                .withDetail(ex.getMessage())
                .with("email", payload.getEmail())
                .build())
            .build();
        }
    }

To trigger this issue, it is enough to run the server instance and try to register the same person twice, like we have done below.

$ curl -X POST "http://localhost:21020/api/people" \ 
     -H  "Accept: */*" -H "Content-Type: application/json" \
     -d '{"email":"john@smith.com", "firstName":"John", "lastName": "Smith"}'

HTTP/1.1 400                                                                              
Content-Type: application/problem+json                                                           
                                                                                                                                                                                   
{                                                                                         
    "type" : "http://localhost:21020/problems/non-unique-email",                            
    "title" : "The email address is not unique",                                            
    "status" : 400,                                                                         
    "detail" : "The email 'john@smith.com' is not unique and is already registered",        
    "instance" : "http://localhost:21020/api/people",                                       
    "email" : "john@smith.com"                                                              
}                                                                                         

Great, so our last example is a bit more complicated but, probably, at the same time, the most realistic one. Our web API heavily relies on Bean Validation in order to make sure the input provided by the consumers of the API is valid. How would we represent the validation errors as the problem details? The most straightforward way is to supply the dedicated ExceptionMapper provider, which is the part of the JAX-RS specification. Let us introduce one.

@Provider
public class ValidationExceptionMapper implements ExceptionMapper<ValidationException> {
    @Context private UriInfo uriInfo;
    
    @Override
    public Response toResponse(final ValidationException ex) {
        if (ex instanceof ConstraintViolationException) {
            final ConstraintViolationException constraint = (ConstraintViolationException) ex;
            
            final ThrowableProblem problem = Problem
                    .builder()
                    .withType(URI.create("http://localhost:21020/problems/invalid-parameters"))
                    .withTitle("One or more request parameters are not valid")
                    .withStatus(Status.BAD_REQUEST)
                    .withInstance(uriInfo.getRequestUri())
                    .with("invalid-parameters", constraint
                        .getConstraintViolations()
                        .stream()
                        .map(this::buildViolation)
                        .collect(Collectors.toList()))
                    .build();

            return Response
                .status(Response.Status.BAD_REQUEST)
                .type("application/problem+json")
                .entity(problem)
                .build();
        }
        
        return Response
            .status(Response.Status.INTERNAL_SERVER_ERROR)
            .type("application/problem+json")
            .entity(Problem
                .builder()
                .withTitle("The server is not able to process the request")
                .withType(URI.create("http://localhost:21020/problems/server-error"))
                .withInstance(uriInfo.getRequestUri())
                .withStatus(Status.INTERNAL_SERVER_ERROR)
                .withDetail(ex.getMessage())
                .build())
            .build();
    }

    protected Map<?, ?> buildViolation(ConstraintViolation<?> violation) {
        return Map.of(
                "bean", violation.getRootBeanClass().getName(),
                "property", violation.getPropertyPath().toString(),
                "reason", violation.getMessage(),
                "value", Objects.requireNonNullElse(violation.getInvalidValue(), "null")
            );
    }
}

The snippet above distingushes two kind of issues: the ConstraintViolationExceptions indicate the invalid input and are mapped to 400 Bad Request, whereas generic ValidationExceptions indicate the problem on the server side and are mapped to 500 Internal Server Error. We only extract the basic details about violations, however even that improves the error reporting a lot.

$ curl -X POST "http://localhost:21020/api/people" \
    -H  "Accept: */*" -H "Content-Type: application/json" \
    -d '{"email":"john.smith", "firstName":"John"}' -i    

HTTP/1.1 400                                                                    
Content-Type: application/problem+json                                              
                                                                                
{                                                                               
    "type" : "http://localhost:21020/problems/invalid-parameters",                
    "title" : "One or more request parameters are not valid",                     
    "status" : 400,                                                               
    "instance" : "http://localhost:21020/api/people",                             
    "invalid-parameters" : [ 
        {
            "reason" : "must not be blank",                                             
            "value" : "null",                                                           
            "bean" : "com.example.problem.resource.PeopleResource",                     
            "property" : "register.payload.lastName"                                    
        }, 
        {                                                                          
            "reason" : "must be a well-formed email address",                           
            "value" : "john.smith",                                                     
            "bean" : "com.example.problem.resource.PeopleResource",                     
            "property" : "register.payload.email"                                       
        } 
    ]                                                                           
}                                                                               

This time the additional information bundled into the invalid-parameters member is quite verbose: we know the class (PeopleResource), method (register), the method's argument (payload) and the properties (lastName and email) respectively (all that extracted from the property path).

Meaningful error reporting is one of corner stones of the modern RESTful web APIs. Often it is not easy but definitely worth the efforts. The consumers (which often are just other developers) should have a clear understanding of what went wrong and what to do about it. The RFC 7807: Problem Details for HTTP APIs is a step into right direction and libraries like problem and problem-spring-web are here to back you up, please make use of them.

The complete source code is available on Github.

Sunday, August 26, 2018

Embracing modular Java platform: Apache CXF on Java 10

It's been mostly a year since Java 9 release finally delivered Project Jigsaw to the masses. It was a long, long journey, but it is there, so what has changed? This is a very good question and the answer to it is not obvious and straightforward.

By and large, Project Jigsaw is a disruptive change and there are many reasons why. Although mostly all of our existing application are going to run on Java 10 (to be replaced by JDK 11 very soon) with minimal or no changes, there are deep and profound implications Project Jigsaw brings to the Java developers: embrace the modular applications the Java platform way.

With the myriads of awesome frameworks and libraries out there, it will surely take time, a lot of time, to convert them to Java modules (many will not ever make it). This path is thorny but there are certain things which are already possible even today. In this rather short post we are going to learn how to use terrific Apache CXF project to build JAX-RS 2.1 Web APIs in a truly modular fashion using latest JDK 10.

Since 3.2.5 release, all Apache CXF artifacts have their manifests enriched with an Automatic-Module-Name directive. It does not make them full-fledged modules, but this is a first step in the right direction. So let us get started ...

If you use Apache Maven as the build tool of choice, not much changed here, the dependencies are declared the same way as before.

<dependencies>
    <dependency>
        <groupId>org.apache.cxf</groupId>
        <artifactId>cxf-rt-frontend-jaxrs</artifactId>
        <version>3.2.5</version>
    </dependency>

    <dependency>
        <groupId>com.fasterxml.jackson.jaxrs</groupId>
        <artifactId>jackson-jaxrs-json-provider</artifactId>
        <version>2.9.6</version>
    </dependency>

    <dependency>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-server</artifactId>
        <version>9.4.11.v20180605</version>
    </dependency>

    <dependency>
        <groupId>org.eclipse.jetty</groupId>
        <artifactId>jetty-webapp</artifactId>
        <version>9.4.11.v20180605</version>
    </dependency>
</dependencies>

The uber-jar or fat-jar packaging are not really applicable to the modular Java applications so we have to collect the modules ourselves, for example at the target/modules folder.

<plugin>
    <artifactId>maven-jar-plugin</artifactId>
    <version>3.1.0</version>
    <configuration>
        <outputDirectory>${project.build.directory}/modules</outputDirectory>
    </configuration>
</plugin>

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>3.1.1</version>
    <executions>
        <execution>
            <phase>package</phase>
            <goals>
                <goal>copy-dependencies</goal>
            </goals>
            <configuration>
                <outputDirectory>${project.build.directory}/modules</outputDirectory>
                <includeScope>runtime</includeScope>
            </configuration>
        </execution>
    </executions>
</plugin>

All good, the next step is to create the module-info.java and list there the name of our module (com.example.cxf in this case) and, among other things, all required modules it needs in order to be functionable.

module com.example.cxf {
    exports com.example.rest;
    
    requires org.apache.cxf.frontend.jaxrs;
    requires org.apache.cxf.transport.http;
    requires com.fasterxml.jackson.jaxrs.json;
    
    requires transitive java.ws.rs;
    
    requires javax.servlet.api;
    requires jetty.server;
    requires jetty.servlet;
    requires jetty.util;
    
    requires java.xml.bind;
}

As you may spot right away, org.apache.cxf.frontend.jaxrs and org.apache.cxf.transport.http come from Apache CXF distribution (the complete list is available in the documentation) whereas java.ws.rs is the JAX-RS 2.1 API module. After that we could proceed with implementing our JAX-RS resources the same way we did before.

@Path("/api/people")
public class PeopleRestService {
    @GET
    @Produces(MediaType.APPLICATION_JSON)
    public Collection<Person> getAll() {
        return List.of(new Person("John", "Smith", "john.smith@somewhere.com"));
    }
}

This looks easy, how about adding some spicy sauce, like server-sent events (SSE) and RxJava, for example? Let us see how exceptionally easy it is, starting from dependencies.

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-rt-rs-sse</artifactId>
    <version>3.2.5</version>
</dependency>

<dependency>
    <groupId>io.reactivex.rxjava2</groupId>
    <artifactId>rxjava</artifactId>
    <version>2.1.14</version>
</dependency>

Also, we should not forget to update our module-info.java by adding the requires directive to these new modules.

module com.example.cxf {
    ...
    requires org.apache.cxf.rs.sse;
    requires io.reactivex.rxjava2;
    requires transitive org.reactivestreams;
    ...

}

In order to keep things simple, our SSE endpoint would just broadcast every new person added through the API. Here is the implementation snippet which does it.

private SseBroadcaster broadcaster;
private Builder builder;
private PublishSubject<Person> publisher;
    
public PeopleRestService() {
    publisher = PublishSubject.create();
}

@Context 
public void setSse(Sse sse) {
    this.broadcaster = sse.newBroadcaster();
    this.builder = sse.newEventBuilder();
        
    publisher
        .subscribeOn(Schedulers.single())
        .map(person -> createEvent(builder, person))
        .subscribe(broadcaster::broadcast);
}

@POST
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public Response add(@Context UriInfo uriInfo, Person payload) {
    publisher.onNext(payload);
        
    return Response
        .created(
            uriInfo
                .getRequestUriBuilder()
                .path(payload.getEmail())
                .build())
        .entity(payload)
        .build();
}
    
@GET
@Path("/sse")
@Produces(MediaType.SERVER_SENT_EVENTS)
public void people(@Context SseEventSink sink) {
    broadcaster.register(sink);
}

Now when we build it:

      mvn clean package

And run it using module path:

      java --add-modules java.xml.bind \
           --module-path target/modules \
           --module com.example.cxf/com.example.Starter

We should be able to give our JAX-RS API a test drive. The simplest way to make sure things work as expected is to navigate in the Google Chrome to the SSE endpoint http://localhost:8686/api/people/sse and add some random people through the POST requests, using the old buddy curl from the command line:

      curl -X POST http://localhost:8686/api/people \
           -d '{"email": "john@smith.com", "firstName": "John", "lastName": "Smith"}' \
           -H "Content-Type: application/json"
      curl -X POST http://localhost:8686/api/people \
           -d '{"email": "tom@tommyknocker.com", "firstName": "Tom", "lastName": "Tommyknocker"}' \
           -H "Content-Type: application/json"

In the Google Chrome we should be able to see raw SSE events, pushed by the server (they are not looking pretty but good enough to illustrate the flow).

So, what about the application packaging? Docker and containers are certainly a viable option, but with Java 9 and above we have another player: jlink. It assembles and optimizes a set of modules and their dependencies into a custom, fully sufficient runtime image. Let us try it out.

      jlink --add-modules java.xml.bind,java.management \
            --module-path target/modules \
            --verbose \
            --strip-debug \
            --compress 2 \
            --no-header-files \
            --no-man-pages \
            --output target/cxf-java-10-app

Here we are hitting the first wall. Unfortunately, since mostly all the dependencies of our application are automatic modules, it is a problem for jlink and we still have to include module path explicitly when running from the runtime image:

      target/cxf-java-10-app/bin/java  \
           --add-modules java.xml.bind \
           --module-path target/modules \
           --module com.example.cxf/com.example.Starter

At the end of the day it turned out to be not that scary. We are surely in the very early stage of the JPMS adoption, this is just a beginning. When every library, every framework we are using adds the module-info.java to their artifacts (JARs), making them true modules despite all the quirks, then we could declare a victory. But the small wins are already happening, make one yours!

The complete source of the project is available on Github.

Monday, April 30, 2018

Moving With The Times: Towards OpenAPI v3.0.0 adoption in JAX-RS APIs

It is terrifying to see how fast time passes! The OpenAPI specification 3.0.0, a major revamp of so-get-used-to Swagger specification, has been released mostly one year ago but it took awhile for tooling to catch up. However, with the recent official release of the Swagger Core 2.0.0 things are going to accelerate for sure.

To prove the point, Apache CXF, the well-known JAX-RS 2.1 implementation, is one of the first adopters of the OpenAPI 3.0.0 and in today's post we are going to take a look how easy your JAX-RS 2.1 APIs could benefit from it.

As always, to keep things simple we are going to design a people management web APIs with just a handful set of resources to support it, nothing too exciting here.

POST   /api/people
GET    /api/people/{email}
GET    /api/people
DELETE /api/people/{email}

Our model would consist of a single Person class.

public class Person {
    private String email;
    private String firstName;
    private String lastName;
}

To add a bit of magic, we would be using Spring Boot to get us up and running as fast as possible. With that, let us start to fill in the dependencies (assuming we are using Apache Maven for build management).

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-spring-boot-starter-jaxrs</artifactId>
    <version>3.2.4</version>
</dependency>

In the recent 3.2.x releases Apache CXF introduces a new module cxf-rt-rs-service-description-openapi-v3 dedicated to OpenAPI 3.0.0, based on Swagger Core 2.0.0.

<dependency>
    <groupId>org.apache.cxf</groupId>
    <artifactId>cxf-rt-rs-service-description-openapi-v3</artifactId>
    <version>3.2.4</version>
</dependency>

<dependency>
    <groupId>org.webjars</groupId>
    <artifactId>swagger-ui</artifactId>
    <version>3.13.6</version>
</dependency>

The presence of the Swagger UI is not strictly necessary, but this is exceptionally useful and beautiful tool to explore your APIs (and if it is available on classpath, Apache CXF seamlessly integrates it into your applications as we are going to see in a bit).

The prerequisites are in place, let us do some coding! Before we begin, it is worth to note that the Swagger Core 2.0.0 has many ways to populate the OpenAPI 3.0.0 definitions for your services, including property files, annotations or programmatically. Along this post we are going to use annotations only.

@OpenAPIDefinition(
    info = @Info(
        title = "People Management API",
        version = "0.0.1-SNAPSHOT",
        license = @License(
            name = "Apache 2.0 License",
            url = "http://www.apache.org/licenses/LICENSE-2.0.html"
        )
    )
)
@ApplicationPath("api")
public class JaxRsApiApplication extends Application {
}

Looks pretty simple, the @OpenAPIDefinition sets the top-level definition for all our web APIs. Moving on to the PeopleRestService, we just add the @Tag annotation to, well, tag our API.

@Path( "/people" ) 
@Tag(name = "people")
public class PeopleRestService {
    // ...
}

Awesome, nothing complicated so far. The meaty part starts with web API operation definitions, so let us take a look on the first example, the operation to fetch everyone.

@Produces(MediaType.APPLICATION_JSON)
@GET
@Operation(
    description = "List all people", 
    responses = {
        @ApiResponse(
            content = @Content(
                array = @ArraySchema(schema = @Schema(implementation = Person.class))
            ),
            responseCode = "200"
        )
    }
)
public Collection<Person> getPeople() {
    // ...
}

Quite a few annotations but by and large, looks pretty clean and straightforward. Let us take a look on another one, the endpoint to find the person by its e-mail address.

@Produces(MediaType.APPLICATION_JSON)
@Path("/{email}")
@GET
@Operation(
    description = "Find person by e-mail", 
    responses = {
        @ApiResponse(
            content = @Content(schema = @Schema(implementation = Person.class)), 
            responseCode = "200"
        ),
        @ApiResponse(
            responseCode = "404", 
            description = "Person with such e-mail doesn't exists"
        )
    }
)
public Person findPerson(
        @Parameter(description = "E-Mail address to lookup for", required = true) 
        @PathParam("email") final String email) {
    // ...
}

In the same vein, the operation to remove the person by e-mail looks mostly identical.

@Path("/{email}")
@DELETE
@Operation(
    description = "Delete existing person",
    responses = {
        @ApiResponse(
            responseCode = "204",
            description = "Person has been deleted"
        ),
        @ApiResponse(
            responseCode = "404", 
            description = "Person with such e-mail doesn't exists"
        )
     }
)
public Response deletePerson(
        @Parameter(description = "E-Mail address to lookup for", required = true ) 
        @PathParam("email") final String email) {
    // ...
}

Great, let us wrap up by looking into last, arguably the most interesting endpoint which adds a new person (the choice to use the @FormParam is purely to illustrate the different flavor of the APIs).

@Consumes(MediaType.APPLICATION_FORM_URLENCODED)
@Produces(MediaType.APPLICATION_JSON)
@POST
@Operation(
    description = "Create new person",
    responses = {
        @ApiResponse(
            content = @Content(
                schema = @Schema(implementation = Person.class), 
                mediaType = MediaType.APPLICATION_JSON
            ),
            headers = @Header(name = "Location"),
            responseCode = "201"
        ),
        @ApiResponse(
            responseCode = "409", 
            description = "Person with such e-mail already exists"
        )
    }
)
public Response addPerson(@Context final UriInfo uriInfo,
        @Parameter(description = "E-Mail", required = true) 
        @FormParam("email") final String email, 
        @Parameter(description = "First Name", required = true) 
        @FormParam("firstName") final String firstName, 
        @Parameter(description = "Last Name", required = true) 
        @FormParam("lastName") final String lastName) {
    // ...
}

If you have an experience with documenting web APIs using older Swagger specifications, you might find the approach quite familiar but more verbose (or better to say, formalized). It is the result of the tremendous work which specification leads and community has done to make it as complete and extensible as possible.

The APIs are defined and documented, it is time to try them out! The missing piece though is the Spring configuration where we would initialize and expose our JAX-RS web services.

@Configuration
@EnableAutoConfiguration
@ComponentScan(basePackageClasses = PeopleRestService.class)
public class AppConfig {
    @Autowired private PeopleRestService peopleRestService;
 
    @Bean(destroyMethod = "destroy")
    public Server jaxRsServer(Bus bus) {
        final JAXRSServerFactoryBean factory = new JAXRSServerFactoryBean();

        factory.setApplication(new JaxRsApiApplication());
        factory.setServiceBean(peopleRestService);
        factory.setProvider(new JacksonJsonProvider());
        factory.setFeatures(Arrays.asList(new OpenApiFeature()));
        factory.setBus(bus);
        factory.setAddress("/");

        return factory.create();
    }

    @Bean
    public ServletRegistrationBean cxfServlet() {
        final ServletRegistrationBean servletRegistrationBean = new ServletRegistrationBean(new CXFServlet(), "/api/*");
        servletRegistrationBean.setLoadOnStartup(1);
        return servletRegistrationBean;
    }
}

The OpenApiFeature is a key ingredient here which takes care of all the integration and introspection. The Spring Boot application is the last touch to finish the picture.

@SpringBootApplication
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(AppConfig.class, args);
    }
}

Let us build and run it right away:

mvn clean package 
java -jar target/jax-rs-2.1-openapi-0.0.1-SNAPSHOT.jar

Having the application started, the OpenAPI 3.0.0 specification of our web APIs should be live and available for consumption in the JSON format at:

http://localhost:8080/api/openapi.json

Or in YAML format at:

http://localhost:8080/api/openapi.json

Wouldn't it be great to explore our web APIs and play with it? Since we included Swagger UI dependency, this is no brainer, just navigate to http://localhost:8080/api/api-docs?url=/api/openapi.json:

A special attention to be made to the small icon Swagger UI places alongside your API version, hinting about its conformity to OpenAPI 3.0.0 specification.

Just to note here, there is nothing special in using Spring Boot. In case you are using Apache CXF inside the OSGi container (like Apache Karaf for example), the integration with OpenAPI 3.0.0 is also available (please check out the official documentation and samples if you are interested in the subject).

It all looks easy and simple, but what about migrating to OpenAPI 3.0.0 from older versions of the Swagger specifications? The Apache CXF has an powerful feature to convert older the specifications on the fly but in general the OpenApi.Tools portal is a right place to evaluate your options.

Should you migrate to OpenAPI 3.0.0? I honestly believe you should, at least should try experimenting with it, but please be aware that the tooling is still not mature enough, expect a few roadblocks along the way (which you would be able to overcome by contributing the patches, by the way). But undoubtedly, the future is bright!

The complete project sources are available on Github.

Monday, September 19, 2016

Stop being clueless, just instrument and measure: using metrics to gain insights about your JAX-RS APIs

How often we, developers, build these shiny REST(ful) APIs (or microservices, joining the hype here) hoping they are going to just work in production? There is enormous amount of frameworks and toolkits out there which give us the ability to be very productive at development, however when things are deployed in production, most of them keep us clueless about what it is going on.

Spring Boot is certainly an exception from this rule and in today's post we are going to talk about using Spring Boot Actuator along with terrific Dropwizard Metrics library to collect and expose metrics about Apache CXF-based JAX-RS APIs. To keep things even more interesting, we are going to feed the metrics into amazing Prometheus collector and visualize using beautiful Grafana dashboards.

With that, let us get started by defining simple JAX-RS service to manage people, PeopleRestService. We are not going to plug it to any external storage or whatnot but instead just cheat a bit by relying on Spring Reactor project and introducing random delays while returning a predefined response.

@Path("/people")
@Component
public class PeopleRestService {
    private final Random random = new Random();
 
    @GET
    @Produces({MediaType.APPLICATION_JSON})
    public Collection<Person> getPeople() {
        return Flux
            .just(
                new Person("a@b.com", "John", "Smith"), 
                new Person("c@b.com", "Bob", "Bobinec")
            )
            .delayMillis(random.nextInt(1000))
            .toStream()
            .collect(Collectors.toList());
    }
}

Because we are going to use Spring Boot and its automagic discovery capabilities, the configuration is going look rather trivial. We have already talked about using Spring Boot along with Apache CXF but now even this part has been improved, thanks to the Apache CXF Spring Boot Starter available (it becomes just a matter of adding one more dependency to your project).

@Configuration
@EnableAutoConfiguration
public class AppConfig {
    @Bean(destroyMethod = "destroy")
    public Server jaxRsServer(final Bus bus) {
        final JAXRSServerFactoryBean factory = new JAXRSServerFactoryBean();

        factory.setServiceBean(peopleRestService());
        factory.setProvider(new JacksonJsonProvider());
        factory.setBus(bus);
        factory.setAddress("/");

        return factory.create();
    }
 
    @Bean
    public PeopleRestService peopleRestService() {
        return new PeopleRestService();
    }
}

At this point we did everything we needed to have a bare bones Spring Boot hosting Apache CXF-based JAX-RS service. Please take a note that by default with Apache CXF Spring Boot Starter all APIs are served under /services/* mapping.

Till now we said nothing about metrics yet and that is what we are going to talk about next. Dropwizard Metrics is a de-facto standard for JVM applications and has a rich set of different kind of metrics (meters, gauges, counters, histograms, ...) and reporters (console, JMX, HTTP, ...). Consequently, the MetricRegistry is a central place to manage all the metrics. And surely, the typical way to expose metrics for JVM-based application is JMX so let us include the respective beans into configuration.

@Bean(initMethod = "start", destroyMethod = "stop")
public JmxReporter jmxReporter() {
    return JmxReporter.forRegistry(metricRegistry()).build();
}
 
@Bean
public MetricRegistry metricRegistry() {
    return new MetricRegistry();
}

You are free to create as many metrics as you need and we could have added a few for our PeopleRestService as well. But luckily, Apache CXF has a dedicated MetricsFeature feature to integrate with Dropwizard Metrics and collect all the relevant ones, with zero effort. A minor update of the JAXRSServerFactoryBean initialization is enough.

@Bean(destroyMethod = "destroy")
public Server jaxRsServer(final Bus bus) {
    final JAXRSServerFactoryBean factory = new JAXRSServerFactoryBean();

    factory.setServiceBean(peopleRestService());
    factory.setProvider(new JacksonJsonProvider());
    factory.setBus(bus);
    factory.setAddress("/");
    factory.setFeatures(Collections.singletonList(
        new MetricsFeature(new CodahaleMetricsProvider(bus))
    ));
    factory.setProperties(
        Collections.singletonMap(
            "org.apache.cxf.management.service.counter.name", 
            "cxf-services."
        )
    );

    return factory.create();
}

Just a quick note about org.apache.cxf.management.service.counter.name. By default, Apache CXF is going to name metrics quite verbosely, including unique bus identifier as part of the name as well. It is not very readable so we just override the default behaviour using static 'cxf-services.' prefix. This is how those metrics are going to look like in the JMX console.

It looks terrific, but JMX is not a very pleasant piece of technology to dial with, could we do better? Here is where Spring Boot Actuator comes into play. Along with many other endpoints, it is able to expose all the metrics over HTTP protocol by adding a couple of properties to application.yml file:

endpoints:
  jmx:
    enabled: true
    unique-names: true

management:
  security:
    enabled: true 

It is important to mention here that metrics, along with other Spring Boot Actuator endpoints, may expose a sensitive details about your application so it is always a good idea to protect them, for example, using Spring Security and HTTP Basic Authentication. Again, a few configuration properties in application.yml will do all the work:

security:
  ignored:
    - /services/**
  user:
    name: guest
    password: guest

Brilliant, if we run our application and access /metrics endpoint (providing guest/guest as credentials), we should see quite an extensive list of metrics, like these ones:

> curl -u guest:guest http://localhost:19090/metrics

{
    "classes": 8673,
    "classes.loaded": 8673,
    "classes.unloaded": 0,
    "counter.status.200.metrics": 5,
    "counter.status.200.services.people": 1,
    "counter.status.401.error": 2,
    "cxf-services.Attribute=Checked Application Faults.count": 0,
    "cxf-services.Attribute=Checked Application Faults.fifteenMinuteRate": 0.0,
    "cxf-services.Attribute=Checked Application Faults.fiveMinuteRate": 0.0,
    "cxf-services.Attribute=Checked Application Faults.meanRate": 0.0,
    ...
}

It would be great to have some dedicated monitoring solution which could understand these metrics, store them somewhere and give us useful insights and aggregations in real-time. Prometheus is exactly the tool we are looking for but there are bad and good news. On a not so good side, Prometheus does not understand the format which Spring Boot Actuator uses to expose metrics. But on a bright side, Prometheus has a dedicated Spring Boot integration so the same metrics could be exposed in Prometheus-compatible format, we are few beans away from that.

@Configuration
public class PrometheusConfig {
    @Bean
    public CollectorRegistry collectorRegistry() {
        return new CollectorRegistry();
    }

    @Bean
    public SpringBootMetricsCollector metricsCollector(
            final Collection<PublicMetrics> metrics, final CollectorRegistry registry) {
        return new SpringBootMetricsCollector(metrics).register(registry);
    }

    @Bean
    public ServletRegistrationBean exporterServlet(final CollectorRegistry registry) {
        return new ServletRegistrationBean(new MetricsServlet(registry), "/prometheus");
    }
}

With this configuration in place, metrics alternatively are going to be exposed under /prometheus endpoint, let us check this out.

> curl -u guest:guest http://localhost:19090/prometheus

# HELP cxf_services_Attribute_Data_Read_fifteenMinuteRate cxf_services_Attribute_Data_Read_fifteenMinuteRate
# TYPE cxf_services_Attribute_Data_Read_fifteenMinuteRate gauge
cxf_services_Attribute_Data_Read_fifteenMinuteRate 0.0
# HELP cxf_services_Attribute_Runtime_Faults_count cxf_services_Attribute_Runtime_Faults_count
# TYPE cxf_services_Attribute_Runtime_Faults_count gauge
cxf_services_Attribute_Runtime_Faults_count 0.0
# HELP cxf_services_Attribute_Totals_snapshot_stdDev cxf_services_Attribute_Totals_snapshot_stdDev
# TYPE cxf_services_Attribute_Totals_snapshot_stdDev gauge
cxf_services_Attribute_Totals_snapshot_stdDev 0.0
...

All the necessary pieces are covered and the fun time is about to begin. Prometheus has a very simple and straightforward installation steps but Docker is certainly the easiest one. The project repository includes docker-compose.yml file in docker folder to get you started quickly. But before, let us build the Docker image of our Spring Boot application using Apache Maven:

> mvn clean install

Upon successful build, we are ready to use Docker Compose tool to start all the containers and wire them together, for example:

> cd docker
> docker-compose up

Recreating docker_cxf_1
Recreating docker_prometheus_1
Recreating docker_grafana_1
Attaching to docker_cxf_1, docker_prometheus_1, docker_grafana_1
...

If you are using native Docker packages, just open your browser at http://localhost:9090/targets where you could see that Prometheus has successfully connected to our application and is consuming its metrics (for older Docker installations, please use the address of your Docker Machine).

The cxf target came preconfigured from Prometheus configuration file, located at docker/prometheus.yml and used to build the respective container in the docker-compose.yml (please notice the presence of the credentials to access /prometheus endpoint):

# my global config
global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s # By default, scrape targets every 15 seconds.

scrape_configs:
  - job_name: 'cxf'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    basic_auth:
      username: guest
      password: guest

    metrics_path: '/prometheus'

    # Default scheme is http
    static_configs:
      - targets: ['cxf:19090']

Prometheus supports graph visualizations but Grafana is unquestionable leader in mastering beautiful dashboards. It needs a bit of configuration though which could be done over web UI or, even better, through API . The data source is the most important one, and in our case should point to running Prometheus instance.

> curl 'http://admin:admin@localhost:3000/api/datasources' -X POST -H 'Content-Type: application/json;charset=UTF-8' --data-binary '{"name": "prometheus", "type": "prometheus", "url":"http://prometheus:9090", "access":"proxy", "isDefault":true}'

Done! Adding a sample dashboard is the next thing to do and again, API is the best way to accomplish that (assuming you are still in the docker folder)

> curl 'http://admin:admin@localhost:3000/api/dashboards/db' -X POST -H 'Content-Type: application/json;charset=UTF-8' --data-binary @cxf-dashboard.json

The same rule applies here, if you are still using Docker Machine, please replace localhost with appropriate virtual machine address. Also please notice that you have to do this only once when the containers are created first time. The configuration will be kept for existing containers.

To finish up, let us open our custom Grafana dashboard by navigating to http://localhost:3000/dashboard/db/cxf-services, using admin/admin as default credentials. Surely, you are going to see no data at first but by generating some load (f.e. using siege), we could have gotten interesting graphs to analyze, for example:

Those graphs were made simple (and not so much useful to be honest) on purpose, just to demonstrate how easy it is to collect and visualize metrics from your Apache CXF-based JAX-RS APIs in real-time. There are so many useful metrics our applications could expose that no shortage of ideas here expected. Plus, Grafana allows to define quite sophisticated graphs and queries, worth of another article, but official documentation is a good point to start off.

Hope this post will encourage everyone to think seriously about monitoring your JAX-RS APIs by exposing, collecting and visualizing important metrics. This is just a beginning ...

The complete project sources are available on Github.