1. JSON

JSON is a the acronym of JavaScript Object Notation. It is used primarly to transmit data between a server and a web application as an alternative to XML.

1.1. Data types

JSON’s basic types are:

Number

Signed decimal number with optional fractional and exponential part.

String

Unicode characters.

Boolean

true or false.

Array

Ordered list of elements in array form.

Object

Unordered associative array of name/value pairs delimited with curly brackets (‘{’) and (‘}’) and each pair separated by colon (‘:’) character.

Null

null.

1.2. Example

{
    "firstName": "John", (1)
    "lastName": "Smith",
    "isAlive": true, (2)
    "age": 25,
    "heightCm": 167.6, (3)
    "address": { (4)
        "streetAddress": "21 2nd Street",
        "city": "New York",
        "state": "NY",
        "postalCode": "10021-3100"
    },
    "phoneNumbers": [ (5)
        {
            "type": "home",
            "number": "212 555-1234"
        },
        {
            "type": "office",
            "number": "646 555-4567"
        }
    ],
    "favoriteColors": ["blue", "yellow"], (6)
    "spouse": null (7)
}
1 Attribute with string value.
2 Attribute with boolean value. Note that it is not quoted (‘"’).
3 Attribute with decimal value and fraction.
4 Definition of object with several attributes divided by comma (‘,’).
5 Arrays are defined between brackets (‘[’) and (‘]’), and each element separated by comma (‘,’). Note that an array may contain objects too.
6 Arrays may contain native types.
7 null attribute value.

1.3. Best Practices

  • Indent in JSON documents must be two or four white spaces.

  • Only one key/value pair per line.

  • In case of objects, opening curly bracket should be placed after colon, and closing curly bracket at same idention level as key.

  • In case of arrays of objects, opening bracket should be placed after colon, and closing bracket at same idention level as key.

  • Arrays of simple types may be defined in a single line.

  • Attribute names must follow camel case notation with lowercase first letter.

2. RESTful Web Services

2.1. What is REST?

Representational state transfer (REST) is an architectural style consisting of a coordinated set of architectural constraints applied to components, connectors, and data elements, within a distributed hypermedia system
— Wikipedia

Web service APIs that follow REST architectural are known as RESTful Web Services. They defines with these aspects:

  • a base URI to identify a resource

  • an internet media type like JSON, XML, image.

  • use of standard HTTP methods such as GET, POST, PUT or DELETE.

  • hypertext links to reference state

  • hypertext links to reference related resources

RESTful resources are identified by URI. Depending on the form of the URI, more or less resources will be involved. Let’s see some examples of valid URIs and their meaning.

URI Description

/items

Represents a collection of items

/items/name/Beer

Represents items with name Beer

/items/ordered

Represents a collection of items that are ordered

/users/12/items

Represents all the items for a user identified with 12

As seen in previous table, URI only are used to represent resources, but not what action (verb) to apply them. Actions are set by HTTP methods.

HTTP methods Description

GET

Get resource/s

POST

Create a resource

PUT

Update a resource if exists or create a new one

DELETE

Delete a resource

HEAD

HTTP headers are returned without content.

PATCH

Apply a set of changes to the resource identified by the request’s URI.

So the combination of HTTP method and URI sets the operation to be executed and which resources will be affected.

HTTP method resource URI Description

GET /items

Gets a list of items

POST /items

Create a new item

PUT /items/name/Beer

Updates some data from items with name Beer

DELETE /users/12/items

Deletes all items of user with id 12

  • Use nouns to navigate through resources. For example /users/12/items is better than /users/12/getItems.

  • Use subresources to refer to resource associations. For example /users/12/items to get all items from an specific user.

  • Use query params for specific variations and path params only for identifiers. For example /users?age=25.

  • Add defaults for output format for just in case client does not specify it.

  • Use plural nouns. For example use /cars instead of /car.

  • JSON document attributes in camel case form.

  • In PUT and POST use body content to send data to be created or updated.

  • By specification PUT is a create or update operation. To avoid confusions PUT should only update resources.

  • Use PATCH to execute a set of operations atomically. PATCH can be used to change a resource partially or apply some operations on it.

About PATCH

The PATCH method requests that a set of changes to be applied to resource identified by the request’s URI. This set contains instructions describing how a resource currently residing on the origin server should be modified.

But PATCH is not about sending an update value rather than the entire resource.

Next example is wrong and should be avoided:

PATCH /users/123

{ "email": "new.email@example.org" }

As mentioned, PATCH should provide a description of changes to be applied. You can think about it as a sequence of operations to apply to a resource.

RFC-6902 [rfc6902] defines a way for expression operations into a JSON document. Also it defines the 6 possible operations that can be executed:

test

tests that a value at the target location is equal to a specified value

remove

removes the value at the target location.

add

if the target location specifies an array index, a new value is inserted into the array at the specified index. If the target location specifies an object member that does not already exist, a new member is added to the object.

replace

replaces the value at the target location with a new value.

move

removes the value at a specified location and adds it to the target location.

copy

copies the value at a specified location to the target location.

A possible document may look like:

[
  { "op": "test", (1)
    "path": "/a/b/c", (2)
    "value": "foo" (3)
  },
  { "op": "replace", "path": "/a/b/c", "value": 42 }
]
1 Operation is set by using op attribute and as value the name of the operation.
2 path defines where the operation is applied within resource.
3 value to apply to given operation.

Previous patch example can be rewritten to:

PATCH /users/123 (1)

[
  { "op": "replace", "path": "/email", "value": "new.email@example.org" } (2)
]
1 The resource to apply changes is a user with id 123.
2 email field of given user is upadated to new value.

You can read more examples at http://tools.ietf.org/html/rfc6902#appendix-A

HTTP 204 status response code is used because the response does not carry any message body. Of course any other valid HTTP status code could be returned too.

Also PATCH requires you set Content-Type header to application/json-patch.

2.2. Content Negotiation

RESTful Web Services can consume and produce different media type like JSON, XML or any other valid type like plain text or binary.

Content negotiation allows different representations of a resource so that clients can consume what suits best for them. The de-facto media type in RESTful Web Services is JSON, but client side must provide to server-side which media type is expecting.

There are two different approaches:

  • Using HTTP headers. Accept HTTP header is used by the client to indicate which media type can handle. The Content-Type HTTP header is used to indicate the MIME type of the entity being sent by the server.

  • Using URL patterns. By using extension of the resource, server side knows which media type is expected by the client. For example http://server/items.xml to retrieve items in XML form.

Use HTTP headers approach instead of URL patterns because HTTP headers provide a clear separation between infrastructure and business.

2.3. Versioning

Probably RESTful API will evolve during its lifetime. For this reason we need a way to version the API and provide some kind of back-compatibility so client can choose which version to use. At least we should ensure that previous applications still works although a new version of the API has been published. There are several approaches to version APIs:

  • Specifying in the URI itself. For example http://server/v2/items/12. Note that ‘v2’ is used as label to set which version of the API client is expecting.

  • Specifying as request parameter. For example http://server/items/12?version=v2. Note that in this case version is set as query parameter ‘?version=v2’.

  • Specifying inside HTTP header in Accept field or in a custom one. For example Accept: application/vnd.server.v2+json accepts the content produced in ‘json’ and from ‘version 2’.

There is no clear approach about versioning. We are going to use the first one by setting in URI the version of API.

2.4. Response Codes

RESTful Web Services use HTTP protocol as communication layer. An HTTP response must contain a response code so caller can inspect if the request it has done is correct or not. RESTful Web Services should follow same rules and return a response code depending on the result of executed operation. Next table summarize typical situations:

Group Code Description

Success 2XX

200 OK

This returns content as a part of the response

201 Created

Used by POST. It must return Location header with resource id

202 Accepted

Used in asynchronous operations. It must return Location header specifying where the client can monitor for the request.

204 No Content

No content retuned as a part of the response. Used by PUT when the resource is updated succesfully.

Redirectional 3XX

301 Permanent

Shows that all requests are directed to new location

302 Found

Shows that a resource already exists and is valid

Client Errors 4XX

400 Bad Request

Used when request cannot be processed due to syntax errors. For example malformed JSON.

401 Unauthorized

Used when request cannot be processed because of current user credentials.

403 Forbidden

Used when security token is valid but it has expired.

404 Not Found

Used when resource is not found, when an unauthenticated user request a secured resource or when security token is missing or invalid.

406 Not Acceptable

Used when the resource cannot produce the the MIME type specified by the client

409 Conflict

Used when two resources are modified concurrently, in this case the latest modification should not be produced and return this error.

415 Unsupported Media Type

Used when the resource cannot consume the the MIME type specified by the client

422 Unprocessable Entity

Used when request cannot be processed due to validation errors. Request body is well formed but semantically erroneous. For example an email field without ‘@’ character.

Server Errors 5XX

500 Internal Server error

Used as a generic error message

Most of these codes are provided by default by container so we don’t have to worry about them. The most used response codes are ‘2XX’, ‘5XX’ and some ‘4XX’.

2.4.1. Error Codes

In case of error codes, a JSON document can be sent back with information about the failure. RESTful Web Service uses HTTP error codes to notify them to caller. Error codes are those starting with ‘4XX’ or ‘5XX’.

When an error is sent to caller, it can contain a JSON document as response body. If a JSON document is sent to explain the reason of the failure, it must follow next schema:

{
  "message": "Validation Failed", (1)
  "errors": [ (2)
    {
      "resource": "Book", (3)
      "field": "title", (4)
      "code": "field_mandatory" (5)
    }
  ]
}
1 Generic message explaining the error that has occured. This field is mandatory.
2 List of specific errors. This section is optional and it may be used when there are field validation errors. This subdocument has sense when an HTTP 422 is sent back.
3 Name of the resource that contains the error.
4 Field with the error.
5 Code that represents the error. Frontend is the responsible of translating this code the user message in the configured locale.

There are some discussions about how to notify validation errors. We have based on rfc-4918 [rfc4918], github API [githubdev] and blog post [bennadel] which argues that the best option is to use the _HTTP 422 code.

2.5. Pagination

REST APIs may return a huge number of resources per request. To avoid overloading client side (and this is specially a problem in case of lightweight clients), we should paged each request with a certain number of items per request. This is known as Response pagination. Along with response is important to add some kind of metadata like current page, number of pages, total number of elements or a link to next set of results.

There are two possible approaches:

  • Offset-based pagination which in general uses two query params named offset which sets which page to return and is zero-based and limit that sets the maximum number of results to be returned. For example GET items?page=1&limit=50.

  • Time-based pagination which uses timestamps to paginate results between a specific timeframe. In this case until query param is used to point the end of the time range, and since for the beginning.

In case client can specify which field is used for sorting and direction, sort query param must be used. As value you set an optional character to set the direction of the order, ‘+’ for ascending and ‘-’ for descending, and finally the field name to sort.

For example GET /books?sort=+name for ascending direction or GET /books?sort=-name for descending.

If no direction is provided, ascending direction is the default one.
Currently you can only sort elements by single field.
Example of Offset-based response
{
  "entities": [ (1)
    {
      "name": "foo",
        "age": 20
    },
    {
      "name": "bar",
      "age": 30
    }
  ],

  "pagination": { (2)
    "limit": 100, (3)
    "offset": 0, (4)
    "count": 2, (5)
  },
  "sorting": { (6)
    "orderDirection": "ASC", (7)
    "orderFieldName": "id" (8)
  }
}
1 entities section adds all elements to be shown in current page.
2 pagination is the parent element for all pagination elements.
3 limit is the number of elements per page to retrieve.
4 offset sets the current page (or offset). It is zero-based.
5 count is the total number of items available.
6 sorting is the parent element for all sort parameters if elements are ordered..
7 Direction of the order. ASC for ascendant and DESC for descendant.
8 Ordered field name.

Obviously Time-based pagination is not always possible, it will depend on the resource nature.

Using offset pagination does not avoid returning duplicate records in case where additional resources are added between pagination requests. This is something that depending on the number of pace of inserts and updates and the criticality of the information shown.

To avoid this problem a cursor-based pagination can be used. Cursor-based pagination is pretty similar to offset approach but uses already known sequencial identifier of entity to know exactly at which point the latest result was returned.

You can read about real-time pagination in [realtimepagination].

2.6. Filtering

Getting a resource may require some kind of filtering such as returning only resources with specific value. In this cases query param with field name and value approach is used.

For example GET /books?title=Bible

You can query by multiple parameters by adding a new query param followed by ampersand (‘&’)

2.7. Field Selection

Sometimes client just need a few attributes instead of all attributes of a resource. In endpoints that client can choose returned fields, a query param named fields must be used. And as value a list of comma-separated values of all fields to be returned.

For example GET /books?fields=title,pages.

In case of no query param, the whole document should be returned to be compliant with REST API.

2.8. Internationalization

RESTful Web Services can require to serve different responsed depending on the country and the locale.

Language negotiation is similar to content negotiation, so you can use diffrent approaches:

  • Using Accept-Language HTTP headers.

  • Using a query parameter such as locale. For example items/?locale=es

Use HTTP headers approach instead of query parameter approach because HTTP headers provide a clear separation between infrastructure and business.

Any internacionalized response should set Content-Language HTTP header in response as well.

In case of ordered Pagination and internationalization you may want to use java.text.Collator class as a helper class for sorting fields depending on locale.

2.9. Existence checks

From prespective of frontend, sometimes you may need to check the existance of one resource before creating it. For example to notify the problem to the user as soon as possible while filling a form, instead of after ‘submit’ action.

As noted at [headverb], existence checks are done by using HEAD HTTP method.

The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response. The metainformation contained in the HTTP headers in response to a HEAD request SHOULD be identical to the information sent in response to a GET request. This method can be used for obtaining metainformation about the entity implied by the request without transferring the entity-body itself.
— RFC - 2616
Section 9.4

So in cases where the content is not important, but the existance of the resource is, HEAD is the verb to be used. We can do everything like a GET and check the response code without the weight of the response body.

For example to know if a user with login attribute (which is unique) is already created, next endpoint can be used:

The response may be a 200 OK if the resource already exists or 404 OK in case it has not been created yet.

2.10. Counting

In some cases you may need to count the number of resources that are stored in backend. In these cases you may use special reserved keyword count.

So if we should return the number of books we should do something like GET /books/count.

The return type is a JSON document that returns the number of elements.

{
  "count": 6 (1) (2)
}
1 count sets the number of elements.
2 Number of elements is an integer.

2.11. REST Polling

Sometimes an action takes too long to be completed in the context of a single HTTP request. Probably some kind of feedback should be provided to users, for example the number of processed elements, remaining elements or expected time to finish the task. Asynchronous actions should only executed for POST, PUT, PATCH or DELETE HTTP methods.

HTTP protocol provides different ways to implement asynchronous callback action:

WebSockets

It is a protocol providing full-duplex communications channels over a single TCP connection.

Server Sent Events (SSE)

Browser receives automatic updates from a server via HTTP connection and Server-Sent Events API.

Polling

The client requests information from the server. Polling cycle is the time in which each element is monitored once.

Long Polling

The client requests information from the server exactly as in normal polling, but in case of no information, the server side holds the connection until one become available.

WebHooks

The source site makes an HTTP request to the URI configured for the webhook for given a given event.

All approaches offer PROs and CONs, but currently the polling way is going to be used because of simplification and easier implementation.

When an asynchronous operation is executed, instead of returning for example an HTTP 201 status code in case of POST or HTTP 204 in case of PUT, an HTTP 202 code is returned with Location header specifying an identifier of the queued task where the client can monitor the request. The HTTP 202 is returned until defined HTTP status code is returned. For example an HTTP 201 in case of POST or HTTP 204 in case of PUT.

To get information from a task, GET request should be done to tasks resource and identified with value of Location attribute.

Let’s see an example of REST polling. Creation of a resource may require validation of external services which can take several seconds. Some feedback to the users could be provided about what steps are being executed. To provide this feedback, client is going to use REST polling to know which step is being executed.

First step is sending a POST to backend POST http://<localhost>/<application>/books and it will return a HTTP 202 OK with header’s attribute Location set to 123fa.

Then client can send GET requests to tasks resource, in this case http://<localhost>/<application>/tasks/<tokenId>;, and polling the resource until a status code not equal to HTTP 202 is received.

Each time an HTTP 202 is returned, a JSON document must be sent in body response with information about the status of the task. This document may follow next schema:

{
  "processed" : 18, (1)
  "of" : 200, (2)
  "estimatedTime" : 30 (3)
}
1 Number of elements that has been processed.
2 Total number of elements.
3 Optional parameter that sets the time in seconds remaining until all elements are processed.
processed and of are integers, but it may contain a null value in case it cannot be calculated.
Protocol Schema
Figure 1. Protocol Schema

2.12. Chunked Upload

Uploads large files to Dropbox in multiple chunks. Also has the ability to resume if the upload is interrupted.

To implement this massive upload two endpoints are required, /chunkedUpload and /commitChunkedUpload.

The typical usage will be:

First we send a POST request to /chunkedUpload with the first chunk of bytes of the file. It will return a JSON document with uploadedId and offset and 202 as HTTP status code.

{
  "uploadId": "5453543AE34348756", (1)
  "offset": 32567 (2)
}
1 Sets the number to reference in successive calls.
2 The byte offset of this chunk relative to the beginning of the full file.

After that, a successive calls to /chunkedUpload using PUT and passing as parameters both uploadId and offset. In case of succesful upload a 202 status code is returned and a JSON document like the previous one with offset containing the total amount transferred.

If uploadId doesn’t exist then a 404 error code is returned. If offset parameter is not the one that the server expects, then a 400 error code is returned.

To mark that an update has finished we simply need to send a POST to /commitChunkedUpload specifing which upload has been finished using uploadId query parameter, and in body content all metadata information that may be required like name of the file, tags, …​ and it will return a 201 status code. In case of uploadId is not found then a 404 error code is returned.

2.13. Security

Security in RESTful Web Services can be implemented in several ways and following different protocols, some of them:

  • Basic Authentication with TLS

  • OAuth 1.0a

  • OAuth2

  • SAML 2.0

  • “Tokens Approach” with JSON Web Token

Based on previous experience and after studying all of them, “Tokens Approach” with JSON Web Token is enough.

2.13.1. JSON Web Token

Instead of supplying credentials such as a username and password with every request, we can allow the client to exchange identification data in a token.

The idea behind “Tokens Approach” is to generate a signed token with some information and send it with every API call. Then in server side the token is verified and if it is correct, we can get some parameters from token and use them in authorization mechanism.

The initial authentication process is out of scope of JSON Web Token, but in this case login and password are going to be sent . After that server authenticates the user the token is created and returned to caller with user information like, loginname, name or any parameter about client required by caller with an HTTP 200 code. Since then the only thing that client and server exchange regarding to authentication is the token. Token is sent to front and back using the HTTP header attribute x-access-token.

JSON Web Token
Figure 2. JSON Web Token

But how is it calculated the token?

The JSON Web Token IETF document defines how token is calculated. A JWT is split into three parts, separated by periods and each one is encoded separately. It can be summarized as <base64-encoded header>.<base64-encoded claims>.<base64-encoded signature>.

The first part is a JOSE Header. It is an encoded string representation of a simple JSON document which describes the token along with the hashing algorithm used.

JWT using HMAC SHA-256
{
  "typ" : "JWT", (1)
  "alg" : "HS256" (2)
}
1 Encoded object is a JSON Web Token.
2 JWT is a JWS MACed using HMAC SHA-256.

The second part of the JWT forms the core of the token. It is known as Claims It is represented by JSON document too and contains a few pieces of information described in https://tools.ietf.org/html/draft-ietf-oauth-json-web-token-30#section-4. In next example we can see a simple claim.

JWT Claims
{
  "iss": "example.com", (1)
  "sub": "john", (2)
  "exp": 1300819380, (3)
  "clientIp": "127.0.0.1" (4)
}
1 Identifies the principal that issued the JWT.
2 Identifies the principal that is the subject of the JWT.
3 Identifies the expiration time on or after which the JWT must not be accepted for processing.
4 You can also add custom attributes not defined in the spec like clientIp.

The third part of the JWT is a signature generated based on the header (part one) and the body (part two).

An example of token may look like: eyJ0eXAiOiJKV1QiLA0KICJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJqb2UiLA0KICJleHAiOjEzMDA4MTkzODAsDQogImh0dHA6Ly9leGFtcGxlLmNvbS9pc19yb290Ijp0cnVlfQ.dBjftJeZ4CVP-mB92K27uhbUJU1p1r_wW1gFWFOEjXk.

JWT does not encrypt the payload, it only signs it. You should not send any secret information using JWT, rather you should send information that is not secret but needs to be verified.

If JWT contains sensitive data you may need to take measures to prevent diclosure of this information to unintended parties. One way to achieve this is to use an encrypted JWT and authenticate the recipient. Another way is to ensure that JWTs containing unencrypted privacy-sensitive information are only transmitted using protocols utilizing encryption that support endpoint authentication, such as TLS.

You could add an additional layer of security by storing a record of issued tokens on the server, then verifying them against that record on each subsequent request. This would prevent a third-party from ‘spoofing’ a token, and also allows the server to invalidate a token.

Also you can implement rotatory key approach so key used for creating the token change every specified time or requests.

2.13.2. Authentication

In JSON Web Token we have only covered the authorization part but not he authentication one. This section is about authorization and how to implement it.

Authorization shall be done by sending a JSON document using POST method to /login. This document contains two attributes username and password and may contain extra attributes like pincode, captcha, …​

Login Example
{
  "username" : "john",
  "password" : "1234"
}

Because authentication data is considered sensitive, it must be transmitted using TLS as encryption protocol.

Encryption protocol may change depending on requirements of the project.

In case of successful login, an HTTP 200 status code is returned containing a JSON document as response. This document may contain a JSON object called user with information required by frontend about the user such as full name, date of birth or username. Also it must contains a token attribute which contains the JSON Web Token calculated as described in this paragraph.

Response Example
{
  "user" : {
    "fullName" : "John Smith",
    "username" : "john"
  },

  "token" : "123ACFF234...."
}

In case of unsuccessful login, an HTTP 401 status code is returned containing a JSON document as response following Error Codes spec.

For logout, there shall be a /logout endpoint under DELETE method which logouts the user identified by the passed token. Token is passed as any other request as an HTTP header attribute named x-access-token. As response an HTTP 204 status code is returned.

3. RAML

RESTful API Modeling Language RAML is a simple and succinct way of describing practically-RESTful APIs. It encourages reuse, enables discovery and pattern-sharing, and aims for merit-based emergence of best practices. The goal is to help our current API ecosystem by solving immediate problems and then encourage ever-better API patterns. RAML is built on broadly-used standards such as YAML and JSON and is a non-proprietary, vendor-neutral open spec.

An example of RAML document may look like:

music.raml
#%RAML 0.8 (1)

title: World Music API
baseUri: http://example.api.com/{version}
version: v1
traits: (2)
  - paged:
    queryParameters:
      pages:
        description: The number of pages to return
        type: number
  - secured: !include http://raml-example.com/secured.yml (3)
/songs: (4)
  is: [ paged, secured ]
  get:
    queryParameters:
      genre:
        description: filter the songs by genre
  post:
  /{songId}: (5)
    get:
      responses:
        200:
          body:
            application/json:
              schema: |
                { "$schema": "http://json-schema.org/schema",
                "type": "object",
                "description": "A canonical song",
                "properties": {
                  "title":  { "type": "string" },
                  "artist": { "type": "string" }
                  },
                  "required": [ "title", "artist" ]
                }
  delete:
    description: |
      This method will *delete* an **individual song** (6)
1 For every API, start by defining which version of RAML you are using, and then document basic characteristics of your API - the title, version, and baseURI.
2 RAML allows you to define patterns using traits, resourceTypes, and securitySchemes, and then use them within a API to minimize repetition.
3 Externalize those patterns, store them on the web, and import them with an !include.
4 Easily define resources and methods, then add as much detail as you want. Apply traits and other patterns, or add parameters and other details specific to each call.
5 Describe expected responses for multiple mime-types and specify schemas and examples for each one. Schemas and examples can be defined in-line, or externalized with !include.
6 Write human-readable, markdown-formatted descriptions throughout your RAML spec, or include entire markdown documentation sections at the root.

3.1. RAML API-Designer

RAML has a RAML editor that can be installed locally. The editor and the instructions to install it are found in https://github.com/Scytl/api-designer.

4. JAX-RS

Java API for RESTful Web Services (JAX-RS) is the specification that provides support in creating web services according to the REST architectural pattern.

RESTful Web Services are configured by using annotations. These annotations can be used in POJOs but also in EJBs. In next examples and for maintaining us container agnostic, we are only going to use JAX-RS specification classes which means they will be portable across containers and JAX_RS implementations. Also to be focused on REST, we are not going to use EJBs but pure POJOs.

About Apache TomEE

Apache TomEE is a Java EE certified web profile application server. We can summarize as Apache TomEE = Apache Tomcat + Java EE.

RESTful Web Services can be done in Java by using any of JAX-RS implementation like Jersey, Apache CXF or REASTEasy. Any of this technologies requires a servlet container to run. All of them can be integrated inside any servlet container like Tomcat, Jetty, …​ but in these examples we are going to use Apache TomEE which it comes with JAX-RS provider integrated.

4.1. @Path

@Path is used to define the endpoint URL of RESTful web service. It can be templatized to pass path parameters from URL path. Query parameters are not set as URL parameters and in case we need to get query parameters, different annotation must be used.

Let’s see first example of simple RESTful Web Service which gets all books using GET HTTP method.

BookResource.java
import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

@Path("/books") (1)
public class BookResource {

  @GET (2)
  @Produces(MediaType.APPLICATION_JSON) (3)
  public Response books() {
    List<Book> books = Arrays.asList(new Book("Book1"), new Book("Book2"));
    String json = toJson(books); (4)
    return Response.ok(json).build(); (5)
  }

}
1 @Path sets a class as REST endpoint. In this case is bound to /books.
2 @GET sets which HTTP method is required for executing this method.
3 Because of content negotiation, the media type must be set. In this case because method does not receive anything, we only need to configure the return type with @Produces annotation.
4 Java EE specification does not provide any specific way to convert from/to object to JSON. There is no JSON-B spec. Most providers implements their own system for converting POJOs into required internet type, but you can also converting entities by your own code.
5 Finally instead of returning a list of books alone, a list of books among with an HTTP 200 code is returned.

Then we can access this resource in http://<host>:<port>/<application>/books.

JAVA EE 8 will face the problem of standarizing the JSON Binding problem. Since that point you can use the method provided by JAX-RS provider, or you can implement your own.

What JAX-RS specification provides is an standard way to deal with binding by providing a standard interface to be implemented for converting ‘string’ content into object and viceversa. It is important to know exactly which library uses JAX-RS for binding JSON. If this library fits your requirements, then go ahead with it but keep in mind that you are loosing portability across vendors. If you want to use another library or you need to maintain portability then use the standard annotations to implement your own method.

One way to pass parameters to REST endpoints is by specifying URI path parameter. In previous example there was no parameters so all books were returned. In next example we are going to pass book id of required book, so instead or returning a list of books, a single book is returned.

BookResource
@Path("/books")
public class BookResource {

  @GET
  @Path("{bookId}") (1) (2)
  @Produces(MediaType.APPLICATION_JSON)
  public Response book(@PathParam("bookId") String bookId) { (3)

    Book book = findBookById(bookId); (4)

    if(book != null) {
      return Response.ok(book).build();
    } else {
      return Response.status(Status.NOT_FOUND).build(); (5)
    }

  }
}
1 @Path annotation can be used in a method as well. It simply adds a new path to the path defined at class level.
2 URI path templates are variables denoted by curly braces. At runtime this template is substituted by real value.
3 To get path template in method @PathParam annotation is used passing the template name.
4 Value is set to variable and can be used directly without any transformation.
5 In case of book not found an HTTP 404 code response is sent back. More information about exception handling in next sections.

We can access this resource in http://<host>:<port>/<application>/books/123.

Caller can add any URL valid character as path parameter. In most cases the path parameter will be an integer or a it will contain a well-known pattern. To protect your endpoints you can set a regular expression on path template.

BookResource
@Path("/books")
public class BookResource {

  @GET
  @Path("{bookId: [0-9]+}") (1)
  @Produces(MediaType.APPLICATION_JSON)
  public Response book(@PathParam("bookId") String bookId) {

    Book book = findBookById(bookId);

    if(book != null) {
      return Response.ok(book).build();
    } else {
      return Response.status(Status.NOT_FOUND).build();
    }

  }
}
1 The way to add a regular expression is by adding the name of the path param and the regular expression separated by colon ‘:’.

Resource in http://<host>:<port>/<application>/books/123 will work but http://<host>:<port>/<application>/books/abc returns an HTTP 404 code error.

4.2. Extracting request parameters

In <<@Path>> we have seen that we can pass path parameters and extract them using @PathParam. But there are other ways to send parameters in RESTful Web Services:

@PathParam

Extracts parameters from URL path.

@QueryParam

Extracts parameters from query path.

@FormParam

Extracts parameters from a request of MIME media type application/x-www-form-urlencoded.

@MatrixParam

Extracts parameters from an HTTP matrix parameters. Matrix parameters are a set of name=value in URL path separated by semicolon ‘;’.

@HeaderParam

Extracts parameters from HTTP header.

@CookieParam

Exrtacts parameters from a cookie.

Avoid using @FormParam because you are tighten endpoint with presentation layer. Also prefer using @QueryParam in front of @MatrixParam.

All previous ways of extracting parameters from a request (@PathParam, @FormParam, @QueryParam …​) can be preceded by @DefaultValue annotation. This annotation sets the default value in case a parameter is not provided.

BookResource
@GET
public String books(@DefaultValue("100") @QueryParam("price")int maxResults) {
  return ...;
}

4.2.1. @QueryParam

@QueryParam extracts parameters from query path. Query is an optional part separated from path with a question mark ‘?’ and contains pairs of key/value separated by ampersand '&'.

BookResource
@Path("/books")
public class BookResource {

  @GET
  @Produces(MediaType.APPLICATION_JSON)
  public Response books(@QueryParam("orderBy")String field) { (1) (2)
    List<Book> books = findAllBooksOrderedBy(field);
    return Response.ok(books).build();
  }
}
1 @QueryParam is used in the same way as @PathParam.
2 In case of no query param provided, the endpoint is called as well but a null value is set in parameter.

4.2.2. @HeaderParam

@HeaderParam extracts parameters from HTTP header.

BookResource
@Path("/books")
public class BookResource {

  @GET
  @Produces(MediaType.APPLICATION_JSON)
  public Response books(@HeaderParam("user-agent")String userAgent) {
    //..
  }
}

4.2.3. @CookieParam

@CookieParam exrtacts parameters from a cookie.

BookResource
@Path("/books")
public class BookResource {

  @GET
  @Produces(MediaType.APPLICATION_JSON)
  public Response books(@CookieParam("Last-Accessed")String lastAccessed) {
    //..
  }
}

4.3. HTTP methods

In all previous examples GET method has been used as a preferred HTTP method. But you can use any other HTTP method like POST for creating resources, PUT for updating resources or DELETE for deleteing resources.

4.3.1. Creating a resource

To create a resource we must use the POST HTTP method with @POST annotation.

BookResource
@Path("/books")
public class BookResource {

  @POST (1)
  @Consumes(MediaType.APPLICATION_JSON) (2)
  public Response createBook(String jsonBook) { (3)
    Book book = fromJson(jsonBook); (4)
    book = insertBook(book);

    return Response.created(URI.create(book.getId())).build(); (5)
  }
}
1 This method is executed when request is of type POST.
2 This method consumes content instead of producing it. For this reason we must set which media type is consumed by the service.
3 JSON payload is set as ‘string’.
4 As mentioned in JSON section there is no specification about how to bind an object from/to JSON. You can implement your own provider or you can leave this responsability to JAX-RS default provider.
5 Creation implies to assign an id to a resource. This id must be returned as HTTP header parameter named Location and HTTP 201 status code.

4.3.2. Updating a resource

To update a resource we must use the PUT HTTP method with @PUT annotation.

BookResource
@Path("/books")
public class BookResource {

  @PUT (1)
  @Path("{bookId: [0-9]+}")
  @Consumes(MediaType.APPLICATION_JSON)
  public Response updateBook(@PathParam("bookId") String bookId, String jsonBook) {
    Book book = fromJson(jsonBook);
    book = updateBook(book);

    return Response.noContent().build(); (2)
  }
}
1 This method is executed when request is of type PUT.
2 When an update is produced a no content HTTP code should be returned.

4.3.3. Deleting a resource

To delete a resource we must use the DELETE HTTP method with @DELETE annotation.

BookResource
@Path("/books")
public class BookResource {

  @DELETE (1)
  @Path("{bookId: [0-9]+}")
  @Consumes(MediaType.APPLICATION_JSON)
  public Response deleteBook(@PathParam("bookId") String bookId) {
    //delete

    return Response.noContent().build(); (2)
  }
}
1 This method is executed when request is of type DELETE.
2 When an update is produced a no content HTTP code should be returned.

4.4. Content Negotiation

RESTful Web Services can consume and produce different media type like JSON, XML or any other valid type like plain text or binary.

JAX-RS provides @Consumes and @Produces annotations to set which media type are consumed by the service or produced to the client. In all previous examples application/json media type has been used, but different kind of type can be used as well. Next list provides a quick overview of the Java types that are supported with respect to media type.

  • All media types (*/*)

    • byte[]

    • java.lang.String

    • java.io.Reader (inbound)

    • java.io.File

    • javax.activation.DataSource

    • javax.ws.rs.core.StreamingOutput (outbound)

  • XML media types (text/xml, application/xml and application/…​+xml)

    • javax.xml.transform.Source

    • javax.xml.bind.JAXBElement

    • Application supplied JAXB classes (types annotated with @XmlRootElement or@XmlType)

  • JSON media types (text/json, application/json) [1]

    • javax.xml.transform.Source

    • javax.xml.bind.JAXBElement

    • Application supplied JAXB classes (types annotated with @XmlRootElement or@XmlType)

  • Form content (application/x-www-form-urlencoded)

    • MultivaluedMap<String,String>

  • Plain text (text/plain)

    • java.lang.Boolean

    • java.lang.Character

    • java.lang.Number

4.5. Errors

RESTful Web Services can fail because of different situations. Some of them can be business errors thrown by our process, and others can be errors generated by a third-party library or because of structure problems.

4.5.1. Response

Exceptions in JAX-RS are propagated through HTTP response codes as explained in Response Codes section. JAX-RS defines javax.ws.rs.core.Response class to create response code and sent back to the client.

BookResource.java
@GET
@Produces(MediaType.APPLICATION_JSON)
public Response book(@PathParam("bookId") String bookId) {

  Book book = findBookById(bookId);

  if (book != null) {
    return Response.ok(book).build();
  } else {
    return Response.status(Status.NOT_FOUND).build(); (1)
  }
}
1 Because element is not found an HTTP 404 error code is thrown to the client.

That’s right for almost all cases, but what’s happening if findBookById instead of returning a null value, it would return an un/checked exception?

4.5.2. Exceptions

Probably the first thing we could do is wrapping code between a try/catch instruction and in catch section add the Response call.

But JAX-RS defines javax.ws.rs.ext.ExceptionMapper to map any exception (checked or not) to an HTTP response code. So if the exception is thrown, transparently to the developer, this exception is transformed to HTTP response code and sent back to client. This class is automatically discovered byt the JAX-RS runtime during provider scanning phase.

PersistenceExceptionMapper.java
import javax.ws.rs.ext.Provider;
import javax.persistence.PersistenceException;

@Provider (1)
public class PersistenceExceptionMapper implements ExceptionMapper<PersistenceException> {

  @Override
  public Response toResponse(PersistenceException exception) { (2)
    return Response.status(Status.INTERNAL_SERVER_ERROR)
          .entity(createErrorMessage(exception.getMessage())).type("application/json").build(); (3)
  }
}
1 Mapper should be annotated with @Provider annotation.
2 toResponse method builds the response to be sent back to caller.
3 Along with HTTP code, a message about the error is also returned following error nomenclature.

4.6. Custom Entity Providers

As we have seen in all previous examples, JAX-RS does not define a way on how to bind an object to its JSON representation and viceversa. As mentioned this binding can be done by using the one provided by JAX-RS implementation. For example Apache CXF provides Jettison. If we choose this approach the application would become not portable across providers, but also we could feel more comfortable with other binder like Gson or Jackson.

To make binders portable, JAX-RS defines javax.ws.rs.ext.MessageBodyWrite interface to map an object to JSON and javax.ws.rs.ext.MessageBodyReader to map a JSON document to object.

Let’s see an example of binder which uses Gson as mapper.

About Gson

Gson is a java library that can be used to convert java objects into their JSON representation. It can also be used to convert a JSON string to an equivalent java object. Gson can work with arbitrary java objects including pre-existing objects that you do not have source-code of.

GsonMessageBodyWriter.java
@Provider (1)
@Produces("application/json") (2)
public class GsonMessageBodyWriter implements MessageBodyWriter<Object> {

  private Gson gson = new Gson();

  @Override
  public long getSize(Object object, Class<?> clazz, Type type, Annotation[] annotations,
      MediaType mediaType) {
    return -1; (3)
  }

  @Override
  public boolean isWriteable(Class<?> clazz, Type type, Annotation[] annotations,
      MediaType mediaType) {
    return true; (4)
  }

  @Override
  public void writeTo(Object object, Class<?> clazz, Type type, Annotation[] annotations,
      MediaType mediaType, MultivaluedMap<String, Object> multivaluedMap,
      OutputStream outputStream) throws IOException {
    String json = gson.toJson(object);
    outputStream.write(json.getBytes()); (5)
  }
}
1 Marks an implementation of an extension interface that should be discoverable by JAX-RS runtime during a provider scanning phase.
2 @Produces sets which media type should enable this writer.
3 If size of return message cannot be calculated a -1 should be returned.
4 Sets which kind of objects can be converted using this writer. In this case all of them.
5 gson serializes object.

And inverse operation:

GsonMessageBodyReader.java
@Provider
@Consumes("application/json") (1)
public class GsonMessageBodyReader implements MessageBodyReader<Object> {

  private Gson gson = new Gson();

  @Override
  public boolean isReadable(Class<?> clazz, Type type, Annotation[] annotations,
      MediaType mediaType) {
    return true;
  }

  @Override
  public Object readFrom(Class<Object> clazz, Type type, Annotation[] arguments,
      MediaType mediaType, MultivaluedMap<String, String> multivaluedMap,
      InputStream inputStream) throws IOException {
    return gson.fromJson(new InputStreamReader(inputStream), type); (2)
  }
}
1 @Consumes is used instead of @Produces.
2 Input stream is converted to required object.

These class is automatically discovered byt the JAX-RS runtime during provider scanning phase because both of them are annotated with @Provider.

A single object implementing both interfaces are the common pattern followed in this cases.

Jackson mapper has one artifact which implements a JAX-RS provider. So if you want to use Jackson as mapper you don’t need to implement it by yourself but just adding a new dependency in your pom.xml file or copying manually the required artifacts into WEB-INF/lib.

pom.xml
<dependency>
  <groupId>com.fasterxml.jackson.jaxrs</groupId>
  <artifactId>jackson-jaxrs-json-provider</artifactId>
  <version>2.4.3</version>
</dependency>

4.7. Packaging and Deployment

JAX-RS applications can be packaged and deployed using different three different approaches:

Application subclass

Use a class that extends javax.ws.rs.core.Application to define the components of a Restful Web Services and provide additional metadata.

Servlet

Update the web.xml deployment descriptor to configure a servlet as dispatcher for RESTful Web Services.

Default

If you don’t configure anything, RESTful Web Services are deployed as defined in classes. An scanning classpath fiding resources are done.

In all our previous examples Default method has been used.

4.7.1. Application

Use a class that extends javax.ws.rs.core.Application to define the components of a Restful Web Services and provide additional metadata. Metadata may include a common base URI for endopints or for example defining which endpoints should be mapped within specified prefix.

In next example BookResource endpoint is going to be registered using javax.ws.rs.core.Application.

BookResource.java
@Path("/books")
public class BookResource {
    //....
}
BookApplication.java
import javax.ws.rs.core.Application;
import javax.ws.rs.ApplicationPath;

@ApplicationPath("public") (1)
public class BookApplication extends Application {

  public Set<Class<?>> getClasses() {
    Set<Class<?>> s = new HashSet<Class<?>>();
    s.add(BookApplication.class); (2)
    return s;
  }
}
1 @ApplicationPath adds a base URI to all components registered inside this application. In this case public.
2 BookResource class is registered within current application.

Now instead of accessing the resource through http://<host>:<port>/<application>/books, we need to go to http://<host>:<port>/<application>/public/books.

For simple deployments, no web.xml deployment descriptor is required. For more complex deployments, for example to secure the Web service or specify initialization parameters, you can package a web.xml deployment descriptor with your application.

4.7.2. Servlet

By default javax.ws.rs.core.Application classes are self-discovered during classpath scanning phase. But in case you need to add custom parameters from web.xml to javax.ws.rs.core.Application, the class must be registered within the file.

web.xml
<web-app>
  <servlet>
    <display-name>BookApplication Servlet</display-name>
    <servlet-name>BookApplication</servlet-name>
    <init-param>
      <param-name>javax.ws.rs.Application</param-name>
      <param-value>myPackage.BookApplication</param-value>
    </init-param>
  </servlet>
  <servlet-mapping>
    <servlet-name>BookApplication</servlet-name>
    <url-pattern>/*</url-pattern>
  </servlet-mapping>
</web-app>

4.7.3. Default

By default all resources annotated with @Path or extending javax.ws.rs.Application are auto-discovered when they are at classpath. This is the default behaviour and the most used.

If application that is being developed contains a single (or two) resources it is not necessary to register them as javax.ws.rs.Application, but on other cases, the best approach is defining them inside it so you can have an overall picture of all REST endpoints. Most of the times you don’t need to do anything with web.xml so rely on scanning process.

One important point is to put javax.ws.rs.Application in standard place so anyone can take a quick overview of which resources compose our application and their implementatin. For this reason a common place to put javax.ws.rs.Application class is in rs package. So a valid package is com.superbiz.app.rs.

Credential Manager
     com.superbiz.app.rs
         CredentialManagerResources.java
     com.superbiz.app.credential.boundary
         CredentialResource.java
     com.superbiz.app.credential.entity
         Credential.java

4.8. Testing

Testing RESTful Web Services can be a bit complicated because there are some logic that are executed by a container. As we have seen in previous examples you can annotate a class with @Path and this class will become a REST endpoint. Container is in charge of configuring, registering, and running the endpoint and this is not managed by the developer.

The same happens for serializing a returned entity like in this example. The serialization of an entity is done internally by the container.

So running your integration tests requires that you deploy the business code inside the container. If not you are writing a test that is not exactly running in the same environment as in production and probably you will end up by mocking a lot of container features.

And this is exactly what Arquillian fixes.

Arquillian

Arquillian is a testing framework that fills the gap between unit tests and integration tests so writting integration tests are as simple as writing a unit tests but running them within/against an application server.

It deals with all the plumbing of container management, deployment, and framework initialization so you can focus on the task of writing your tests—real tests. Arquillian minimizes the burden on you by covering aspects surrounding test execution; some of these aspects are as follows:

  • Managing the life cycle of the container (start/stop)

  • Bundling the test class with the dependent classes and resources into a deployable archive

  • Enhancing the test class (for example, resolving @Inject, @EJB, and @Resource injections)

  • Deploying the archive to test applications (deploy/undeploy), and capturing results and failures

You can read more about Arquillian in [ArquillianHome].

So let’s start by adding Arquillian dependencies.

4.8.1. Arquillian dependencies and configuration

pom.xml
<dependencyManagement>
  <dependencies>
    <dependency>
    <groupId>org.jboss.arquillian</groupId> (1)
    <artifactId>arquillian-bom</artifactId>
    <version>1.1.7.Final</version>
    <scope>import</scope>
    <type>pom</type>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.apache.openejb</groupId> (2)
    <artifactId>arquillian-tomee-embedded</artifactId>
    <version>1.7.1</version>
    <scope>test</scope>
  </dependency>
  <dependency>
    <groupId>org.apache.openejb</groupId> (3)
    <artifactId>tomee-jaxrs</artifactId>
    <version>1.7.1</version>
    <scope>test</scope>
  </dependency>
</dependencies>
1 Adds Arquillian bom file. This makes that all component added as dependency will contain exactly the same version.
2 Because we are using Apache TomEE, this dependency adds Apache TomEE as container to be managed by Arquillian.
3 Embedded Apache TomEE doesn’t comes with JAX-RS libraries so we need to add them manually.

Let’s see how an Arquillian test looks like:

4.8.2. Arquillian Test

@RunWith(Arquillian.class) (1)
public class MoviesServiceTest {

  @Deployment (2)
  public static JavaArchive createDeploymentPackage() {
    return createDeploymentFile();
  }

  @EJB (3)
  MoviesService moviesService;

  @Test
  public void shouldCreateMovies() {
    (4)
  }
}
1 Test must use Arquillian runner.
2 Arquillian needs to know what to deploy on the started container. Deployment file is created in an static method annotated with Deployment.
3 By default Arquillian runs tests inside the container so we can use any Java EE annotation available inside the container. In this case javax.ejb.EJB.
4 The test itself using the EJB or whatever we want.

As you may see test looks pretty similar to a unit test, a lot of details about application server or some aspects on how is started and stopped the application server are hidden to the developers. So what is happening under the covers when previous test is executed?

Arquillian Lifecycle
Figure 3. Arquillian Lifecycle

First of all Arquillian starts an embedded Apache TomEE (we have defined this in pom.xml in section 2). Then a method annotated with Deployment annotation is called and returns a JavaArchive to be deployed.

After that the generated deployment file and the test itself is packaged and deployed to Apache TomEE. The test is run inside the container and the results are sent back to IDE. Note that this step is required because tests are run inside the container, and the result should be notified to the caller (surefire plugin, IDE, …​).

And finally if there is no more tests to execute container is cleaned from deployed artifacts and Apache TomEE is stopped.

And how should looks like a @Deployment annotated method? To create a deployment file you need to use a project called Shrinkwrap. Arquillian has already this project as dependency so you don’t need to worry about including it in pom.xml.

@Deployment
public static WebArchive createDeploymentPackage() {
  WebArchive deploymentFile = ShrinkWrap.create(WebArchive.class) (1)
  .addPackage("com.scytl.hibernate") (2)
  .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml") (3)
  return deploymentFile;
}
1 Creates a war file in memory. You could create an ear or jar file as well.
2 Adds inside the war file all classes under the package com.scytl.hibernate. That is inside a virtual directory called WEB-INF/classes as war spec suggests.
3 Adds an empty file called beans.xml as WEB-INF resource. That is in WEB-INF directory.

And with this virtual object file Arquillian manages itself to deploy it to container.

By default Arquillian runs tests inside the container. This is known as in-container mode because Arquillian will package the deployment file along with test and sent it to the container for execution.

But there is also another mode called client. In this case Arquillian will simply deploy the deplpoyment file to the server but the test will remain on client side (running from your JVM). In these cases you are free to test the container from the outside, as any client (for example browser) will see it. So for testing JAX-RS applications it may be really useful using client mode instead of in-container ones.

The Arquillian lifecycle in client mode looks like:

Arquillian container Lifecycle
Figure 4. Arquillian container Lifecycle

Note that now the deployment file is deployed (but not the test) and instead of running test inside container, test is run inside the client and it communicates with container as any other client would do, by using a URL.

Let’s see an example on how to use client mode with RESTful Web Services.

com/scytl/rest/PersonResource.java
package com.scytl.rest;


import javax.ws.rs.GET;
import javax.ws.rs.Path;
import javax.ws.rs.PathParam;
import javax.ws.rs.Produces;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;

@Path("/person") (1)
public class PersonResource {

  private String OUTPUT = "{\n" +
  "  \"firstName\": \"John\",\n" +
  "  \"lastName\": \"Smith\",\n" +
  "  \"isAlive\": true,\n" +
  "  \"age\": 25,\n" +
  "  \"height_cm\": 167.6,\n" +
  "  \"address\": {\n" +
  "    \"streetAddress\": \"21 2nd Street\",\n" +
  "    \"city\": \"New York\",\n" +
  "    \"state\": \"NY\",\n" +
  "    \"postalCode\": \"10021-3100\"\n" +
  "  },\n" +
  "  \"phoneNumbers\": [\n" +
  "    {\n" +
  "      \"type\": \"home\",\n" +
  "      \"number\": \"212 555-1234\"\n" +
  "    },\n" +
  "    {\n" +
  "      \"type\": \"office\",\n" +
  "      \"number\": \"646 555-4567\"\n" +
  "    }\n" +
  "  ],\n" +
  "  \"children\": [],\n" +
  "  \"spouse\": null\n" +
  "}";

  @GET
  @Path("{personId}")
  @Produces(MediaType.APPLICATION_JSON)
  public Response person(@PathParam("personId") String bookId) {
    return Response.ok(OUTPUT).build(); (2)
  }

}
1 Common JAX-RS annotations are used in the example.
2 Service returns a JSON document.

So now we need to write a test. It will use Arquillian to deploy the PersonResource inside container and then use Rest-Assured for connecting to the service and assert the output message.

About REST-Assured

Testing and validating REST services in Java is harder than in dynamic languages such as Ruby and Groovy. REST Assured brings the simplicity of using these languages into the Java domain.

More examples of how to use it can be found at https://code.google.com/p/rest-assured/wiki/Usage.

Only one dependency is required to use REST-assured in a project.

pom.xml
<dependency>
  <groupId>com.jayway.restassured</groupId>
  <artifactId>rest-assured</artifactId>
  <version>2.4.0</version>
  <scope>test</scope>
</dependency>
com/scytl/rest/PersonResourceTest.java
package com.scytl.rest;

import static com.jayway.restassured.RestAssured.get;
import static org.junit.Assert.assertThat;
import static org.hamcrest.CoreMatchers.is;


import com.jayway.restassured.path.json.JsonPath;
import org.jboss.arquillian.container.test.api.Deployment;
import org.jboss.arquillian.junit.Arquillian;
import org.jboss.arquillian.test.api.ArquillianResource;
import org.jboss.shrinkwrap.api.ShrinkWrap;
import org.jboss.shrinkwrap.api.spec.WebArchive;
import org.junit.Test;
import org.junit.runner.RunWith;

import javax.ws.rs.core.UriBuilder;
import java.io.InputStream;
import java.net.URI;
import java.net.URL;

@RunWith(Arquillian.class) (1)
public class PersonResourceTest {

  @Deployment(testable = false) (2)
  public static WebArchive createDeployment() {
    return ShrinkWrap.create(WebArchive.class, "persontest.war")
    .addClass(PersonResource.class);
  }

  @ArquillianResource (3)
  URL baseUrl;

  @Test
  public void shouldReturnExpectedPerson() {
    URI endpoint = UriBuilder.fromPath(baseUrl.toExternalForm()).path(PersonResource.class).path("1").build(); (4)
    InputStream response = get(endpoint).then().statusCode(200).extract().asInputStream(); (5)

    JsonPath jsonPath = new JsonPath(response);
    String firstName = jsonPath.getString("firstName"); (6)
    String streetAddress = jsonPath.getString("address.streetAddress"); (7)

    assertThat(firstName, is("John"));
    assertThat(streetAddress, is("21 2nd Street")); (8)
  }

}
1 Tests should be annotated with Arquillian runner.
2 Sets test to be run in container by using testeable = false.
3 The deployed file is deployed in a server. Using @ArquillianResource the URL where the file is deployed is injected on test.
4 Creates the URI where endpoint is mapped.
5 Uses get static method (provided by REST-assured) to do an invocation using HTTP GET to procided direction and asserts that the status code is the expected one.
6 JsonPath class allows you to use XPath like expressions but for JSON.
7 streetAddress value is got from returned JSON document.
8 Standard assertThat method from JUnit is used.

4.9. Advanced

4.9.1. Matrix Params

4.9.2. Sub-resource locator

Resource classes can partially process a request and then provide another sub-resource object to process the remainder of the request. This approach can be used in different situations, but one situation that fits pretty well is to deal with “M to 1” relationships.

Resource methods with a @Path annotation and no HTTP method are considered sub-resource locators and they provide an object that can process the request.

Let’s see an example. In previous examples we have worked with Book entity which can be something like:

Book.java
public class Book {

  private long id;
  private String title;

  public Book(long id, String title) {
    this.title = title;
    this.id = id;
  }

  public String getTitle() {return title;}
  public int getId() {return 1;}
}

And when GET /book/12 is executed, the book with id 12 is returned.

But let’s add a Chapter entity and how it is related with Book.

Chapter.java
public class Chapter {

  private long id;
  private String title;
  private String content;

  private Book book;

  public Chapter(long id, String title, String content, Book book) {
    this.id = id;
    this.title = title;
    this.content = content;
    this.book = book;
  }

  public long getId() { return id; }
  public String getTitle() { return title; }
  public String getContent() { return content; }
  public Book getBook() { return book; }
}

One book contains one or more chapters. So new book entity should be modified to:

Book.java
public class Book {

  private long id;
  private String title;

  private List<Chapter> chapters = new ArrayList<Chapter>();

  public Book(long id, String title) {
    this.title = title;
    this.id = id;
  }

  public void addChapter(Chapter chapter) { this.chapters.add(chapter); }
  public List<Chapter> getChapters() { return chapters; }
  public String getTitle() { return title; }
  public long getId() { return id; }
}

To return all chapters of given book, you can do something like:

BookResource.java
@Path("/books")
public class BookResource {

  @GET
  @Path("{bookId: [0-9]+}/chapters")
  @Produces(MediaType.APPLICATION_JSON)
  public Response chapters(@PathParam("bookId")Long bookId) {
    Book book = findBookById(bookId);
    return Response.ok(book.getChapters()).build(); (1)

}
1 List of chapters are returned from BookResource class.

We can access this resource in http://<host>:<port>/<application>/books/123/chapters and the list of chapters will be returned.

No secret here and no sub resources involved yet. But note that we are mixing books with chapters in same endpoint. It is better to maintain classes as simple as possible, and more important doing the work they were designed for, in case of BookResource it is dealing with books and not chapters.

So one way is make Book as sub resource. Thanks of subresources, all RESTful endpoint matching releated with returning a list of chapters will be delegated to Book itself. Because Book is responsible of knowing that it contains a list of chapters, it should be the responsible of returning them as well. Let’s see how to do it in JAX-RS.

First thing to do is set that chapters method from BookResource returns a subresource instead of a resource. This is accomplished by removing HTTP method annotation.

BookResource.java
@Path("/books")
public class BookResource {

  @Path("{bookId: [0-9]+}") (1) (2)
  public Book chapters(@PathParam("bookId")Long bookId) {

    Book book = findBookById(bookId);
    return book; (3)
  }
}
1 Only path realated with book is set.
2 No HTTP method annotation is present.
3 Finally we return the subresource.

And next step is define that Book is a subresource and must contains the path part not set in BookResource.

Book.java
public class Book {

  private long id;
  private String title;

  private List<Chapter> chapters = new ArrayList<Chapter>();

  public Book() {
  }

  public Book(long id, String title) {
    this.title = title;
    this.id = id;
  }

  public void addChapter(Chapter chapter) {
    this.chapters.add(chapter);
  }

  @GET (1)
  @Produces(MediaType.APPLICATION_JSON)
  @Path("/chapters") (2)
  public List<Chapter> getChapters() {
    return chapters; (3)
  }
}
1 HTTP method is set so this method returns a full resource.
2 @Path sets the subresource path.
3 Returns a list of all chapters.

We can access this resource in http://<host>:<port>/<application>/books/123/chapters and the list of chapters will be returned. But now the code will do next steps. First of all it will run http://<host>:<port>/<application>/books/123 in BookResource. Because it returns a subresource, the engine will continue inspecting the part not processed of the URI, in this case /chapters, through returned subresource. So it will find that Book class contains a method annotated with @Path("/chapters"), and because it matches, the method will be executed by returning a list of chapters.

You can use the same approach to return single attribute in case client needs to access them from your REST API, for example /books/title.

In this example the entity class has been used as subresource. But if you want to maintain your entity classes anemic, you can create a subresource class such as BookSubresource that wraps Book.

4.9.3. Versioning

As seen in REST versioning, RESTful Web Services are going to be versioned (in case it is necessary) using URL path to set version number of API.

Versioning can be implemented following different approaches, but using Application may be cleaner way to implement it.

Let’s suppose we have different resources and we want to give support to two different versions ‘v1’ and ‘v2’. The first thing to do is create two applications, one for each version.

V1Application.java
@ApplicationPath("/v1") (1)
public class V1Application {

  @Override
  public Set<Class<?>> getClasses() {
    Set<Class<?>> s = new HashSet<Class<?>>();
    s.add(ChapterResourceV1.class); (2) (3)
    return s;
    }
}
1 Version 1 is identified as ‘v1’ in URL.
2 It registers the Chapter resource that contains the implementation of version 1.
3 And the same for all resources available in version 1 of the API.
ChapterResourceV1.java
public class ChapterResourceV1 {

  @GET
  @Path("/") (1)
  @Produces(MediaType.APPLICATION_JSON)
  public Chapter chapter() {
    return ...;
  }
}

And then a second version of the API is developed. What we need to do is create a new application for version 2.

V2Application.java
@ApplicationPath("/v2") (1)
public class V2Application {

  @Override
  public Set<Class<?>> getClasses() {
    Set<Class<?>> s = new HashSet<Class<?>>();
    s.add(ChapterResourceV2.class);
    return s;
    }
}
1 Version 2 is identified as ‘v2’ in URL.

We can access this resource in http://<host>:<port>/<application>/v1/chapter and the list of chapters will be returned using ‘v1’ resource. Accessing http://<host>:<port>/<application>/v2/chapter the list of chapters will be returned as wel but using ‘v2’ resource instead.

4.9.4. PATCH and JAX-RS

Current version of JAX-RS (v2.0) does not implement PATCH HTTP method. But JAX-RS allows developers to implement their own methods, so in this case we could implement by ourselves the PATCH one.

PATCH.java
@Target({ElementType.METHOD})
@Retention(RetentionPolicy.RUNTIME)
@HttpMethod("PATCH") (1)
public @interface PATCH {

}
1 To implement an HTTP mehtod, HttpMethod must be used.

And PATH can be used without any problem as GET, POST, …​

BookResource.java
@PATCH (1)
@Consumes("application/json-patch") (2)
public Response patch(JsonArray jsonObject) { (3)

  //...

  return Response.noContent().build();
}
1 PATCH annotation is used without any problem.
2 Content-Type for Patch operation should be application/json-patch.
3 Patch content is an array of operations to be executed.

We can access this resource in http://<host>:<port>/<application>/book with HTTP method PATCH and Content-Type as application/json-patch.

4.9.5. Chunk Upload

In Chunked Upload we described how implement an upload of a huge chunked bytes in REST.

Initial upload
@Inject
TemporaryFolder temporaryFolder;

@POST
@Path("/chunkedUpload")
@Consumes("*/*")
@Produces(MediaType.APPLICATION_JSON)
public Response chunked(InputStream reader) throws IOException {
  UUID uploadId = UUID.randomUUID(); (1)

  java.nio.file.Path file = temporaryFolder.newFile(uploadId.toString()); (2)
  long offset = IOUtil.copyToFile(file, reader, Integer.MAX_VALUE); (3)
  return Response
    .status(202)
    .entity("{\n" + "  \"uploadId\": \"" + uploadId + "\",\n"
      + "  \"offset\": " + offset + "\n" + "}").build();
}
1 Generates a random number as uploadId.
2 Creates a new file using the generated UUID in a temporal directory.
3 Copies content to temporal file. Lat argument is to limit the amount of bytes that can be saved per chunked.
Successive calls
@PUT
@Path("/chunkedUpload")
@Consumes("*/*")
@Produces(MediaType.APPLICATION_JSON)
public Response chunked(InputStream reader, @QueryParam("uploadId") String uploadId, @QueryParam("offset") Long offset) throws IOException {
  File file = new File(temporaryFolder.getRoot(), uploadId);

  if(isUploadIdValid(file)) {
    if(isOffsetValid(file, offset)) { (1)
      long newOffset = IOUtil.copyToFile(file.toPath(), reader, Integer.MAX_VALUE); (2)

      return Response.status(202).entity("{\n" +
        "  \"uploadId\": \""+uploadId+"\",\n" +
        "  \"offset\": " + (offset + newOffset) + "\n" +
        "}").build();

    } else {
      return Response.status(400).build();
    }
  } else {
    return Response.noContent().build();
  }
}

private boolean isOffsetValid(File file, long offset) {
  return file.length() == offset;
}

private boolean isUploadIdValid(File file) {
  return file.exists();
}
1 Checks if uploadId exists and if the offset is correct.
2 Appends content to the already created file.
Commit
@POST
@Path("/commitChunkedUpload")
@Consumes(MediaType.APPLICATION_JSON)
@Produces(MediaType.APPLICATION_JSON)
public Response commit(@QueryParam("uploadId") String uploadId, String content) {
  File file = new File(temporaryFolder.getRoot(), uploadId);

  if(isUploadIdValid(file)) {
    JsonReader parser = Json.createReader(new StringReader(content));
    JsonObject readObject = parser.readObject(); (1)
    file.renameTo(new File(temporaryFolder.getRoot(), readObject.getString("filename"))); (2)

    return Response.created(URI.create(uploadId)).build();
  } else {
    return Response.noContent().build();
  }
}
1 Parses the JSON document-
2 Gets filename provided and renames the uploaded file. Note that this operation can be insecure and should require security checks depending on project.

And finally a grey box test which allows us test from client side and server side at same time:

RestUploadTest.java
@RunWith(Arquillian.class)
@WarpTest (1)
public class RestUploadTest {

  @Deployment
  public static final WebArchive deployment() {
    return ShrinkWrap
      .create(WebArchive.class)
      .addClasses(BoundedInputStream.class, IOUtil.class,
        P12Resource.class, TemporaryFolder.class)
      .addAsWebInfResource(EmptyAsset.INSTANCE, "beans.xml")
      .addAsLibraries(
        Maven.resolver().loadPomFromFile("pom.xml")
        .resolve("org.glassfish:javax.json")
        .withTransitivity().as(JavaArchive.class));
  }

  @ArquillianResource
  URL baseUrl;

  @Test
  @RunAsClient (2)
  public void shouldUploadContent() {
    Warp.initiate( (3)
        () -> {
          URI endpoint =
          UriBuilder.fromPath(baseUrl.toExternalForm())
            .path(P12Resource.class).path("chunkedUpload")
            .build();
          given().request().body("This is a test file.".getBytes())
            .post(endpoint).then().assertThat()
            .body("uploadId", notNullValue()).and()
            .body("offset", notNullValue());

        })
        .inspect(new Inspection() { (4)
          private static final long serialVersionUID = 1L;

          @Inject
          TemporaryFolder temporaryFolder;

          @AfterServlet
          public void assertCreatedFile() {
            asserThat(temporaryFolder.getRoot(), notNullValue());
          }

        });
  }
}
1 Sets a test as Arquillian Warp test.
2 @Deployment sets test as server test and test method is configured to run in client.
3 initiate method is executed on client side.
4 inspect method is executed on server side.

4.9.6. REST Polling

4.9.7. Security

Authentication

As mentioned in JSON Web Token section, JWT is the approach used to secure RESTful Web Service endpoints. For this example and to simplify things we are going to use the JAX-RS 2.0 specification because it implements a concept called server filters that really fits our requirements. Filters can be used when you want to modify any request or response parameters as well as implementing authorization strategies.

First thing is creating a class that represents our user and it must implements java.security.Principal.

User.java
public class User implements Principal {

  private String username; (1)

  private List<String> roles = new ArrayList<String>();

  public User(String username, String... roles) {
    super();
    this.username = username;
    this.roles = Arrays.asList(roles);
  }

  @Override
  public String getName() { (2)
    return username;
  }

  public boolean isUserInRole(String role) { (3)
    return this.roles.contains(role);
  }
}
1 All required information about user can be set as attributes. In this example only username is required.
2 Principal has only one method to override and it is an string that identifies a user.
3 Checks if current user belongs to passed role.

Next step is create an authentication filter. This filter is responsible of getting token from x-access-token header attribute, validate the token and get user information to be able to find the roles which belongs, and finally create the security context based on jwt.

JWTAuthenticationFilter.java
@Provider (1)
@Priority(Priorities.AUTHENTICATION) (2)
public class JWTAuthenticationFilter implements ContainerRequestFilter { (3)

  private static final List<Class<? extends Annotation>> securityAnnotations = Arrays.asList(DenyAll.class, PermitAll.class, RolesAllowed.class);

  @Context
  private ResourceInfo resourceInfo; (4)

  @Override
  public void filter(ContainerRequestContext request) throws IOException {


      if(isSecuredResource()) { (5)

        String token = request.getHeaderString("x-access-token"); (6)

        try {
          String username = getUsernameFromToken(token); (7)
          final User user = getUserByName(username); (8)

          request.setSecurityContext(new SecurityContext() { (9)

            @Override
            public boolean isUserInRole(String role) { return user.isUserInRoles(role);}

            @Override
            public boolean isSecure() { return false;}

            @Override public Principal getUserPrincipal() { return user;}

            @Override
            public String getAuthenticationScheme() { return SecurityContext.BASIC_AUTH;}
            });

        } catch (ParseException | JOSEException e) {
          request.abortWith(Responses.NOT_FOUND);
        }
      }
  }

  private boolean isSecuredResource() {

    for (Class<? extends Annotation> securityClass : securityAnnotations) {
      if(resourceInfo.getResourceMethod().isAnnotationPresent(securityClass)) {
        return true;
      }
    }

  for (Class<? extends Annotation> securityClass : securityAnnotations) {
      if(resourceInfo.getResourceClass().isAnnotationPresent(securityClass)) {
        return true;
      }
    }

    return false;
  }

  private String getUsernameFromToken(String token) throws ParseException, JOSEException {

    SignedJWT signedJWT = SignedJWT.parse(token); (10)
    JWSVerifier verifier = new MACVerifier(SharedSecret.getSecret());

    if(signedJWT.verify(verifier)) {
      return signedJWT.getJWTClaimsSet().getSubject(); (11)
    } else {
      throw new JOSEException("Firm is not verified.");
    }
  }
}
1 To plug a filter you must annotate it with @Provider.
2 Filter is executed during authentication phase.
3 Because filter is executed in server-side, it must implements ContainerRequestFilter.
4 ResourceInfo is injected to get information about endpoint method that is going to be called.
5 Checks if method is secured or not by verifying the presence of Java EE Security annotation.
6 As described in JSON Web Token section, token must be set in x-access-token header attribute.
7 Gets username value from token.
8 With given username, the information about the user is returned from repository. Basically roles where user has access are returned.
9 A custom SecurityContext class to verify if user has the expected roles.
10 Parse the passed token.
11 If token is valid and is verified as a token generated by the system, we can get the subject field. Subject claim field is used to save username.
Nimbus JOSE + JWT

In this example we are using Nimbus JOSE + JWT library to create, parse, verify, …​ JWT tokens.

Nimbus JOSE + JWT is an open source Java library which implements the Javascript Object Signing and Encryption (JOSE) spec suite and the closely related JSON Web Token (JWT) spec.

A full description can be read at http://connect2id.com/products/nimbus-jose-jwt

And finally an authorization filter. This filter is responsible of ensuring that the user has the required rights to access to resource.

RolesAllowedFilter.java
@Provider
@Priority(Priorities.AUTHORIZATION) (1)
public class RolesAllowedFilter implements ContainerRequestFilter {

  private static final Response NOT_FOUND = Response.status(
      Response.Status.NOT_FOUND).entity("{\"message\": \"Resource Not Found\"}").build();

  @Context
  private ResourceInfo resourceInfo;

  @Override
  public void filter(ContainerRequestContext requestContext)
      throws IOException {
    Method resourceMethod = resourceInfo.getResourceMethod();

    // DenyAll on the method take precedence over RolesAllowed and PermitAll
    if (resourceMethod.isAnnotationPresent(DenyAll.class)) {
      requestContext.abortWith(NOT_FOUND);
      return;
    }

    // RolesAllowed on the method takes precedence over PermitAll
    RolesAllowed ra = resourceMethod.getAnnotation(RolesAllowed.class);
    if(assertRole(requestContext, ra)) { (2)
      return;
    }

    // PermitAll takes precedence over RolesAllowed on the class
    if (resourceMethod.isAnnotationPresent(PermitAll.class)) {
      // Do nothing.
      return;
    }

    if (resourceInfo.getResourceClass().isAnnotationPresent(DenyAll.class)) {
      requestContext.abortWith(NOT_FOUND);
    }

    // RolesAllowed on the class takes precedence over PermitAll
    ra = resourceInfo.getResourceClass().getAnnotation(RolesAllowed.class);
    if(assertRole(requestContext, ra)) {
      return;
    }
  }

  private boolean assertRole(ContainerRequestContext requestContext, RolesAllowed ra) {

    if (ra != null) {
      String[] roles = ra.value();
      for (String role : roles) {
        if (requestContext.getSecurityContext().isUserInRole(role)) { (3)
          return true;
        }
      }
      requestContext.abortWith(NOT_FOUND);
    }
    return false;
  }
}
1 Filter is executed during authentication phase.
2 Asserts if current user is in role list.
3 Uses SecurityContext provided in authentication phase, to check if current user is in endpoint role.

Having both classes in classpath, we can use Java EE security annotations to secure RESTful web services.

Let’s see how to implement a login service. Note that this service is not securized.

UserResource.java
@Path("login")
public class UserResource {

  @POST
  @Consumes(MediaType.APPLICATION_JSON)
  @Produces(MediaType.APPLICATION_JSON)
  public Response login(JsonObject jsonObject) throws JOSEException {

    JsonString username = (JsonString) jsonObject.get("username"); (1)
    JsonString password = (JsonString) jsonObject.get("password");

    if (authenticate(username.getString(), password.getString())) { (2)

      String token = createToken(username.getString(), "example.com"); (3)

      JsonObject responseDocument = Json.createObjectBuilder()
        .add("user", Json.createObjectBuilder().add("username", username).build())
        .add("token", token)
        .build(); (4)

      return Response.ok(responseDocument).build();

    }

    return Response.status(Status.NOT_FOUND).build();

  }

  private String createToken(String subject, String issuer) throws JOSEException {

    JWSSigner signer = new MACSigner(SharedSecret.getSecret());

    JWTClaimsSet claimsSet = new JWTClaimsSet();
    claimsSet.setSubject(subject); (5)
    claimsSet.setIssueTime(new Date());
    claimsSet.setIssuer(issuer);

    SignedJWT signedJWT = new SignedJWT(new JWSHeader(JWSAlgorithm.HS256), claimsSet);
    signedJWT.sign(signer);

    return signedJWT.serialize(); (6)

  }
}
1 username and password are sent as document body.
2 Authenticates given user and password.
3 Creates the token by using example.com domain as issuer.
4 Creates the response body document.
5 Sets username as subject of the token.
6 Returns the token as string.

And a secured resource:

BookResource.java
import javax.annotation.security.RolesAllowed;

@Path("/book")
public class BookResource {

  @GET
  @Produces(MediaType.APPLICATION_JSON)
  @RolesAllowed("admin") (1)
  public String book() {
    //...
  }

  @GET
  @Path("/article")
  @Produces(MediaType.APPLICATION_JSON)
  @RolesAllowed("superadmin")
  public String article() {
    //...
  }

}
1 Only users with role admin can access to this resource.

Bibliography

88x31

1. You need to provide the binding library or rely on the one provided by JAX-RS provider.