API testing is a type of software testing that involves testing application programming interfaces (APIs) directly and as part of integration testing to determine if they meet expectations for functionality, reliability, performance, and security. Since APIs lack a GUI, API testing is performed at the message layer.
There are many different tools out there that can assist you in writing these automated tests for APIs. This article will focus on getting our hands dirty with one of the most popular open-source Java library
To have a clear picture on how to implement powerful and maintainable API tests suite, for the purpose of this article we will setup a simple CRUD API application and start writing tests to handle different REST operations and response verifications.
We had a previous article published on
HTTP Method | API Name | Path | Response Status Code |
---|---|---|---|
POST | Create Catalogue Item | / | 201 (Created) |
GET | Get Catalogue Items | / | 200 (Ok) |
GET | Get Catalogue Item | /{sku} | 200 (Ok) |
PUT | Update Catalogue Item | /{sku} | 200 (Ok) |
DELETE | Delete Catalogue Item | /{sku} | 204 (No Content) |
POST | Upload Catalog Item Picture | /{sku}/image | 201 (Created) |
We will be implementing our API tests for these operations that are exposed via the application to manage items for a Catalogue Management System.
Java is one of the world’s most widely used computer language. Java is a simple, general-purpose, object-oriented, interpreted, robust, secure, architecture-neutral, portable, high-performance, multithreaded computer language. It is intended to let application developers write once, run anywhere (WORA), meaning that code that runs on one platform does not need to be recompiled to run on another.
Java technology is both a programming language and a platform. It is a high level, robust, secured and object-oriented programming language. And any hardware or software environment in which a program runs, is known as a platform. Since Java has its own runtime environment (JRE) and API, it is called platform.
Maven is a project management and comprehension tool that provides developers a complete build lifecycle framework. Development team can automate the project’s build infrastructure in almost no time as Maven uses a standard directory layout and a default build lifecycle.
To summarize, Maven simplifies and standardizes the project build process. It handles compilation, distribution, documentation, team collaboration and other tasks seamlessly. Maven increases reusability and takes care of most of the build related tasks
Junit 5 is a powerful and popular Java testing framework and is the next generation of JUnit. It is composed of many different modules and can be summarized as below:
JUnit 5 = JUnit Platform + JUnit Jupiter + JUnit Vintage
JUnit Platform serves as a foundation for
JUnit Jupiter is the combination of the new
TestEngine
for running Jupiter based tests on the platform.JUnit Vintage provides a TestEngine for running JUnit 3 and JUnit 4 based tests on the platform.
JUnit 5 requires Java 8 (or higher) at runtime.
REST-assured is highly opted when implementing Java based Automated API Tests suite because of its powerful features such as:
Making an API call needs lot of boilerplate code for setting up HTTP Connection, sending request and extracting response status & payload for test validation. REST-assured eliminates us to write this repeated code.
Easy to integrate with existing Java Unit Test libraries such as
And lastly, making the tests human readable by implementing the tests with
Hamcrest is a framework for writing matcher
objects allowing match
rules to be defined declaratively. Hamcrest is commonly used with junit and TestNG making assertions
. Single assertThat
statement with appropriate matchers
will replace many of JUnit and TestNG assert
methods.
Hamcrest comes with a library of useful matchers. Here are some of the most important ones.
Matcher | Description |
---|---|
allOf | matches if all matchers match (short circuits) |
anyOf | matches if any matchers match (short circuits) |
not | matches if the wrapped matcher doesn’t match and vice |
equalTo | test object equality using the equals method |
is | decorator for equalTo to improve readability |
hasToString | test Object.toString |
instanceOf, isCompatibleType | test type |
notNullValue, nullValue | test for null |
sameInstance | test object identity |
hasEntry, hasKey, hasValue | test a map contains an entry, key or value |
hasItem, hasItems | test a collection contains elements |
hasItemInArray | test an array contains an element |
closeTo | test floating point values are close to a given value |
greaterThan, greaterThanOrEqualTo, lessThan, lessThanOrEqualTo | test ordering |
equalToIgnoringCase | test string equality ignoring case |
equalToIgnoringWhiteSpace | test string equality ignoring differences in runs of whitespace |
containsString, endsWith, startsWith | test string matching |
For the complete list of matchers, follow
Below is usage of Hamcrest for few matches:
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.*;
// Using instanceOf and shortcut for instanceOf
assertThat(Long.valueOf(1), instanceOf(Integer.class));
assertThat(Long.valueOf(1), isA(Integer.class));
// Verifying Lists for size, contains, anyOrder, greaterThan
List<Integer> list = Arrays.asList(4, 2, 3);
assertThat(list, hasSize(3));
assertThat(list, contains(4, 2, 3)); // Verify if order is in exact order
assertThat(list, containsInAnyOrder(3, 4, 2)); // Verify in any order
assertThat(list, everyItem(greaterThan(1))); // verify if every element is greater than 1
// Using with REST-assured
given()
.get("/")
.then()
.assertThat().spec(prepareResponseSpec(200))
.and()
.assertThat().body("data", is(not(empty())));
We will be heavily relying on
Below is sample Lombok code for a POJO class:
@Data
@AllArgsConstructor
@RequiredArgsConstructor(staticName = "of")
public class CatalogueItem {
private Long id;
private String sku;
private String name;
}
@Data
is a convenient shortcut annotation that bundles the features of @ToString
, @EqualsAndHashCode
, @Getter
/ @Setter
and @RequiredArgsConstructor
all together.@AllArgsConstructor
generates a constructor with 1 parameter for each field in your class. Fields marked with @NonNull result in null checks on those parameters.@RequiredArgsConstructor
generates a constructor with 1 parameter for each field that requires special handling.IDE Plugins
Install Lombok plugin in
Oracle has
An alternative is to use
Why Red Hat’s Build of OpenJDK?
At bare minimum we need to have JDK installed, maven configured and setup IDE of your choice to start implementing tests.
OpenJDK 8 for Windows can be installed manually using a ZIP bundle or through a graphical user interface using an MSI-based installer.
Download the
The %JAVA_HOME%
environment variable must also be set to use some developer tools. Set the %JAVA_HOME%
environment variable as follows:
Open Command Prompt as an administrator.
Set the value of the environment variable to your OpenJDK 11 for Windows installation path:
~:\> setx /m JAVA_HOME "C:\Progra~1\RedHat\java-11-openjdk-11.0.7.10-1"
Note: If the path contains spaces, use the shortened path name.
Follow the steps provided
~:\> mvn -v
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: E:\binaries\apache-maven-3.6.3\bin\..
Java version: 11.0.6, vendor: Oracle Corporation, runtime: C:\Program Files\RedHat\java-11-openjdk-11.0.7.10-1\jre
Default locale: en_AU, platform encoding: Cp1252
OS name: "windows 10", version: "10.0", arch: "amd64", family: "windows"
Execute the below command to create base project which can be used to implement our tests. Provide details when prompted.
~:\> mvn archetype:generate -DarchetypeGroupId=org.toomuch2learn -DarchetypeArtifactId=rest-assured-crud-api-tests -DarchetypeVersion=1.4
Ensure development environment is configured to start proceeding with implementing API Tests for the Catalogue Management Service CRUD Application.
Clone or download
~:\> git clone https://github.com/2much2learn/article-feb20-crud-rest-api-using-spring-boot-spring-data-jpa
~:\> cd article-feb20-crud-rest-api-using-spring-boot-spring-data-jpa
Run the below maven
command to start the application.
~:\> mvn clean spring-boot:run
By default port 8080 will be used. If need to change the port which should be used, pass additional argument to maven command.
~:\> mvn clean spring-boot:run -Drun.arguments="--server.port=9000"
Or, we can package the application to Jar and start the jar by running the below command
~:\> mvn clean package
~:\> java -jar target/catalogue-crud-0.0.1-SNAPSHOT.jar
~:\> java -jar -Dserver.port=9000 target/catalogue-crud-0.0.1-SNAPSHOT.jar
With all the supporting dependencies, maven build file looks something like below. Replace pom.xml generated in the base project with this.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.toomuch2learn</groupId>
<artifactId>rest-assured-crud-api-tests</artifactId>
<version>1.0-SNAPSHOT</version>
<name>rest-assured-crud-api-tests</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
<junit.jupiter.version>5.6.0</junit.jupiter.version>
</properties>
<dependencies>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.12</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.10.2</version>
</dependency>
<dependency>
<groupId>org.hamcrest</groupId>
<artifactId>hamcrest</artifactId>
<version>2.2</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter</artifactId>
<version>${junit.jupiter.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.rest-assured</groupId>
<artifactId>rest-assured</artifactId>
<version>3.0.0</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
</plugin>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version>
</plugin>
</plugins>
</build>
<repositories>
<repository>
<id>spring-milestones</id>
<name>Spring Milestones</name>
<url>https://repo.spring.io/milestone</url>
</repository>
<repository>
<id>spring-snapshots</id>
<name>Spring Snapshots</name>
<url>https://repo.spring.io/snapshot</url>
<snapshots>
<enabled>true</enabled>
</snapshots>
</repository>
</repositories>
</project>
REST-assured provides a fluent API in Given/When/Then syntax based on
given()
.pathParam("sku", catalogueItem.getSku())
.when()
.get("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(200))
.and()
.assertThat().body("name", equalTo(catalogueItem.getName()))
.and()
.assertThat().body("category", equalTo(catalogueItem.getCategory()));
Create the package structure and classes as below which are the building blocks for this test suite.
With
And hence, we included the same model classes that are defined in crud-springboot-data-jpa
project. This will ease creating the request body and handling the response content.
Test class RestAssuredCatalogueCRUDTest
is defined to handle all tests against the RESTful APIs. It imports classes from JUnit5, REST-assured and Hamcrest to setup, access and validate APIs.
Below are the tests that we are planning to include as part of this Test class:
As each of this is an individual test, the context of complete test execution should be wrapped within its own method. To test Update Catalogue Item, we need to create, update and then get the catalogue item to verify if item is actually updated or not.
This can lead to lot of code duplication as most of the tests would have to include creating the catalogue item before testing its context of execution. For code reusability, we need to create functional methods which can be called from the test methods and should not impact other tests.
Below is such reusable method which handles creating catalogue item request and returning the response.
private Response postCreateCatalogueItem(CatalogueItem catalogueItem) throws Exception {
RequestSpecification request
= given()
.contentType("application/json")
.body(catalogueItem);
return request.post("/");
}
We should consider creating new instance of CatalogueItem created with unique values, else there will be unique constraint exceptions occurring in the application when persisting the catalogue items to the database. Below reusable methods are created to create Catalogue Item with fields assigned with distinct values.
// Create Catalogue Item
CatalogueItem catalogueItem = prepareCatalogueItem(prepareRandomSKUNumber());
final Random random = new Random();
private String prepareRandomSKUNumber() {
return "SKUNUMBER-"+
random.ints(1000, 9999)
.findFirst()
.getAsInt();
}
private CatalogueItem prepareCatalogueItem(String skuNumber) {
CatalogueItem item
= CatalogueItem.of(
skuNumber,
"Catalog Item -"+skuNumber,
"Catalog Desc - "+skuNumber,
Category.BOOKS.getValue(),
10.00,
10,
new Date()
);
return item;
}
As observed, we created prepareRandomSKUNumber
method to generate unique SKU number which will be passed to prepareCatalogueItem
to create instance of Catalogue Item with random SKU number. This will ensure unique constraint fields are kept unique when executing tests.
And finally, we will be creating one more reusable method to create instance of ResponseSpecification
based on the expected response HTTP Status code. This method will be used in all test classes to verify if the response received is with the expected response HTTP Status code.
private ResponseSpecification prepareResponseSpec(int responseStatus) {
return new ResponseSpecBuilder()
.expectStatusCode(responseStatus)
.build();
}
With the common reusable code separated, implementing test methods will be easy by following the Given\When\Then syntax.
As all the tests are performed on the single API endpoints, they all share the same API Base URI. REST-assured provides a convenient way to configure this base uri to be used by all the tests.
static {
RestAssured.baseURI = "http://localhost:8080/api/v1";
}
Access /actuator/health
endpoint and verify if response is successful with HTTP status code 200
and response has got body containing status
field as UP
.
@Test
@DisplayName("Test if Application is up by accessing health endpoint")
public void test_applicationIsUp() {
try {
given()
.get("http://localhost:8080/actuator/health")
.then()
.assertThat().spec(prepareResponseSpec(200))
.and()
.assertThat().body("status", equalTo("UP"));
}
catch(Exception e) {
fail("Error occurred while tesing application health check", e);
}
}
Create Instance of Catalogue Item with random SKU Number and pass this to postCreateCatalogueItem
to post request to create new Catalogue Item. Verify if the request is handled properly by verifying response HTTP status code is 201
and response body containing id
field with value grater than 0
.
@Test
@DisplayName("Test Create Catalogue Item")
public void test_createCatalogueItem() {
try {
postCreateCatalogueItem(prepareCatalogueItem(prepareRandomSKUNumber()))
.then()
.assertThat().spec(prepareResponseSpec(201))
.and()
.assertThat().body("id", greaterThan(0));
}
catch(Exception e) {
fail("Error occurred while testing catalogue item create endpoint", e);
}
}
For the context of this test execution, send two sequential requests for creating catalogue items. Upon accessing Get Catalogue Items, response status should be 200
and the response body should contain list of catalogue items assigned to data
field is
not
empty
.
@Test
@DisplayName("Test Get Catalogue Items")
public void test_getCatalogueItems() {
try {
postCreateCatalogueItem(prepareCatalogueItem(prepareRandomSKUNumber()));
postCreateCatalogueItem(prepareCatalogueItem(prepareRandomSKUNumber()));
given()
.get("/")
.then()
.assertThat().spec(prepareResponseSpec(200))
.and()
.assertThat().body("data", is(not(empty())));
}
catch (Exception e) {
fail("Error occurred while testing fetch catalogue items", e);
}
}
Post create catalogue item request and verify if we are receiving the same catalogue item when get request sent with the same sku
number in pathParam
. Validate the response status is 200
and response body name
and category
fields match with those that are sent for create request.
@Test
@DisplayName("Test Get Catalogue Item")
public void test_getCatalogueItem() {
try {
// Create Catalogue Item
CatalogueItem catalogueItem = prepareCatalogueItem(prepareRandomSKUNumber());
postCreateCatalogueItem(catalogueItem);
// Get Catalogue item with the sku of the catalogue item that is created and compare the response fields
given()
.pathParam("sku", catalogueItem.getSku())
.when()
.get("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(200))
.and()
.assertThat().body("name", equalTo(catalogueItem.getName()))
.and()
.assertThat().body("category", equalTo(catalogueItem.getCategory()));
}
catch(Exception e) {
fail("Error occurred while testing fetch catalogue item", e);
}
}
Multiple operations are handled for update request. Create instance of CatalogueItem and send request to create it. Update few fields in the CatalogueItem and pass it to update it by its sku
number. Now, access get request passing sku
and validating if we are receiving the response body with updated field values.
@Test
@DisplayName("Test Update Catalogue Item")
public void test_updateCatalogueItem() {
try {
// Create Catalogue Item
CatalogueItem catalogueItem = prepareCatalogueItem(prepareRandomSKUNumber());
postCreateCatalogueItem(catalogueItem);
// Update catalogue item
catalogueItem.setName("Updated-"+catalogueItem.getName());
catalogueItem.setDescription("Updated-"+catalogueItem.getDescription());
given()
.contentType("application/json")
.body(catalogueItem)
.pathParam("sku", catalogueItem.getSku())
.when()
.put("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(200));
// Get updated catalogue item with the sku of the catalogue item that is created and compare the response fields
given()
.pathParam("sku", catalogueItem.getSku())
.when()
.get("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(200))
.and()
.assertThat().body("name", equalTo(catalogueItem.getName()))
.and()
.assertThat().body("category", equalTo(catalogueItem.getCategory()));
}
catch(Exception e) {
fail("Error occurred while testing catalogue item update", e);
}
}
Similar to Update request, to test Delete request follow the sequence to Create
-> Delete
-> Get
and verify if Get request is returning back with HTTP status code 404
which is Resource Not Found
.
@Test
@DisplayName("Test Delete Catalogue Item")
public void test_deleteCatalogueItem() {
try {
// Create Catalogue Item
CatalogueItem catalogueItem = prepareCatalogueItem(prepareRandomSKUNumber());
postCreateCatalogueItem(catalogueItem);
// Delete Catalogue Item
given()
.pathParam("sku", catalogueItem.getSku())
.when()
.delete("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(204));
// Trying to get the deleted catalogue item should throw 400
given()
.pathParam("sku", catalogueItem.getSku())
.when()
.get("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(404));
}
catch(Exception e) {
fail("Error occurred while testing catalogue item update", e);
}
}
Accessing Get request with random SKU which is not available in the application should throw HTTP status 404
.
@Test
@DisplayName("Test Resource not found")
public void test_resourceNotFound() {
try {
given()
.pathParam("sku", prepareRandomSKUNumber())
.get("/{sku}")
.then()
.assertThat().spec(prepareResponseSpec(404));
}
catch(Exception e) {
fail("Error occurred while testing resource not found", e);
}
}
Accessing API with some invalid uri paths should throw HTTP status 404
.
@Test
@DisplayName("Test Handler not found")
public void test_handlerNotFound() {
try {
given()
.get("/invalid/handler")
.then()
.assertThat().spec(prepareResponseSpec(404));
}
catch(Exception e) {
fail("Error occurred while testing handler not found", e);
}
}
Accessing Create
or Update
requests with request body containing invalid field values should throw response with HTTP status code 400
and response body containing list of errors
which is not empty and the error
matching the expected description.
@Test
@DisplayName("Test validation error")
public void test_validationErrors() {
try {
CatalogueItem catalogueItem = prepareCatalogueItem(prepareRandomSKUNumber());
catalogueItem.setCategory("INVALID");
Response response
= postCreateCatalogueItem(catalogueItem)
.then()
.assertThat().spec(prepareResponseSpec(400))
.and()
.extract().response();
List<Error> errors = Arrays.asList(response.getBody().jsonPath().getObject("errors", Error[].class));
assertTrue(errors != null && errors.size() > 0);
assertTrue(errors.get(0).getDescription().equalsIgnoreCase("Invalid category provided"));
}
catch(Exception e) {
fail("Error occurred while testing validation errors", e);
}
}
To test Invalid Request, we need to set invalid data to fields which is not acceptable. We cannot create instance of CatalogueItem with unacceptable values apart from what is expected, else compilation issues will occur.
To test such scenario, instead of using model class we are creating instance of Jackson’s JsonNode
and constructing the request body structure with one of the field values to invalid datatypes. In this case, it is price
field which is double
but setting value to string
.
This should fail both Create
and Update
requests with response HTTP status code as 400
and response body containing list of errors and containing expected message Invalid Request
.
@Test
@DisplayName("Test Invalid Request")
public void test_invalidRequest() {
try {
// Create Catalogue Item via JsonObject
ObjectMapper mapper = new ObjectMapper();
JsonNode rootNode = mapper.createObjectNode();
((ObjectNode) rootNode).put("name", "INVALID");
((ObjectNode) rootNode).put("sku", prepareRandomSKUNumber());
((ObjectNode) rootNode).put("price", "INVALID");
String catalogueItem = mapper.writerWithDefaultPrettyPrinter().writeValueAsString(rootNode);
Response response =
given()
.contentType("application/json")
.body(catalogueItem)
.post("/")
.then()
.assertThat().spec(prepareResponseSpec(400))
.and()
.extract().response();
List<Error> errors = Arrays.asList(response.getBody().jsonPath().getObject("errors", Error[].class));
assertTrue(errors != null && errors.size() > 0);
assertTrue(errors.get(0).getMessage().equalsIgnoreCase("Invalid Request"));
}
catch(Exception e) {
fail("Error occurred while testing invalid request", e);
}
}
Before executing the tests, Ensure CRUD Application is started and the base uri is using the port on which the Application is running.
To run the tests from IntelliJ Idea, run the test class by right-click and choosing Run RestAssuredCatalogueCRUDTest
.
This should result all the tests to be successfully as below:
If CRUD API application is started successfully and no changes done to test project, then there would not be a chance for the tests to fail. But if there is any test method failure, investigate the exception and fix it accordingly.
Run the below command to execute the tests by maven
~:\> mvn clean test
If tests are successful, something like this should be displayed.
[INFO] Compiling 4 source files to E:\Projects\2much2learn\2much2learn_examples\testing\rest-assured-crud-api-tests\target\test-classes
[INFO]
[INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ rest-assured-crud-api-tests ---
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.toomuch2learn.crud.catalogue.RestAssuredCatalogueCRUDTest
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.373 s - in org.toomuch2learn.crud.catalogue.RestAssuredCatalogueCRUDTest
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 10, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.681 s
[INFO] Finished at: 2020-02-22T21:30:04+11:00
[INFO] ------------------------------------------------------------------------
Apart from what we went through in this article, REST-assured offers many more useful features that can accommodate creating tests suite for different usecases. REST-assured