Microservices Basics Training – 3. Microservices Rest

Docs: 3.1 Getting Started

Mon, 01 Jan 0001 00:00:00 +0000

Source Code

The source-code used for this lab is in the folder /code/rest

Online Repository Link: https://github.com/acend/microservices-lab/tree/main/code/rest .

Task 3.1.1 - Getting your workspace ready

If you still have the previous lab running: stop it with docker-compose down.
Open your IDE and switch to the referenced source folder.

Docs: 3.2 Docker Environment

Mon, 01 Jan 0001 00:00:00 +0000

Environment Setup

With completing this section the environment is configured. However, in the next few steps we will build the order microservice and its container. After this section you will not be able to start the environment as this container is still missing. You will be able to start it later.

Changes to the docker-compose file

To split the monolith in two microservices we need to change our docker-compose.yaml file.

Microservice container specification

First we deleted the application and application-db specification which belonged to our monolith. Then we created a specification for the two microservices order and stock.

As we are building the order service by ourselves we reference a local container image. The stock service uses a pre-built image from the quay.io/acend registry.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28


 order:
 image: microservices-lab-rest-order:latest
 hostname: order
 container_name: rest-order
 ports:
 - 8080:8080
 networks:
 - rest
 depends_on:
 - order-db
 environment:
 - QUARKUS_HTTP_PORT=8080
 - QUARKUS_DATASOURCE_JDBC_URL=jdbc:tracing:postgresql://order-db:5432/admin
 - APPLICATION_ARTICLESTOCK_API_URL=http://stock:8080/

 stock:
 image: quay.io/acend/microservices-lab-rest-stock:latest
 hostname: stock
 container_name: rest-stock
 ports:
 - 8081:8080
 networks:
 - rest
 depends_on:
 - stock-db
 environment:
 - QUARKUS_HTTP_PORT=8080
 - QUARKUS_DATASOURCE_JDBC_URL=jdbc:tracing:postgresql://stock-db:5432/admin

As both microservices need their own postgres database we added two databases to our docker-compose.yaml.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27


 order-db:
 image: docker.io/postgres:13
 hostname: order-db
 container_name: rest-order-db
 volumes:
 - order-db-data:/var/lib/postgresql/data/
 networks:
 - rest
 environment:
 - POSTGRES_USER=admin
 - POSTGRES_PASSWORD=1234
 ports:
 - 5432:5432

 stock-db:
 image: docker.io/postgres:13
 hostname: stock-db
 container_name: rest-stock-db
 volumes:
 - stock-db-data:/var/lib/postgresql/data/
 networks:
 - rest
 environment:
 - POSTGRES_USER=admin
 - POSTGRES_PASSWORD=1234
 ports:
 - 5433:5432

Task 3.2.1 - Inspecting the docker environment

Have a look at the /code/rest/docker folder and get familiar with the docker environment specified in docker-compose.yaml.

Docs: 3.3 Building a Microservice

Mon, 01 Jan 0001 00:00:00 +0000

Stock Microservice

The order and stock microservice have a similar setup. Therefore, we will only work on the order microservice.

Order Microservice

We have already finished the specification of our environment in the docker compose file. Now we will write our order microservice.

As an overview this is the final structure of the order microservice at the end of the tasks in this section. You will be guided the implementation of these in the next steps.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


.
├── pom.xml
└── src
 └── main
    ├── docker
    ├── java
    │   └── ch
    │   └── puzzle
    │   └── mm
    │   └── rest
    │   ├── order
    │   │   ├── boundary
    │   │   │   └── ShopOrderResource.java # Order RESTful API
    │   │   ├── control
    │   │   │   └── ShopOrderService.java # Business code
    │   │   └── entity
    │   │   ├── ArticleOrderDTO.java # Entities and DTOs...
    │   │   ├── ArticleOrder.java
    │   │   ├── ShopOrderDTO.java
    │   │   ├── ShopOrder.java
    │   │   └── ShopOrderStatus.java
    │   └── stock
    │   └── boundary
    │   └── ArticleStockService.java # Stock REST-Client
    └── resources
    ├── application.properties # Application Properties
    ├── db
    │   └── changeLog.xml # Database migration
    └── META-INF
    └── resources
    └── index.html # Static webpage

Datasource and database schema

First we are going to setup the database connection and tables.

Task 3.3.1 - Setup database dependencies

Limited lab duration

This step is already implemented. For a better understanding we encourage you to have a quick read but you may also skip it.

Ensure the following dependencies are specified in the pom.xml.

GroupId	ArtifactId	Description	Detailed information
`io.quarkus`	`quarkus-hibernate-orm-panache`	Hibernate ORM with Panache support	Simplified Hibernate ORM with Panache
`io.quarkus`	`quarkus-liquibase`	Liquibase¹ Quarkus integration for database schema change management	Using Liquibase
`io.quarkus`	`quarkus-jdbc-postgresql`	JDBC PostgreSQL support for Quarkus	Datasources

Dependencies Task Hint

Ensure the following dependencies are in your pom.xml. Dependency quarkus-hibernate-orm-panache is already added.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


<dependencies>
 ...
 <!-- database dependencies -->
 <dependency>
 <groupId>io.quarkus</groupId>
 <artifactId>quarkus-hibernate-orm-panache</artifactId>
 </dependency>
 <dependency>
 <groupId>io.quarkus</groupId>
 <artifactId>quarkus-liquibase</artifactId>
 </dependency>
 <dependency>
 <groupId>io.quarkus</groupId>
 <artifactId>quarkus-jdbc-postgresql</artifactId>
 </dependency>
 ...
</dependencies>

After adding these dependencies, we are ready to configure quarkus to use the database. Add the following configuration to the application.properties file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


# datasource
quarkus.datasource.db-kind = postgresql
quarkus.datasource.username = admin
quarkus.datasource.password = 1234
quarkus.datasource.jdbc.url = jdbc:tracing:postgresql://localhost:5432/admin
quarkus.datasource.jdbc.driver=io.opentracing.contrib.jdbc.TracingDriver
quarkus.hibernate-orm.dialect=org.hibernate.dialect.PostgreSQLDialect

# drop and create the database at startup (use `update` to only update the schema)
quarkus.hibernate-orm.database.generation = none

# liquibase properties
quarkus.liquibase.migrate-at-start=true
quarkus.liquibase.clean-at-start=true

This will defines our data source connection and specify that Liquibase should clean and migrate the database schema at start up.

Task 3.3.2 - Setup database schema

Limited lab duration

This step is already implemented. For a better understanding we encourage you to have a quick read but you may also skip it.

Let’s build the database schema with liquibase.

Create folder src/main/resources/db
Create file src/main/resources/db/changeLog.xml

You have to use the liquibase XML syntax to create the tables and sequence. You may find some details here:

For details about the supported data types in postgres, please check the PostgreSQL Data Types page.

You can use the following template:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31


<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
 xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext
 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd
 http://www.liquibase.org/xml/ns/dbchangelog
 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">

 <changeSet author="main" id="1">
 <!-- simple table -->
 <createTable tableName="TABLENAME">
 <column name="PKNAME" type="PKTYPE">
 <constraints primaryKey="true" nullable="false"/>
 </column>
 <column name="NAME" type="TYPE"/>
 </createTable>

 <!-- relation table -->
 <createTable tableName="TABLE1_TABLE2">
 <column name="TABLE1_id" type="TABLE1_PKTYPE">
 <constraints nullable="false" foreignKeyName="FK_TABLE1_NAME" referencedColumnNames="id"/>
 </column>
 <column name="TABLE2_id" type="TABLE2_PKTYPE">
 <constraints nullable="false" foreignKeyName="FK_TABLE2_NAME" referencedColumnNames="id"/>
 </column>
 </createTable>

 <createSequence sequenceName="SEQUENCE_NAME" startValue="STARTVALUE"/>
 </changeSet>
</databaseChangeLog>

Naming

Please be precise with the naming of tables and fields. If you name things differently you may run into trouble in later steps.

Create the shoporder table in changeLog.xml according to the details below. This table will store our orders.

1
2
3
4
5
6
7
8
9


admin=# \d shoporder;
 Table "shoporder"
 Column | Type | Collation | Nullable | Default
--------+------------------------+-----------+----------+---------
 id | bigint | | not null |
 status | character varying(255) | | |

Indexes:
 "shoporder_pkey" PRIMARY KEY, btree (id)

Create the articleorder table in changeLog.xml according to the details below. This table reflects the effective articles which have been ordered.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


admin=# \d articleorder;
 Table "articleorder"
 Column | Type | Collation | Nullable | Default
-----------+---------+-----------+----------+---------
 id | bigint | | not null |
 articleid | bigint | | |
 amount | numeric | | |

Indexes:
 "articleorder_pkey" PRIMARY KEY, btree (id)

Create the mapping table in changeLog.xml according to the details below. This table is used to build the mapping from shoporder to the articleorder. Define any needed foreignKeyName yourself.

1
2
3
4
5
6


admin=# \d shoporder_articleorder;
 Table "shoporder_articleorder"
 Column | Type | Collation | Nullable | Default
------------------+--------+-----------+----------+---------
 shoporder_id | bigint | | not null |
 articleorders_id | bigint | | not null |

Finally, create a hibernate sequence in changeLog.xml according to the details below.

1
2
3
4
5


admin=# \d hibernate_sequence;
 Sequence "hibernate_sequence"
 Type | Start | Minimum | Maximum | Increment | Cycles? | Cache
--------+--------+---------+---------------------+-----------+---------+-------
 bigint | 100000 | 1 | 9223372036854775807 | 1 | no | 1

Task Hint

Your changeLog.xml should look like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37


<?xml version="1.1" encoding="UTF-8" standalone="no"?>
<databaseChangeLog xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
 xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog-ext
 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd
 http://www.liquibase.org/xml/ns/dbchangelog
 http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.5.xsd">

 <changeSet author="main" id="1">
 <createTable tableName="shoporder">
 <column name="id" type="bigint">
 <constraints primaryKey="true" nullable="false"/>
 </column>
 <column name="status" type="varchar(255)"/>
 </createTable>

 <createTable tableName="articleorder">
 <column name="id" type="bigint">
 <constraints primaryKey="true" nullable="false"/>
 </column>
 <column name="articleid" type="bigint"/>
 <column name="amount" type="number"/>
 </createTable>

 <createTable tableName="shoporder_articleorder">
 <column name="shoporder_id" type="bigint">
 <constraints nullable="false" foreignKeyName="shoporder_article_shoporder_fk" referencedColumnNames="id"/>
 </column>
 <column name="articleorders_id" type="bigint">
 <constraints nullable="false" foreignKeyName="shoporder_article_articleorder_fk" referencedColumnNames="id"/>
 </column>
 </createTable>

 <createSequence sequenceName="hibernate_sequence" startValue="100000"/>
 </changeSet>
</databaseChangeLog>

Entities and Data Transfer Objects (DTOs)

The next step is to define our Domain Entities and DTOs. For our order microservice we will need the following entities:

ShopOrder: The entity representing an order
ArticleOrder: The entity representing an ordered article and its order count
ShopOrderStatus: An enum representing the status of the order

We will trigger a new order using a RESTful API call to our order microservice. For this we will use DTOs instead of entities. Define the following DTOs:

ShopOrderDTO: A DTO to specify which articles have been ordered with this order.
ArticleOrderDTO: A DTO to specify the article id and the amount of items ordered.

Task 3.3.3 - Implement entities

Limited lab duration

This step is already implemented. For a better understanding we encourage you to have a quick read but you may also skip it.

Entity overview of the order microservice.

PanacheEntity

We are using the Quarkus Hibernate ORM with Panache extension. The classical approach with the EntityManager would work as well and might be required for complex use cases. However, in our simple implementation we will make use of Panache. For more information about Panache have a look at: Simplified Hibernate ORM with Panache

We will create our entities in src/main/java/ch/puzzle/mm/rest/order/entity.

Implement enum ShopOrderStatus
Implement entity ArticleOrder
- Implement the default Constructor and a Constructor with both fields as arguments.
Implement entity ShopOrder
- Annotate articleOrders with @OneToMany(cascade = CascadeType.ALL)
- Annotate status with @Enumerated(EnumType.STRING)
Do not forget to annotate the entities with @Entity and extending from PanacheEntity which provides us the id.

ShopOrderStatus enum Task Hint

1
2
3
4
5


public enum ShopOrderStatus {
 NEW,
 COMPLETED,
 INCOMPLETE,
}

ArticleOrder Entity Task Hint

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30


@Entity
public class ArticleOrder extends PanacheEntity {

 Long articleId;
 int amount;

 public ArticleOrder() {
 }

 public ArticleOrder(Long articleId, int amount) {
 this.articleId = articleId;
 this.amount = amount;
 }

 public Long getArticleId() {
 return articleId;
 }

 public void setArticleId(Long articleId) {
 this.articleId = articleId;
 }

 public int getAmount() {
 return amount;
 }

 public void setAmount(int amount) {
 this.amount = amount;
 }
}

ShopOrder Entity Task Hint

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26


@Entity
public class ShopOrder extends PanacheEntity {
 @OneToMany(cascade = CascadeType.ALL)
 private List<ArticleOrder> articleOrders;

 @Enumerated(EnumType.STRING)
 private ShopOrderStatus status;

 public ShopOrder() { }

 public List<ArticleOrder> getArticleOrders() {
 return articleOrders;
 }

 public void setArticleOrders(List<ArticleOrder> articles) {
 this.articleOrders = articles;
 }

 public ShopOrderStatus getStatus() {
 return status;
 }

 public void setStatus(ShopOrderStatus status) {
 this.status = status;
 }
}

Task 3.3.4 - Implement DTOs

Limited lab duration

This step is already implemented. For a better understanding we encourage you to have a quick read but you may also skip it.

The following JSON corresponds to the POST Request body which is received by the order microservice. It reflects both DTOs ShopOrderDTO and ArticleOrderDTO.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12


{
 "articleOrders" : [
 {
 "articleId" : 1,
 "amount" : 1
 },
 {
 "articleId" : 2,
 "amount" : 3
 }
 ]
}

Implement both DTOs according to the given JSON. Put them in the same folder as the entities.

Implement ArticleOrderDTO
- set fields public, getters and setters are not needed
- articleId: use Long as type
- amount: use int as type
Implement ShopOrderDTO
- set field public, getters and setters are not needed
- articleOrders: use List as type

ArticleOrderDTO Task Hint

1
2
3
4


public class ArticleOrderDTO {
 public Long articleId;
 public int amount;
}

ShopOrderDTO Task Hint

1
2
3


public class ShopOrderDTO {
 public List<ArticleOrderDTO> articleOrders;
}

Building a Business Service for processing new orders

Processing of a new order will be handled by to the ShopOrderService.

Task 3.3.5 - Create service for order creation

Create the following ShopOrderService class in src/main/java/ch/puzzle/mm/rest/order/control

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


@ApplicationScoped
public class ShopOrderService {

 @Transactional
 public ShopOrder createOrder(ShopOrderDTO dto) {
 ShopOrder shopOrder = new ShopOrder();
 shopOrder.setStatus(ShopOrderStatus.NEW);

 // create order articles
 List<ArticleOrder> articleOrders = dto.articleOrders.stream()
 .map(s -> new ArticleOrder(s.articleId, s.amount))
 .collect(Collectors.toList());

 shopOrder.setArticleOrders(articleOrders);
 shopOrder.persist();

 return shopOrder;
 }
}

Building a Rest Client for the Stock Microservice

For calling our stock microservice we have to define a RESTful client. For a straightforward implementation we will use the MicroProfile:

MicroProfile Rest-Client²: Provides us a very simple way of creating rest clients by simply writing an interface using proper JAX-RS and MicroProfile annotations

Task 3.3.6 - Rest Client

Limited lab duration

This step is already implemented. For a better understanding we encourage you to have a quick read but you may also skip it.

Ensure the following dependencies are specified in the pom.xml.

GroupId	ArtifactId	Description	Detailed information
`io.quarkus`	`quarkus-rest-client`	Implementation of MicroProfile REST Client	Using the Rest Client

Dependencies Task Hint

The following dependencies have to be added.

1
2
3
4
5
6
7
8


<dependencies>
 ...
 <dependency>
 <groupId>io.quarkus</groupId>
 <artifactId>quarkus-rest-client</artifactId>
 </dependency>
 ...
</dependencies>

Task 3.3.7 - Define the Rest Client

Create the rest client by implementing a RestClient-Interface. Since this is a boundary service the interface goes in src/main/java/ch/puzzle/mm/rest/stock/boundary.

Create the interface ArticleStockService
Annotate the interface with @RegisterRestClient(configKey = "article-stock-api")
Annotate the interface with @Path("/article-stocks"). This matches the RESTful endpoint on the stock microservice.
Define the orderArticles method in ArticleStockService
- Use annotations like @POST, @Produces and @Consume as you would do in a RESTful endpoint.
- The method takes a List<ArticleOrderDTO> as argument

ArticleStockService Task Hint

1
2
3
4
5
6
7
8
9


@Path("/article-stocks")
@RegisterRestClient(configKey = "article-stock-api")
public interface ArticleStockService {

 @POST
 @Produces(MediaType.APPLICATION_JSON)
 @Consumes(MediaType.APPLICATION_JSON)
 Response orderArticles(List<ArticleOrderDTO> orders);
}

Use the following configuration in application.properties to define where our client will connect to

1
2
3
4


# ArticleStock API
application.articlestock.api.url: http://localhost:8081/
article-stock-api/mp-rest/url: ${application.articlestock.api.url}
article-stock-api/mp-rest/scope: javax.inject.Singleton

JSON RESTful Web Services

Our Microservice will provide a RESTful API to create and list existing orders. For this we need to create a JAX-RS resource.

The implementation is pretty straightforward. Define a ShopOrderResource in src/main/java/ch/puzzle/mm/rest/order/boundary endpoint and use the JAX-RS annotations. Below is a very simple implementation of a json rest service supporting GET and POST requests for http://<host>:<port>/shop-orders. For further details have a look at the Quarkus Guide Writing Json Rest Services .

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


@Path("/shop-orders")
@ApplicationScoped
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class ShopOrderResource {

 @GET
 public List<ShopOrder> list() {
 return null;
 }

 @POST
 @ChaosMonkey
 @Transactional
 public Response createShopOrder(ShopOrderDTO shopOrderDTO) {
 return null;
 }
}

Task 3.3.8 - JSON RESTful Resource

The ShopOrderResource is a boundary class and will be placed in src/main/java/ch/puzzle/mm/rest/order/boundary. Implement it according to the simple sniped above.

Inject the ArticleStockService to trigger the stock microservice
Inject the ShopOrderService to process the new order
- Annotate the ArticleStockService with @RestClient to tell Quarkus that this is a rest client.
Implement @GET which returns all ShopOrders
- Our entities are PanacheEntities and support the Active Record Pattern. They do provide handy static methods like listAll().
Implement @POST method to create a new order
- Annotate the method with @Transactional and @ChaosMonkey
- Use the injected ShopOrderService to create the order
- Use the injected ArticleStockService to trigger the stock handling
- After triggering the stock service, set the order status to COMPLETED
- Return the ShopOrder

GET Method Task Hint

1
2
3
4


@GET
public List<ShopOrder> listAll() {
 return ShopOrder.listAll();
}

POST Method Task Hint

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14


@POST
@ChaosMonkey
@Transactional
public Response createShopOrder(ShopOrderDTO shopOrderDTO) {
 // create ShopOrder locally
 ShopOrder shopOrder = shopOrderService.createOrder(shopOrderDTO);

 // call remote service
 articleStockService.orderArticles(shopOrderDTO.articleOrders);

 shopOrder.setStatus(ShopOrderStatus.COMPLETED);

 return Response.ok(shopOrder).build();
}

Complete Task Hint

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33


@ApplicationScoped
@Path("/shop-orders")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class ShopOrderResource {

 @Inject
 @RestClient
 ArticleStockService articleStockService;

 @Inject
 ShopOrderService shopOrderService;

 @GET
 public List<ShopOrder> listAll() {
 return ShopOrder.listAll();
 }

 @POST
 @ChaosMonkey
 @Transactional
 public Response createShopOrder(ShopOrderDTO shopOrderDTO) {
 // create ShopOrder locally
 ShopOrder shopOrder = shopOrderService.createOrder(shopOrderDTO);

 // call remote service
 articleStockService.orderArticles(shopOrderDTO.articleOrders);

 shopOrder.setStatus(ShopOrderStatus.COMPLETED);

 return Response.ok(shopOrder).build();
 }
}

Liquibase: https://www.liquibase.org/ ↩︎
MicroProfile Rest Client: https://github.com/eclipse/microprofile-rest-client/blob/main/spec/src/main/asciidoc/microprofile-rest-client-spec.asciidoc ↩︎

Docs: 3.4 Running the Lab

Mon, 01 Jan 0001 00:00:00 +0000

In the previous section we implemented the microservice. Lets build and run it.

Building

The order microservice provides a multistage Dockerfile for building the java application and creating a docker container.

Task 3.4.1 - Build docker container

Docker Version

Depending on your docker version you have to specify the output format. If you get a warning that there is no output specified for docker-container driver just add -o type=docker to your command line.

Change to base directory of this lab <PROJECT_ROOT>/code/rest and run docker build.

1

docker build -f rest-order/src/main/docker/Dockerfile.multistage.jvm -t microservices-lab-rest-order:latest rest-order

Running

Task 3.4.2 - Starting with Docker Compose

Enter a shell and change your directory to the <PROJECT_ROOT>/code/rest/docker folder.

If you have any previous lab running stop these containers

1

docker-compose down --remove-orphans

Start the microservices with

1
2
3
4
5
6
7
8
9


$ docker-compose up -d
Starting rest-alertmanager ... done
Starting rest-jaeger ... done
Starting rest-grafana ... done
Starting rest-prometheus ... done
Starting rest-stock-db ... done
Starting rest-order-db ... done
Starting rest-stock ... done
Starting rest-order ... done

Docs: 3.5 Using the Lab

Mon, 01 Jan 0001 00:00:00 +0000

Available Endpoints

Endpoint	Microservice	Description
http://localhost:8080/shop-orders	Order	Return and create new orders
http://localhost:8080/chaos-monkey	Order	Inject errors for testing our application
http://localhost:8081/articles	Stock	Query article metadata
http://localhost:8081/article-stocks	Stock	List or change the article stock count
http://localhost:8081/chaos-monkey	Stock	Inject errors for testing our application

Order state

The order is expected to have the following states

State	Description
NEW	Order is created. Stock reservation has not happened.
COMPLETE	Order is created and the article-stock count has been decreased according to the article counts.
INCOMPLETE	Order is created but reservation could not complete.

Sample Requests

In the source folder <PROJECT_ROOT>/code/rest/requests there is an IntelliJ compatible HTTP Request file. You may run requests directly from IntelliJ or use it as reference for your requests.

Get article stock information

1

curl --request GET --url http://localhost:8081/article-stocks

Get order information

1

curl --request GET --url http://localhost:8080/shop-orders

Create new order

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


curl --request POST \
 --url http://localhost:8080/shop-orders \
 --header 'Content-Type: application/json' \
 --data '{
 "articleOrders" : [
 {
 "articleId" : 1,
 "amount" : 1
 }
 ]
}'

Task 3.5.1 - Testing scenario

Run the following queries:

Check the article-stock count
Create an order for one item of article 1
Check the orders and article-stock again

Answer the questions:

Is the article-stock count correct?
Do you see any problems with this implementation?
What happens if you are running out of stock?
What’s the role of a transaction in this case?

Task Hint

This implementation seems to be the most logical and straightforward implementation when we just split a monolith in two microservices and let them communicate with a simple RESTful API.

The happy-path seems to be working quite well. But the edge cases do not behave quite well. Running out of stock will return an exception from the stock service. The TransactionalInterceptor will be triggered by the exception and will issue a rollback.

In the best case our order is not created and the stock is also not reserved. But are we sure that the stock reservation has not been made? The exception might have occurred after the reservation on the stock service has been completed. Think about a simple timeout between these two services.

We could remove the Transactional annotation to not cover the rest call to our stock service. We could then have a look at the response code to determine if the operation was successful. But would that solve all problems? What if an exception occurred in the order microservice just after the stock reservation?

Task 3.5.2 - Injecting an error

Let us inject an error at the end of the createShopOrder method of the order microservice.

Use the chaos-monkey rest endpoint to inject an error with the following request:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


curl --request POST \
 --url http://localhost:8080/chaos-monkey \
 --header 'Content-Type: application/json' \
 --data '[
 {
 "clazzName": "ShopOrderResource",
 "methodName": "createShopOrder",
 "enabled": true,
 "throwException": true
 }
]'

This will throw an InternalServerErrorException after execution of the method createShopOrder.

Ask yourself what to expect if an exception is thrown?

Issue the request from the Task 3.5.1 but use article 3 for your order.

What did you observe?
Is there any difference?
Is the data still consistent between stock and order microservice?
Can you imagine why any inconsistency may happen?
What’s the role of a transaction in this case?

Task Hint

As expected, the reservation took place in the stock microservice. However, our local transaction in the order microservice was rolled back as there was an exception. Our Data is no inconsistent as the stock count was decreased but the order was not created.

Docs: 3.6 Distributed Transactions (optional)

Mon, 01 Jan 0001 00:00:00 +0000

Optional Section

The following content is optional. If you have time, feel free to proceed.

Implementation

Do not make the described changes below to your implementation.

The only code change needed is described in Task 3.6.2 and targets your docker-compose.yaml file.

In this section we explain the changes made to the order and stock microservice. At the end of the section you will be switching to the provided pre-built docker images which contain all these changes.

Both our microservices use local transactions to store their data. There are several cases where data may get out of sync between order and stock microservice.

Problem

Think of the following scenario.

New order is requested using the RESTful API
- Order microservice will create and store a new order
- Order microservice will trigger the stock microservice to change the amount of in-stock items.
- Stock microservice changes the stock count and returns successfully.
- Order microservice fails for any reason

Since the stock successfully applied the reservation, and the order microservice failed, this results in an inconsistent state.

MicroProfile LRA (Long Running Actions)

Motivation for Long Running Actions

The proposal introduces annotations and APIs for services to coordinate long running activities whilst still maintaining loose coupling and doing so in such a way as to guarantee a globally consistent outcome without the need to take locks on data. […]

Traditional techniques for guaranteeing consistency in distributed environments has focused on XA transactions where locks may be held for long periods thereby introducing strong coupling between services and decreasing concurrency to unacceptable levels. Additionally, if such a transaction aborts then valuable work which may be valid will be rolled back. In view of these issues an alternative approach is desirable. - github.com/eclipse/microprofile-lra

LRA Protocol Sequence - source: github.com/eclipse/microprofile-lra

The MicroProfile LRA is built for microservices with RESTful communication. Each microservice participating in a LRA will have to provide a compensate action which will be invoked if the transaction is cancelled. The life cycle of LRAs can be managed by annotating the JAX-RS resources with the following annotations.

Annotation	Description
`@LRA`	Controls the life cycle of an LRA.
`@Compensate`	Indicates that the method should be invoked if the LRA is cancelled.
`@Complete`	Indicates that the method should be invoked if the LRA is closed.
`@Forget`	Indicates that the method may release any resources that were allocated for this LRA.
`@Leave`	Indicates that this class is no longer interested in this LRA.
`@Status`	When the annotated method is invoked it should report the status.
`@AfterLRA`	When an LRA has reached a final state the annotated method is invoked.
Source: github.com/eclipse/microprofile-lra

See the basic sample below.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45


@Path("/")
@ApplicationScoped
public class SimpleLRAParticipant {
 @LRA(LRA.Type.REQUIRES_NEW)
 @Path("/cdi")
 @PUT
 public void doInTransaction(@HeaderParam(LRA_HTTP_CONTEXT_HEADER) URI lraId) {
 /*
 * Perform business actions in the context of the LRA identified by the
 * value in the injected JAX-RS header. This LRA was started just before
 * the method was entered (REQUIRES_NEW) and will be closed when the
 * method finishes at which point the completeWork method below will be
 * invoked.
 */
 }

 @Complete
 @Path("/complete")
 @PUT
 public Response completeWork(@HeaderParam(LRA_HTTP_CONTEXT_HEADER) URI lraId) {
 /*
 * Free up resources allocated in the context of the LRA identified by the
 * value in the injected JAX-RS header.
 *
 * Since there is no @Status method in this class, completeWork MUST be
 * idempotent and MUST return the status.
 */
 return Response.ok(ParticipantStatus.Completed.name()).build();
 }

 @Compensate
 @Path("/compensate")
 @PUT
 public Response compensateWork(@HeaderParam(LRA_HTTP_CONTEXT_HEADER) URI lraId) {
 /*
 * The LRA identified by the value in the injected JAX-RS header was
 * cancelled so the business logic should compensate for any actions
 * that have been performed while running in its context.
 *
 * Since there is no @Status method in this class, compensateWork MUST be
 * idempotent and MUST return the status
 */
 return Response.ok(ParticipantStatus.Compensated.name()).build();
 }
}

Source: github.com/eclipse/microprofile-lra

Handling Compensations

What does providing a compensation mean to our microservices?

Each operation like order creation is run in a LRA context. The LRA context is reflected by an associated lra Id. The lraId will be injected in the HTTP-Headers. The call for compensation will also contain this id. This means that we must be able to revert our database changes by using the lra id whenever a REST call is made to the method annotated with @Compensate.

Task 3.6.1 - How to track LRAs

Think of compensating an order, which changes are necessary to our entities that we can revert the change later on?

Task Hint

Lets assume the LRA with id lra-1 needs to be compensated.

To compensate the database changes we need to know:

Which order is created by lra-1
- If the order is compensated we set a different order status
Which articles were reserved by lra-1
- If the order is compensated we have to revert the stock deduction

For this to achieve a simple approach could be:

Store lra-1 which was used to create the order in the ShopOrder entity.
Track the ArticleOrder with the associated lra-1 in the stock microservice as a new table/entity articlestockchange

Database Changes

This leads to the following changes in our database schema for the order microservice.

1
2
3
4
5
6


<changeSet author="lra" id="2">
 <comment>Add LRA field to store associated lra-id</comment>
 <addColumn tableName="shoporder">
 <column name="lra" type="varchar(255)" />
 </addColumn>
</changeSet>

On the stock microservice we will add the following migration to create a new table articlestockchange.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10


<createTable tableName="articlestockchange">
 <column name="id" type="bigint">
 <constraints primaryKey="true" nullable="false"/>
 </column>
 <column name="article_id" type="bigint">
 <constraints nullable="false" foreignKeyName="articlestockchange_article_fk" referencedColumnNames="id"/>
 </column>
 <column name="count" type="int"/>
 <column name="lra" type="varchar(255)"/>
</createTable>

LRA Coordinator

The LRA management will be made by an LRA coordinator. The Narayana Transaction Manager¹, which is the default JTA Transaction manager in Quarkus and WildFly Application servers, provides an implementation of an LRA Coordinator.

We will use this coordinator for our environment. Therefore, the following changes are made in the docker/docker-compose.yaml:

Add LRA Coordinator (do not forget to add the volume)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


 lra-coordinator:
 image: quay.io/jbosstm/lra-coordinator:latest
 hostname: lra-coordinator
 container_name: rest-lra-coordinator
 volumes:
 - lra-data:/opt/jboss
 networks:
 - rest
 ports:
 - 8090:8080
 environment:
 - LOG_LEVEL=DEBUG

# ... omitted

volumes:
 # ....
 lra-data:

Dependencies

Our application includes the following dependency in the pom.xml.

GroupId	ArtifactId	Description
`org.jboss.narayana.rts`	`narayana-lra`	Provides MicroProfile LRA implementation

Bridged Docker Network

The library uses the base url extracted form the request-uri to register itself at the LRA Coordinator. Our lab runs in a bridged docker network. Our requests are made to localhost with the mapped port of the docker host and are bridged to the container port inside the docker network.

This is why the order microservice would register itself as localhost if we call the order microservice with http://localhost:8080/shop-orders from the docker host.

To fix this environment specific issue we rewrite the host of the incoming requests from localhost to order. The registering at the LRA coordinator is therefore made correctly with order instead of localhost

To fix the docker network issue and specify the needed system variables we add the following to the order container specification:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


 order:
 # ... omitted
 environment:
 - JAVA_TOOL_OPTIONS=-Dlra.http.host=lra-coordinator -Dlra.http.port=8080
 - APPLICATION_REWRITE_BASE_ENABLED=true
 - APPLICATION_REWRITE_BASE_HOST=order

 stock:
 # ... omitted
 environment:
 - JAVA_TOOL_OPTIONS=-Dlra.http.host=lra-coordinator -Dlra.http.port=8080
 - APPLICATION_REWRITE_BASE_ENABLED=true
 - APPLICATION_REWRITE_BASE_HOST=stock

Using LRAs

Now we are going to change our code to make use of LRAs. We have to annotate the JAX-RS resources with the specific LRA-Annotations.

Changes in Order Microservice

We have to make the following changes to our ch.puzzle.mm.rest.order.boundary.ShopOrderResource

Annotate the POST method createShopOrder and specify the HeaderParams to be used to extract the LRA ID. The extracted LRA ID will be stored in the database to be able to complete or compensate the LRA.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


@POST
@ChaosMonkey
@LRA(value = LRA.Type.REQUIRES_NEW,
 cancelOn = {
 Response.Status.INTERNAL_SERVER_ERROR // cancel on a 500 code
 },
 cancelOnFamily = {
 Response.Status.Family.CLIENT_ERROR // cancel on any 4xx code
 })
public Response createShopOrder(@HeaderParam(LRA.LRA_HTTP_CONTEXT_HEADER) String lraUrl,
 ShopOrderDTO shopOrderDTO) {
 // extract lraId from headerstring
 String lraId = getLraId(lraUrl);

 // create ShopOrder locally
 ShopOrder shopOrder = shopOrderService.createOrder(shopOrderDTO, lraId);

 // call remote service
 articleStockService.orderArticles(shopOrderDTO.articleOrders);

 return Response.ok(shopOrder).build();
}

Implement @Complete method and set the COMPLETE status for this order. The LRA expects us to return the Status.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


@PUT
@Path("/complete")
@Complete
@Consumes(MediaType.TEXT_PLAIN)
@Transactional
public Response completeShopOrder(@HeaderParam(LRA.LRA_HTTP_CONTEXT_HEADER) String lraUrl) {
 // extract lraId from headerstring
 String lraId = getLraId(lraUrl);

 int updated = ShopOrder.setStatusByLra(ShopOrderStatus.COMPLETED, lraId);

 return Response.ok(ParticipantStatus.Completed.name()).build();
}

Implement @Compensate method and set the INCOMPLETE status. Also return the Status for the LRA.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


@PUT
@Path("/compensate")
@Compensate
@Consumes(MediaType.TEXT_PLAIN)
@Transactional
public Response compensateShopOrder(@HeaderParam(LRA.LRA_HTTP_CONTEXT_HEADER) String lraUrl) {
 // extract lraId from headerstring
 String lraId = getLraId(lraUrl);

 int updated = ShopOrder.setStatusByLra(ShopOrderStatus.INCOMPLETE, lraId);

 return Response.ok(ParticipantStatus.Compensated.name()).build();
}

Further you have to add a simple method to extract the LRA-id

1
2
3


String getLraId(String longRunningHeader) {
 return longRunningHeader.substring(longRunningHeader.lastIndexOf('/') + 1);
}

Complete Source of ShopOrderResource

We have also added method to the ShopOrder entity to change the ShopOrder by a given lraId. The entity also includes the lra field which we’ve been added to our database schema.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


@Entity
public class ShopOrder extends PanacheEntity {

 @NotNull
 @Column(unique=true, nullable = false)
 private String lra;

 // omitted

 public String getLra() {
 return lra;
 }

 public void setLra(String lra) {
 this.lra = lra;
 }

 public static int setStatusByLra(ShopOrderStatus status, String lra) {
 return update("status = :status where lra = :lra",
 Parameters.with("status", status).and("lra", lra));
 }
}

Complete Source of ShopOrder

Our ShopOrderService now takes an additional argument the lraId which is stored in the newly created field on the ShopOrder entity.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13


@ApplicationScoped
public class ShopOrderService {

 @Transactional
 public ShopOrder createOrder(ShopOrderDTO dto, String lra) {
 // omitted

 // set lra id
 shopOrder.setLra(lra);

 // omitted
 }
}

Complete Source of ShopOrderService

At the end we switch our rest-client to the LRA-Enabled version for the stock service by changing the path to @Path("/lra/article-stocks").

Complete Source of ArticleStockService

Changes in Stock Microservice

Pre-built Container

The pre-built version of quay.io/acend/microservices-lab-rest-stock:latest supports both versions, the original one and the LRA enabled rest endpoint. That’s why there exists a second JAX-RS resource ArticleStockLraResource in the solution code.

We have to make the following changes to our ch.puzzle.mm.rest.stock.boundary.ArticleStockResource

Annotate the POST method createStockOrder and specify the HeaderParams to be used to extract the LRA ID. The extracted LRA ID will be stored in the database to be able to complete or compensate the LRA.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


@POST
@ChaosMonkey
@LRA(value = LRA.Type.MANDATORY, end = false)
public Response createStockOrder(@HeaderParam(LRA.LRA_HTTP_CONTEXT_HEADER) String lraUrl,
 List<ArticleOrderDTO> articles) {
 // extract lraId from headerstring
 String lraId = getLraId(lraUrl);

 // handle stock count
 articleStockService.orderArticles(articles, lraId);
}

Implement @Complete method to just return the LRA status.

1
2
3
4
5
6


@PUT
@Path("/complete")
@Complete
public Response completeShopOrder(@HeaderParam(LRA.LRA_HTTP_CONTEXT_HEADER) String lraUrl) {
 return Response.ok(ParticipantStatus.Completed.name()).build();
}

Implement @Compensate method which uses the stored LRA from the database and adds the reserved (decreased) amount of an article to the stock count.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


@PUT
@Path("/compensate")
@Compensate
public Response compensateShopOrder(@HeaderParam(LRA.LRA_HTTP_CONTEXT_HEADER) String lraUrl) {
 String lraId = getLraId(lraUrl);

 // find article stock change by LRA-ID and increment the stockcount.
 ArticleStockChange.findByLraId(lraId).forEach(asc -> {
 ArticleStock stock = ArticleStock.findByArticleId(asc.getArticle().id);
 int oldStockCount = stock.getCount();
 stock.setCount(stock.getCount() + asc.getCount());
 log.info("Compensating for Article {}: oldCount={}, release={}, newCount={}", stock.getArticle().id, oldStockCount, asc.getCount(), stock.getCount());
 });

 return Response.ok(ParticipantStatus.Compensated.name()).build();
}

Complete Source of ArticleStockLraResource

Changing to the LRA enabled version

To change to the LRA version, switch to the pre-built docker images.

Task 3.6.2 - Switching to the pre-built images

We provide a docker-compose.solution.yaml docker file which includes all changes necessary to use the LRA enabled version.

You may simply start the environment using this docker-compose file. Restart your docker environment with

1
2


docker-compose down --remove-orphans
docker-compose -f docker-compose.solution.yaml up -d

Source of docker-compose.solution.yaml

Task 3.6.3 - Rerun the tests from previous section

Issue the request from the Task 3.5.1.

What did you observe?
Is there any difference?
Is the data consistent between stock and order microservice?
What’s the role of a transaction in this case?
What’s the role of the LRA coordinator?

Use the chaos-monkey rest endpoint to inject an errors at the order microservice.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


curl --request POST \
 --url http://localhost:8080/chaos-monkey \
 --header 'Content-Type: application/json' \
 --data '[
 {
 "clazzName": "ShopOrderResource",
 "methodName": "createShopOrder",
 "enabled": true,
 "throwException": true
 }
]'

Rerun some tests with the monkey in place. What did happen?

You may delete the ChaosMonkey on the order microservice with

1
2
3


curl --request DELETE \
 --url http://127.0.0.1:8080/chaos-monkey/ShopOrderResource%23createShopOrder \
 --header 'Content-Type: application/json'

Now create a chaos-monkey at the stock microservice:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


curl --request POST \
 --url http://127.0.0.1:8081/chaos-monkey \
 --header 'Content-Type: application/json' \
 --data '[
 {
 "clazzName": "ArticleStockLraResource",
 "methodName": "createStockOrder",
 "enabled": true,
 "errorRate": 0,
 "latencyMs": 0,
 "permitsPerSec": 9.223372036854776E18,
 "rateLimiterType": "BLOCK",
 "throwException": true
 }
]'

Now rerun some tests with the monkey on the stock microservice.

Is your data still consistent?

Task Hint

Using LRAs the LRA coordinator tracks the state of the LRA. If an error is occurred it will notify every participant of the LRA about the error by triggering the method annotated with @Compensate. Since our microservices now track which entities have been created in which LRA contexts they are able to revert or compensate the changes.

You may see this clearly in the logs of the stock or order microservice.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


----------------------------
Create ArticleOrder: 0_ffffc0a86003_-1be006ed_5ff8d079_12
LRA Transaction Id: 0_ffffc0a86003_-1be006ed_5ff8d079_12
----------------------------
Article with id 1 processed. Stock oldCount=10, ordered=1, newCount=9

---------------------------
Compensate ArticleOrder
LRA Transaction Id: 0_ffffc0a86003_-1be006ed_5ff8d079_12
----------------------------
Compensating for Article 1: oldCount=9, release=1, newCount=10

Narayana Transaction Manager: https://narayana.io ↩︎