5.8 Using the Lab

Making requests against the Lab environment.

We have finished our work on the environment and our order microservice. We are now ready to setup our debezium connectors.

Debezium Connectors

With starting the environment we have also started Kafka Connect. We need to configure our Debezium PostgreSQL Connector to listen for events on the order-db and stock-db database.

Task 5.8.2 - Configuring Debezium Connectors

Configuring the Debezium PostgreSQL Connector for the order-db database.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
curl --request POST \
  --url http://localhost:8083/connectors \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "order-events",
    "config": {
      "connector.class" : "io.debezium.connector.postgresql.PostgresConnector",
      "tasks.max" : "1",
      "database.hostname" : "order-db",
      "database.port" : "5432",
      "database.user" : "admin",
      "database.password" : "1234",
      "database.dbname" : "admin",
      "database.server.name" : "order-db",
      "schema.include.list" : "public",
      "table.include.list" : "public.outboxevent",
      "tombstones.on.delete" : "false",
      "transforms" : "outbox,topiccase",
      "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter",
      "transforms.outbox.route.by.field" : "type",
      "transforms.outbox.route.topic.replacement" : "order-${routedByValue}-events",
      "transforms.outbox.table.field.event.timestamp" : "timestamp",
      "transforms.outbox.table.fields.additional.placement" : "type:header:eventType",
      "transforms.topiccase.type" : "ch.puzzle.mm.debezium.connect.smt.ChangeTopicCase",
      "transforms.topiccase.toCase" : "lower"
    }
}'

Configuring the Debezium PostgreSQL Connector for the stock-db database.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
curl --request POST \
  --url http://localhost:8083/connectors \
  --header 'Accept: application/json' \
  --header 'Content-Type: application/json' \
  --data '{
    "name": "stock-events",
    "config": {
      "connector.class" : "io.debezium.connector.postgresql.PostgresConnector",
      "tasks.max" : "1",
      "database.hostname" : "stock-db",
      "database.port" : "5432",
      "database.user" : "admin",
      "database.password" : "1234",
      "database.dbname" : "admin",
      "database.server.name" : "stock-db",
      "schema.include.list" : "public",
      "table.include.list" : "public.outboxevent",
      "tombstones.on.delete" : "false",
      "transforms" : "outbox,topiccase",
      "transforms.outbox.type" : "io.debezium.transforms.outbox.EventRouter",
      "transforms.outbox.route.by.field" : "type",
      "transforms.outbox.route.topic.replacement" : "stock-${routedByValue}-events",
      "transforms.outbox.table.field.event.timestamp" : "timestamp",
      "transforms.outbox.table.fields.additional.placement" : "type:header:eventType",
      "transforms.topiccase.type" : "ch.puzzle.mm.debezium.connect.smt.ChangeTopicCase",
      "transforms.topiccase.toCase" : "lower"
    }
}'

As you can see in the configuration of the connectors we specify our connection information and our tables with the table.include.list and our transforms. We use the EventRouter transformation which handles our outboxevent table.

Check if the connectors are created successfully

1
2
3
4
5
6
7
8
curl --request GET \
  --url http://127.0.0.1:8083/connectors \
  --header 'Accept: application/json'

[
  "order-events",
  "stock-events"
]

Sample Requests

We provide a requests/api.http file containing all requests needed for this lab. This file is compatible with IntelliJ IDEA and also includes the curl version for the requests.

Get article stock information

1
curl --request GET --url http://localhost:8081/article-stocks

Get order information

1
curl --request GET --url http://localhost:8080/shop-orders

Create new order

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
curl --request POST \
  --url http://localhost:8080/shop-orders \
  --header 'Content-Type: application/json' \
  --data '{
    "articleOrders" : [
        {
            "articleId" : 1,
            "amount" : 1
        }
    ]
}'

Task 5.8.3 - Using the application

There is a Viewer Microservice running which provides an overview of events in the Kafka Topics. It uses websockets and shows newly arriving events.

Feel free to test the following scenarios:

Order Article 1 with quantity 9 (e.g. orderId: 100000)
Order Article 1 with quantity 3

What do you expect? Was the second order rejected?

What happens if you proceed like this?

Cancel OrderId 100000
Order Article 1 with quantity 3

What do you expect? Was the second order rejected?

What happens if you proceed like this?

Delete the connectors (order-events and stock-events)
Order Article 1 with quantity 3
Order Article 1 with quantity 3
Order Article 1 with quantity 3
Recreate the connectors (order-events and stock-events)

Were there any errors when the connectors were down? Do the microservices catch up after recreating?

What does the Jaeger Tracing show?

Last modified February 12, 2024: fix debezium lab version (9e6013e)