DirectJobHub
Jobs · Marketing · Indiana

Kafka REST Proxy and Oracle APEX – GPM Factory

GPM FACTORY · Indiana, United States · 1 mo ago
MarketingFull-time

About the Sample Application

This Oracle APEX sample application demonstrates how to use the Kafka REST Proxy through an Oracle APEX client. The application, available on GitHub, includes a simple Kafka producer and consumer, helping users familiarize themselves with the produce/consume process and commit feature subtleties.

The demo assumes a collection of devices spread across several cities, allowing operators to insert manual messages into the stream or poll events from a given topic.

Prerequisites

  • Install an on-premise Apache Kafka cluster, use a Docker image, or subscribe to the Confluent platform.
  • For on-premise Kafka, install the community version of Confluent REST Proxy and set up TLS to enable calls from a free-tier APEX instance (e.g., apex.oracle.com). HTTP endpoints are also supported per Oracle rules.
  • The APEX application is available on GitHub and was exported using version 23.2.

Installation of Kafka

For a single broker on a Linux server, follow these steps:

  • Download Apache Kafka
  • Install Kafka
  • Install REST Proxy
  • Create start/stop scripts (see Appendices).

To set up TLS for the REST Proxy, refer to resources like Ken Coenen’s post for guidance on OpenSSL, keystores, and adapting the etc/kafka-rest/kafka-rest.properties file. Review REST Proxy Security and adjust ssl.client.authentication as needed.

Description of the Application

A regular Oracle APEX application named "Kafka" relies on a PL/SQL package (KAFKA_PKG) that wraps calls to the REST Proxy. The application is available on GitHub and can be imported into an Oracle APEX instance (version 23.2 or later).

During import:

  • Set the REST Proxy endpoint.
  • Accept the installation of supporting objects.

After launching the Kafka app:

  • Navigate to the setup option to check the default consumer name (e.g., "patrick") and the Consumer Group name (default: "my_json_consumer_group").

The producer menu allows adding a single message or a batch of ten records based on the content of the VILLES table (customizable in KAFKA_PKG).

Features

  • Listing existing topics.
  • Creating/deleting a consumer instance and subscribing to a topic.
  • Consuming records from an offset (the records page uses a data source, while other actions are implemented in the KAFKA_PKG PL/SQL package).

Notes About the Consumer Instance

When creating a new consumer instance in a consumer group, the max idle session is set to approximately 4 minutes on the server side. Regular polling is required to avoid recreating the consumer instance. The sample application includes a page with a chart that refreshes periodically to prevent excessive idle time.

Clicking on a topic entry displays the lag between the last commit point and the latest entry.

Appendices

Scripts for Starting and Stopping Apache Kafka

Starting Kafka at boot can be tricky due to permission issues. Below are manual scripts for launching the required modules. Since Kafka was used with Zookeeper, the first script starts Zookeeper, then a Kafka server in the background (alternatively, Kafka with KRaft can be used). The second script launches the REST Proxy, either in the foreground or as a daemon.

Scripts are available in the GitHub repository:

  • Start/Stop/Status for Zookeeper and Kafka.
  • Start/Stop/Status for REST Proxy.

Adapted from: StackOverflow.

Author

Patrick GPM Factory

Similar jobs