Writing Data from Apache Kafka to Text File

When working with Apache Kafka you might want to write data from a Kafka topic to a local text file. This is actually very easy to do with Kafka Connect. Kafka Connect is a framework that provides scalable and reliable streaming of data to and from Apache Kafka. With Kafka Connect, writing a topic’s content

read more

Writing Text File contents to Kafka with Kafka Connect

apache kafka logo

When working with Kafka you might need to write data from a local file to a Kafka topic. This is actually very easy to do with Kafka Connect. Kafka Connect is a framework that provides scalable and reliable streaming of data to and from Apache Kafka. With Kafka Connect, writing a file’s content to a

read more

Pushing Tags to a Remote Repository in Git

Tags in Git are a way to bookmark specific commits. This is often done to indicate different versions or releases of a project. When pushing to a remote repository tags are not included by default. To push a tag to the remote repository use: git push origin <tag name> To push all local tags to

read more

Linux sort Command Examples

What is Linux sort? sort is a very useful command line utility used to sort the lines of a file or input stream. sort can be used to sort input by entire lines, single columns, or different column ranges in a variety of ways. Sorting by Entire Lines The default behavior of the sort command

read more

Sending Key Value Messages with the Kafka Console Producer

When working with Kafka you might find yourself using the kafka-console-producer (kafka-console-producer.sh). The kafka-console-producer is a program included with Kafka that creates messages from command line input (STDIN). However, simply sending lines of text will result in messages with null keys. In order to send messages with both keys and values you must set the

read more

Creating a Simple Kafka Consumer

Apache Kafka is a fault tolerant publish-subscribe streaming platform that lets you process streams of records as they occur. If you haven’t installed Kafka yet, see our Kafka Quickstart Tutorial to get up and running quickly. In this post we will talk about creating a simple Kafka consumer in Java. Kafka Consumer Code The example

read more

Count JSON Array Elements with jq

Sometimes when working with JSON on the command line, it is helpful to know how many elements exist in a JSON array. Fortunately this is easy to do with jq. jq is a lightweight, easy to use, command line JSON processor. *More information about jq can be found at https://stedolan.github.io/jq/manual/. Counting Array Elements in single

read more

Count the Number of Fields in JSON with jq

Sometimes when working with JSON on the command line, it is helpful to know how many fields exist in each document. Fortunately this is easy to do with jq. jq is a lightweight, easy to use, command line JSON processor. Counting Fields in single JSON String If you want to test individual JSON strings, you

read more

Creating a Simple Kafka Producer in Java

apache kafka logo

Apache Kafka is a fault tolerant publish-subscribe streaming platform that lets you process streams of records as they occur. If you haven’t installed Kafka yet, see our Kafka Quickstart Tutorial to get up and running quickly. In this post we discuss how to create a simple Kafka producer in Java. Kafka Producer Java Code The

read more

Apache Kafka Docker Image Example

apache kafka logo

Apache Kafka is a fault tolerant publish-subscribe streaming platform that lets you process streams of records as they occur. This post is a step by step guide of how to build a simple Apache Kafka Docker image. The original Dockerfile can be found here: https://github.com/nsonntag/docker-images/tree/master/kafka-quickstart. The Dockerfile This Dockerfile is very simple. It installs Java

read more