How to Keep Docker Containers Running

Docker containers, when run in detached mode (the most common -d option), are designed to shut down immediately after the initial entrypoint command (program that should be run when container is built from image) is no longer running in the foreground. This can cause problems because often servers or services running in Docker containers are

read more

Creating a Hadoop Docker Image

Apache Hadoop is a core big data technology. Running Hadoop on Docker is a great way to get up and running quickly. Below are the basic steps to create a simple Hadoop Docker image. Pick an OS Hadoop runs great on a variety of Linux distos. In this post we use Ubuntu 16.04. Install Required

read more

How to Change Hadoop Output Delimiter

Hadoop’s default output delimiter (character separating the output key and value) is a tab (“\t”). This post explains how to change the default Hadoop output delimiter. Output Delimiter Configuration Property The output delimiter of a Hadoop job can easily be changed by setting the mapred.textoutputformat.separator configuration property. This property can be set from the code

read more

How to Create a Disk Based List in Java

Occasionally when programming in Java you will need a List too large to hold in memory. MapDB is an open source Java library that allows you to create a disk based List (and other Java Collections) very easily. MapDB is a hybrid of an embedded database engine and the Java Collections framework. It provides Maps,

read more

How to Create a Disk Based Map in Java

Occasionally when programming in Java you will need a Map too large to hold in memory. MapDB is an open source Java library that allows you to create a disk based Map (and other Java Collections) very easily. MapDB is a hybrid of an embedded database engine and the Java Collections framework. It provides Maps,

read more

How to Uppercase/Lowercase Text in Unix/Linux

Uppercasing and lowercasing is a common task when working with text. Below are examples of doing this with standard Linux/Unix utilities. Lowercasing text with tr

Uppercasing text with tr

Lowercasing text with awk

Uppercasing text with awk

Lowercasing text with sed

Uppercasing text with sed

Introduction to Java Streams

What are Java Streams? A Java stream is a sequence of elements of a specific type that are consumed from a source like Collections, arrays, or I/O resources. Streams are similar to collections in that they can both be used to process and aggregate data. However, there are some big differences. Although the idea of

read more

How to Uppercase/Lowercase text with sed

sed is a common Linux/Unix utility used to parse and transform text. Below are examples of uppercasing and lowercasing text with sed: Uppercasing Text with sed

[a-z] is the regular expression which will match lowercase letters. \U& is used to replace these lowercase letters with the uppercase version. Lowercasing Text with sed

[A-Z]

read more

Convert JSON to TSV with jq

jq is a lightweight JSON command line processor that makes many JSON processing tasks very easy. A common task when working with JSON is to convert records into a TSV (tab delimited) format. Below are examples of how to convert JSON to TSV with jq. Converting JSON to TSV In order to convert JSON to

read more

Convert JSON to CSV with jq

jq is a lightweight JSON command line processor that makes many JSON processing tasks very easy. A common task when working with JSON is to convert records into a CSV format. Below are examples of how to convert JSON to CSV with jq. Converting JSON to CSV In order to convert JSON to CSV with

read more