Creating Graphs with the Apache Zeppelin Shell Interpreter

Using Apache Zeppelin’s shell (sh) interpreter is a great way to run and share shell scripts. This way you can use all your favorite command line tools like sed and awk to create great looking visualizations using Apache Zeppelin. By default your shell output will not be displayed in a tabular format or leverage Zeppelin’s

read more

How to Left Pad Numbers in AWK

Occasionally when working in a Linux/Unix environment comes the need to left pad numbers with zeros (add leading zeros to numbers). This is often done to make a series of numbers the same length. An example of doing this with awk can be seen below:

How to Left Pad Numbers in Linux

There are many ways to left pad numbers in Linux. Usually this seems to be done by adding leading zeros in front of the main number. A few examples of doing this can be seen below: Using printf

Using AWK

Looping in Bash

Count Number of Occurrences of Characters in Line with AWK

Being able to count the number of occurrences of characters or words in text is a handy trick. Fortunately this is very easy to do in awk with the gsub() function. The syntax for using gsub() looks like this: gsub(regexp, replacement [, target]) gsub() will search target for substrings matching the provided regular expression and

read more

How to Sum Values in Awk

Knowing how to sum values in Awk can be extremely useful when working with data. Fortunately, it is very easy. Using variables in Awk will feel different than in other languages. There are no data types in Awk. A variable will either be considered a number or string depending on the context in which it

read more

3 ways to get the Nth Line of a File in Linux

The need to get/print a particular line of a file on the Linux shell is a common task. Luckily there are various ways to do this. Below are three great ways to get the nth line of a file in Linux. 1. head / tail Simply using the combination of the head and tail commands

read more

Count Unique Elements in Text File with AWK

Counting the distinct/unique elements of text file is a common task.  Below is an example of doing this is AWK, using sample_data_1.txt.

Here is what is happening above: cat sample_data_1.txt – reading the file piping the data to AWK BEGIN{FS=”\t”} – specifying the field separators of the file NR>1 – Only executing the following

read more