Connecting Docker Containers with Networking

Docker Containers are isolated environments often running a single service. Usually individual containers need to be connected to others to accomplish more complex tasks. Starting in Docker 1.9 this has become very easy to do with Networking. Docker Networking Docker Networking is a feature that allows you to create a virtual network and attach containers

read more

Select Random Lines from file in Linux

Being able to select random lines from a file in Linux can be very helpful and convenient. There are a few easy ways to do this, including using the shuf utilty. shuf is included on many Linux/Unix systems as part of GNU coreutils. GNU coreutils can be easily installed if not already. Selecting Random Lines

read more

Set AWS Credentials in Cloudera Quickstart Docker Container

Cloudera’s Quickstart Image is a fantastic way to get started quickly with the big data ecosystem. With software such as Hadoop, Spark, Hive, Pig, Impala, and Hue already set up, this Docker image is a must in your big data toolkit. One thing the Cloudera Quickstart container is lacking however, is an easy way to

read more

How to Generate a UUID in Linux

UUID stands for Universally Unique Identifier. UUIDs are used as IDs (to identify) unique objects or records. An easy way to generate UUIDs in Linux is to use the uuidgen utility on the Linux/Unix command line. Generating UUID with uuidgen Simply executing uuidgen will generate a random UUID. Using the -t option will create a

read more

How to Delete Local and Remote Branches in Git

Working with branches is a core part of using Git. Often you will want to clean up and remove local and/or remote branches. Below are examples of how to delete local and remote branches in Git. Delete a Local Branch 1) git branch -d <branch-name> 2) git branch –delete <branch-name> The results of these two

read more

How to Copy local files to S3 with AWS CLI

AWS CLI has made working with S3 very easy. Once you get AWS CLI installed you might ask “How do I start copying local files to S3?” The syntax for copying files to/from S3 in AWS CLI is: aws s3 cp <source> <destination> The “source” and “destination” arguments can either be local paths or S3

read more

Copy all Files in S3 Bucket to Local with AWS CLI

The AWS CLI makes working with files in S3 very easy. However, the file globbing available on most Unix/Linux systems is not quite as easy to use with the AWS CLI. S3 doesn’t have folders, but it does use the concept of folders by using the “/” character in S3 object keys as a folder

read more

How to Sum Values in Awk

Knowing how to sum values in Awk can be extremely useful when working with data. Fortunately, it is very easy. Using variables in Awk will feel different than in other languages. There are no data types in Awk. A variable will either be considered a number or string depending on the context in which it

read more

How to Combine multiple Commits in Git

The ability to combine multiple commits in Git is a huge plus. Combining commits requires “rebasing” which will essentially rewrite the project history appending commits onto the last commit (by default) in a different branch, or even to an earlier commit in the same branch. Rebasing can have some damaging effects, so be careful when

read more