Count JSON Array Elements with jq

Sometimes when working with JSON on the command line, it is helpful to know how many elements exist in a JSON array. Fortunately this is easy to do with jq. jq is a lightweight, easy to use, command line JSON processor. *More information about jq can be found at https://stedolan.github.io/jq/manual/. Counting Array Elements in single

read more

Count the Number of Fields in JSON with jq

Sometimes when working with JSON on the command line, it is helpful to know how many fields exist in each document. Fortunately this is easy to do with jq. jq is a lightweight, easy to use, command line JSON processor. Counting Fields in single JSON String If you want to test individual JSON strings, you

read more

Sorting JSON by Value with JQ (Command Line JSON Processor)

jq is a lightweight command line JSON processor that is very easy to use. Sometimes it is helpful to see your data sorted by a particular field value. Luckily jq makes this easy to do. Here are some sample JSON records we will be working with in this post:

Sorting JSON by value with

read more

Using Variables in JQ (Command Line JSON Parser)

jq is a lightweight command line JSON processor that is very easy to use. Sometimes being able to use variables within a jq script is very useful. Below are various examples of doing this. Here is a sample record from the JSON file we use in most examples:

Using Simple Variables in jq To

read more

How to filter JSON records by value with jq

Often when working with JSON in a Linux/Unix environment, it is nice to be able to filter records based on the values of certain fields. jq is a lightweight command line JSON processor that is very easy to use. jq offers an easy way to filter JSON records based on field values with the select()

read more

How to Pretty Print JSON on the Command Line

JSON is a very popular platform independent data format. One of the great benefits of working with JSON is that it is generally easy to read. However, reading JSON objects becomes more difficult as the objects become large, especially on the command line. Pretty printing JSON records on the command line makes reading it much

read more

Creating JSON with JSON.simple (Java)

JSON is a popular way to represent and transfer data. Creating JSON with JSON.simple (a Java library from Google) is very easy. JSON.simple also performs very well compared to other Java JSON libraries when parsing a variety of file sizes (see results of performance tests here). Below is a simple example of building a JSON

read more

How to Extract Nested JSON Data in Spark

JSON is a very common way to store data. But JSON can get messy and parsing it can get tricky. Here are a few examples of parsing nested data structures in JSON using Spark DataFrames (examples here done with Spark 1.6.0). Our sample.json file:

Assuming you already have a SQLContext object created, the examples

read more

How to Read / Write JSON in Spark

Needing to read and write JSON data is a common big data task. Thankfully this is very easy to do in Spark using Spark SQL DataFrames. Spark SQL can automatically infer the schema of a JSON dataset, and use it to load data into a DataFrame object. A DataFrame’s schema is used when writing JSON

read more

How to Exclude Fields using Gson

There are a lot of great Java libraries for working with JSON. One of these is Google’s Gson, which has the goal of providing simple toJson() and fromJson() methods to convert Java objects to JSON and vice-versa. Often when serializing objects into JSON you will want to exclude certain object fields. There a couple of

read more