How to parsing JSON with Unix tools

In Unix systems, there are multiple methods to parse JSON files. The most commonly used tools include jq, awk, and sed. Below, I will introduce the usage of each tool step by step:

1. Using the jq Tool

jq is a lightweight and flexible command-line JSON processor. It provides a DSL (Domain-Specific Language) that makes it very convenient to read, filter, map, and transform JSON data. Here is an example of using jq:

Suppose we have a JSON file named data.json with the following content:

json
{
  "name": "John Doe",
  "age": 30,
  "location": "New York"
}

To retrieve the person's name, we can use the following jq command:

sh
jq '.name' data.json

This will output:

shell
"John Doe"

2. Using the awk Tool

Although awk is not specifically designed for processing JSON, it can handle text data and thus parse simple JSON strings. For complex JSON structures, awk can be cumbersome and error-prone. Here is an example of using awk:

sh
echo '{"name": "John Doe", "age": 30}' | awk -F '[:,}]' '{for(i=1;i<=NF;i++){if($i~/name/){print $(i+1)}}}' | tr -d '" '

This command outputs:

shell
John Doe

The command leverages multiple utilities: awk processes the text to locate 'name' and print the subsequent field, while tr removes extraneous characters such as quotes and spaces.

3. Using the sed Tool

Similar to awk, sed is not specifically designed for processing JSON data but can handle simple cases. Here is an example of using sed:

sh
echo '{"name": "John Doe", "age": 30}' | sed -n 's/.*"name": "\([^\"]*\)".*/\1/p'

This will output:

shell
John Doe

In this example, sed uses regular expressions to extract the value following the 'name' key and print it.

In summary, although awk and sed can parse JSON, they are not purpose-built for this task and may not be suitable for complex JSON structures. In practice, jq is the preferred tool for handling JSON data due to its specialized design, concise syntax, and robust capabilities. Of course, other tools such as Python's json module can also be used to parse and process JSON data.