jq
jq linux command cheatsheet by Thamizhiniyan C S
Introduction
jq is a tool for processing JSON inputs, applying the given filter to its JSON text inputs and producing the filter's results as JSON on standard output.
Syntax
jq [options...] filter [files...]
Installation
For Installing on a Debian based OS or Ubuntu use the following command:
Important Flags
-c
Compact instead of pretty-printed output.
-n
Use `null` as the single input value.
-e
Set the exit status code based on the output.
-s
Read (slurp) all inputs into an array; apply filter to it.
-r
Output raw strings, not JSON texts. (prints without quotes)
-R
Read raw strings, not JSON texts.
-C
Colorize JSON.
-M
Monochrome (don't colorize JSON).
-S
Sort keys of objects on output.
--tab
Use tabs for indentation.
--arg a v
Set variable $a to value .
--argjson a v
Set variable $a to JSON value .
--slurpfile a f
Set variable $a to an array of JSON texts read from .
--rawfile a f
Set variable $a to a string consisting of the contents of .
--args
Remaining arguments are string arguments, not files.
--jsonargs
Remaining arguments are JSON arguments, not files.
--
Terminates argument processing.
jq Filters
Basic concepts
The syntax for jq is pretty coherent:
,
Filters separated by a comma will produce multiple independent outputs
?
Will ignores error if the type is unexpected
[]
Array construction
{}
Object construction
+
Concatenate or Add
-
Difference of sets or Substract
length
Size of selected element
|
Pipes are used to chain commands in a similar fashion than bash
Dealing with json objects
Display all keys
jq 'keys'
Adds + 1 to all items
jq 'map_values(.+1)'
Delete a key
jq 'del(.foo)'
Convert an object to array
to_entries | map([.key, .value])
Dealing with fields
Concatenate two fields
fieldNew=.field1+' '+.field2
Dealing with json arrays
Slicing and Filtering
All
jq .[]
First
jq '.[0]'
Range
jq '.[2:4]'
First 3
jq '.[:3]'
Last 2
jq '.[-2:]'
Before Last
jq '.[-2]'
Select array of int by value
jq 'map(select(. >= 2))'
Select array of objects by value
jq '.[] | select(.id == "second")'
Select by type
jq '.[] | numbers' with type been arrays, objects, iterables, booleans, numbers, normals, finites, strings, nulls, values, scalars
Mapping and Transforming
Add + 1 to all items
jq 'map(.+1)'
Delete 2 items
jq 'del(.[1, 2])'
Concatenate arrays
jq 'add'
Flatten an array
jq 'flatten'
Create a range of numbers
jq '[range(2;4)]'
Display the type of each item
jq 'map(type)'
Sort an array of basic type
jq 'sort'
Sort an array of objects
jq 'sort_by(.foo)'
Group by a key - opposite to flatten
jq 'group_by(.foo)'
Minimun value of an array
jq 'min' .See also min, max, min_by(path_exp), max_by(path_exp)
Remove duplicates
jq 'unique' or jq 'unique_by(.foo)' or jq 'unique_by(length)'
Reverse an array
jq 'reverse'
Examples
In the given examples, the following JSON data is used which is stored in a file "json.json".
Pretty Printing JSON
Using jq along with cat for Pretty printing the JSON
Output a JSON file, in pretty-print format
To pretty print the json
Extracting Keys
To extract all keys from json
To extract all keys from json
Array Manipulation
Output all elements from arrays (or all key-value pairs from objects) in a JSON file
Read JSON objects from a file into an array, and output it [ Note: In our case the given the given JSON data is already an array of JSON objects. Thus, first I extracted the JSON objects from the array and piped it again to jq to use -slurp, for the sake of demonstration ]
Extract as stream of values instead of a list (array)
To count elements
Create a range of numbers
Array Indexing
To access first list item
Output the first element in a JSON file
Output the value of a given key of the first element in a JSON file
Array Slicing
Slicing the elements from 1 to 4 ( i.e, 1,2,3 ) and printing the value of the layout key as an array
Slicing the elements from 1 to 4 ( i.e, 1,2,3 ) and printing the value of the layout key as stream
Slicing example 1
Slicing example 2
Slicing example 3
Filtering and Manipulating Data
Concatenate arrays
Delete 2 items
Only print records where given field matches a value
Group by a key - opposite to flatten
Minimum value of an array
Reverse an array
Mapping
Display the type of each item
Output the value of a given key of each element in a JSON file
Dictionary subset shorthand
Filter a list of objects
Add + 1 to all items
Sorting
Sort an array of basic type
Sort an array of objects
Sort lines of a file
Unique
Remove duplicates
Remove duplicates by key
Remove duplicates by length
Converting / Parsing / Serializing
Flatten an array
Parsing json
Serializing json
Converting to CSV
Converting to csv
This command takes each object in the JSON array, extracts the specified fields, and formats them as CSV.
Same as the previous command but also includes histogram (nested array) data
Formatting
Print only selected fields
Print selected fields as text instead of json
URL Encoding
URL Encode the data
Exporting Data / Environment Variables
To create proper JSON from a shell script and properly escape variables
To fill environment variables from JSON object key
To fill environment variables from the JSON file
With cat Command
Execute a specific script
Pass specific arguments
Print specific keys
Print specific array items
Print all array items/object keys
Add/remove specific keys
Last updated
Was this helpful?