Query JSON Documents in the Terminal with GROQ

css-tricks.com css-tricks.com3 years ago in #Dev Love121

JSON documents are everywhere today, but they are rarely structured the way you want them to be. They often include too much data, have weirdly named fields, or place the data in unnecessary nested objects. Graph-Relational Object Queries (GROQ) is a query language (like SQL, but different) which is designed to work directly on JSON documents. It basically lets you write queries you can quickly filter and then reformat JSON documents to get them into the most convenient shape. GROQ was developed by Sanity.io (where it’s used as the primary query language). It’s open source and it gives us built-in ways to use it in JavaScript and the command line on any JSON source. Together, we’ll add GROQ to the terminal toolkit, which will save you time whenever you need to wrangle some JSON data on a poject. Let’s install GROQ Like most things, we need to install the GROQ CLI tool and can do so with with npm (or Yarn) in the terminal: $ npm install -g groq-cli In order to play with it, we need to have a JSON file available. We’ll use curl to download an example dataset of todo data: $ curl -o todos.json https://jsonplaceholder.typicode.com/todos Let’s take a quick look at a sample item in the data: { “userId”: 1, “id”: 1, “title”: “delectus aut autem”, “completed”: false }, Pretty straightforward. We have a user ID, a todo item ID, a todo title and a boolean specifying whether the todo item is completed or not. Now let’s run a basic GROQ query: finding all completed todos, but only return the todo titles and user IDs. It’s OK to copy/paste this line because we’ll walk through what it means in a bit. $ cat todos.json | groq ‘*[completed == true]{title, userId}’ –pretty The groq command line tools accept a JSON document on standard input. This works very nicely with the Unix philosophy of “doing one thing and work on a text stream.” In order to read JSON from a file we’ll use the cat command. Also note that groq will output a minimal JSON on a single line by default, but by passing –pretty, we get a nicely indented and highlighted syntax. To store the result, we can pipe it to a new file using >: $ cat todos.json | groq ‘*[completed == true]{title, userId}’ > result.json The query itself consists of three parts: * refers to the dataset (i.e. the data in the JSON file). [completed == true] is a filter that removes items that are marked as incomplete. {title, userId} is a projection that causes the query to only return the “title” and “userId” properties. Let’s warm up with some exercises You probably didn’t think you’d need to exercise to get through this post! Well, the good news is that we’re only exercising the mind with a few things to try out with GROQ before we get into more details. What happens if you remove [completed == true] and/or {title, userId}? How can you change the query to find all todos by the user with an ID of 2? How can you change the query to find uncompleted todos by the user with an ID of 2? What happens if the filter in the original query example swaps places with the projection? How would you write a single command (with pipes) that downloads the JSON and processes it with GROQ? We’ll put the answers at the end of the post for you to reference. Querying the Nobel prize winners The todo data is nice for a warmup, but let’s be honest: It’s not very motivating…

Like to keep reading?

This article first appeared on css-tricks.com. If you'd like to keep reading, follow the white rabbit.

View Full Article

Leave a Reply