Bash: AWS SQS CLI + jq

With the almost ubiquitous adoption of JSON as the message format for Restful API calls now when working with bash scripts JSON parsing command line tool jq is invaluable

Make sure to download the jq library for your distribution

sudo apt-get install jq -y

Start out with some helper bash functions

#!/bin/bash

timestamp() {
	date +%Y-%m-%dT%H:%M:%S%z
}

printLine() {
	printf "$(timestamp): $1\n"
}

In this case want to work with AWS SQS command line and parse out the Message Body of a SQS message

Grab the name of the SQS queue you want to read with jq.  In this case a named queue (you would pass the name of the queue made say it was called transactions and queues are tied to regions so you should use the appropriate region say Oregon us-west-2

aws sqs list-queues --queue-name-prefix transactions --region us-west-2

This command gives you a results like this

{
    "QueueUrls": [
        "https://us-west-2.queue.amazonaws.com/AWS_ID/transactions",
        "https://us-west-2.queue.amazonaws.com/AWS_ID/transactions_dead"
    ]
}

In this case there are 2 named queues as the second one has been configured as a dead letter queue to deal with failure or inabilities to process a queue message in the first queue.

Now to get the first queue use jq with the -r option and reference the property name of QueueUrls

aws sqs list-queues --queue-name-prefix transactions --region us-west-2 | jq -r '.QueueUrls[0]'

Resulting in

https://us-west-2.queue.amazonaws.com/AWS_ID/transactions

Now you can assign the parsed value to a bash variable $url in this case

aws sqs receive-message --queue-url $url --region us-west-2

Running this command will give you the JSON result of any messages in the queue that are available (the automatically hide after being read)

{
    "Messages": [
        {
            "MessageId": "GUID",
            "ReceiptHandle": "Really long security id",
            "MD5OfBody": "MDF5 hash of this message",
            "Body": "{\"transaction\":1201}"
        }
    ]
}

In this case take the aws sqs command from above and set it to the bash variable $cmd and then assign the results to the bash variable message

message=`$cmd`

Now to extract the contents of the messages (first one only in this case)

This is to get the SQS MessageId, using <<< the set to string variable so jq can query the contents

id=`jq -r '.Messages[0].MessageId' <<< "$message"`

Here we are reading the message receipt

receipt=`jq -r '.Messages[0].ReceiptHandle' <<< "$message"`

and now extract the body of the SQS message into a variable

data=`jq -r '.Messages[0].Body' <<< "$message"`

Further finally extract the transaction body as a variable from the JSON encoded data within the SQS message (Note you could have posted this not as JSON, but it makes the system seem more consistent if you place the value so that everything will be JSON when read and extracted)

transaction=`jq -r '.transaction' <<< "$data"`