Bash cURL, arrays, date formatting with CSV

In celebration of daylight savings time and all the misery it causes with timezones.

The Solcast API delivers forecast results in CSV format along with others (JSON/XML), but I wanted to load the data into a bash array and convert it to my localized time by parsing out the ending times (period_end) and taking on of the radiation values (GHI).

There are numerous libraries and code techniques for converting UTC ISO8601 timestamps to localized timestamps, but this one is intended to avoid dependencies as it uses bash, curl, and date utilities.  I would never recommend to do this other than a proof of concept.

#!/usr/bin/env bash

if [[ -z "${SOLCAST_API_KEY}" ]]; then
	printf 'SOLCAST_API_KEY is not set example: `export SOLCAST_API_KEY=S*******************************` \n'
	exit 1
fi

lat=$1
lng=$2
printf 'Latitude: %d, Longitude: %d' $lat $lng

forecast=`echo "$(date "+%Y%m%d%H%M") - ($(date +%M)%30)" | bc | xargs -I{} echo "{}_forecast.csv"`

echo $api_key
echo $forecast

api_url="https://api.solcast.com.au/radiation/forecasts?longitude=$lng&latitude=$lat&api_key=$SOLCAST_API_KEY"
echo $api_url

readarray -t current_forecast < <(curl -X GET $api_url -H 'Content-Type: text/csv' -H 'cache-control: no-cache')

readarray -td, headers <<<"$current_forecast[0],";
unset 'headers[-1]';
## declare -p headers;
declare headers;

for key in "${!headers[@]}"
do
	header_name=${headers[$key]}
	if [[ "${header_name,,}" == "period_end" ]]; then
		period_end=$key
	fi
	if [[ "${header_name,,}" == "ghi" ]]; then
		ghi=$key
	fi	
done

printf 'period_end,Ghi\n' >> $forecast
for line in "${current_forecast[@]:1}"  
do  
	readarray -td, data_line <<<"$line,";
	unset 'data_line[-1]';
	declare data_line;
	printf '%s,%d\n' $(date -d ${data_line[${period_end}]} +"%Y-%m-%dT%H:%M:%S") ${data_line[${Ghi}]} >> $forecast
done

Usage: Above script saved as localize.sh

  • First parameter Latitude: 32
  • Second parameter Longitude: -97
./localize.sh 32 -97

Expected run output

./localize.sh 32 -97
Latitude: 32, Longitude: -97
201903101700_forecast.csv
https://api.solcast.com.au/radiation/forecasts?longitude=-97&latitude=32&api_key=S*******************************
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20770    0 20770    0     0  21590      0 --:--:-- --:--:-- --:--:-- 21590

robin@wasp:/mnt/c/Users/siliconrob/Desktop$ cat 201903101700_forecast.csv
period_end,Ghi
2019-03-10T17:30:00,276
2019-03-10T18:00:00,174
2019-03-10T18:30:00,121

Then you can open the file in your favorite spreadsheet editor the below is LibreOffice and plot the data.