parse error when using httpie and jq in cron entry

parse-error-when-using-httpie-and-jq-in-cron-entry.md

Intro

The following content is part of Bings Copilot and me. We had a long chat (for testing purposes) and this is the solution. Also a part of jq syntax was optimized (shortend, more specific) with the help of Copilot. Especially the last part with the select was a good example of how an AI can make things "elegant".

The starting point.

I wanted to have a solution to send me one value from a remote system to a Teams Channel aka webhook. Using just httpie and jq with cron Pretty easy. Right? Yeah. Not so much....

The problem(s)

Normally i use for different scenarios the tool httpie. Which is actually pretty handy for some cases. Yet in this case, it was a "small" nightmare. But a very good learning experience with Copilot and also jq.

Some of the next parts are partially from Copilot

Using httpie und jq in cron

When a script that uses httpie and jq is run in a cron job, an parse error: Invalid numeric literal at line 1, column 4 occured. However, the same script works as expected when run in a normal shell. In the end, the output was always the (already given) Text, not the value extracted from the JSON output Because of the parse error. On a regular shell, it all worked very fine. No matter if it was bash oder zsh.

Cause

The problem seems to be that the environment in which the cron job is executed is different from the normal shell. In particular, it could be that httpie requires an environment variable that is not present in the cron environment.

Solution

One way to fix this problem is to use curl instead of httpie. Unlike httpie, curl seems to work without problems in the cron environment.

All other trys with login shells (bash -l -c) in crontab, setting SHELL in crontab or both did really not work. Also parsing with source .bashrc, .bash_profile oder /etc/profile did not do the trick ether.

Here is an example of a script that uses curl instead of httpie:

#!/bin/bash
curl -sS ://localapi/infos.json | /usr/bin/jq -r '.[].measure | {content: ."content"} | select(.content != null) | .content'

In this script, the -sS option is used to run curl in silent mode but display errors when they occur. This, in the end, worked now as expected.

Testing Env

minimal-shell.sh
#!/bin/sh

env

VAL1=`http ://localapi/infos.json | /usr/bin/jq -r '.[].measure | {content: ."content"} | select(.content != null) | .content'`

VAL2=`curl -sS http://localapi/infos.json | /usr/bin/jq -r '.[].measure | {content: ."content"} | select(.content != null) | .content'`

echo "Your VAL1: ${VAL1}"
echo "##################"
echo "Your VAL2: ${VAL2}"

crontab entry

min hh * * * /home/user1/bin/minimal-shell.sh > /home/user1/ausgabe.txt 2>&1

The same script on the shell

Will produce a real good output. For both of the $VAL

Path of the used tools

which curl httpie jq                                                                                                             1 ↵
/usr/bin/curl
/usr/bin/httpie
/usr/bin/jq

OS

• Ubuntu 20.04 LTS