How to handle \0 in bash

bash-bin-zero.sh

#!/bin/bash
set -x

declare -a str

read_w_null() {
    local a
    while [ 1 = 1 ]; do
        if IFS= read -r -d '' a; then
            str+=("${a}x")
            str+=("")
        elif [[ "$a" ]]; then
            str+=("${a}x")
        else
            break
        fi
    done <"$1"
}

for f in "$@"; do
    str=()
    read_w_null "$f"
    echo -E "$f:"
    for a in "${str[@]}"; do
        if [ -z "$a" ]; then
            printf "\0"
        else
            echo -n -E "${a%x}"
        fi
    done
done

bash-how-to-handle-null.md

If you run bash with input where there is \0 / null byte, you get warning:

bash: warning: command substitution: ignored null byte in input

Way to manage that kind of files:

./bash-bin-zero.sh /etc/issue /proc/$$/environ | less

There is no clean way to manage \0 in bash. This script uses arrays to go around limitations and then uses "x" as prefix to differentiate between empty value and \0.

To "return" value from function we use global str array.

Only way to "output" \0 from bash is to use external command, here printf. Actually bash does not output here, it is the external command.

If you need to do this you are doing it wrong most probably. Bash is not really optimized. It reads one char per time. So your script cost does scale up with length of input.

There is easy ways to handle \0 terminated string, like:

xargs -0l1 </proc/$$/environ
tr '\0' '\n' </proc/$$/environ
jq -Rs . </proc/$$/environ | jq -jr .
base64 </proc/$$/environ | base64 -d

You can simplify methot in bash-bin-zero.sh by using readarray:

IFS= readarray -d '' a </proc/$$/environ;for b in "${a[@]}";do echo -E ">$b<";done
myecho(){ echo -E ">$2<";};IFS= readarray -c 1 -C myecho -d '' </proc/$$/environ

It does not differentiate when \0 is last char or some other char is by default:

IFS= readarray -d '' a < <(printf "a\0b");for b in "${a[@]}";do echo -E ">$b<";done
IFS= readarray -d '' a < <(printf "a\0b\0");for b in "${a[@]}";do echo -E ">$b<";done
myecho(){ echo -E ">$@<";};IFS= readarray -c 1 -C myecho -d '' a < <(printf "a\0b")
myecho(){ echo -E ">$@<";};IFS= readarray -c 1 -C myecho -d '' a < <(printf "a\0b\0")

But if you do it like this things work as it should:

input="a\0\0b\0\0"
(IFS= readarray -d '' a < <(printf "$input";printf "\0");
c=0;for b in "${a[@]}"; do [ $c -eq 0 ] || printf "\0";c=1;echo -nE "$b";done)|less

But there is no way to handle file input without external command:

IFS= readarray -d "" a < <(cat /proc/$$/environ;printf "\0")
(c=0;for b in "${a[@]}"; do [ $c -eq 0 ] || printf "\0";c=1;echo -nE "$b";done)|less