Bash urlencode and urldecode

gistfile1.sh

urlencode() {
    # urlencode <string>

    old_lc_collate=$LC_COLLATE
    LC_COLLATE=C

    local length="${#1}"
    for (( i = 0; i < length; i++ )); do
        local c="${1:$i:1}"
        case $c in
            [a-zA-Z0-9.~_-]) printf '%s' "$c" ;;
            *) printf '%%%02X' "'$c" ;;
        esac
    done

    LC_COLLATE=$old_lc_collate
}

urldecode() {
    # urldecode <string>

    local url_encoded="${1//+/ }"
    printf '%b' "${url_encoded//%/\\x}"
}

if you have node js installed:

alias urlencode='node --eval "console.log(encodeURIComponent(process.argv[1]))"'

Thanks for the gist. It was helpful. You can also encode URLs online using URLEncoder.io

have a little problem with Chinese Character.

this solution below using curl command to encode url can work with Chinese Character.

function urlencode() {
if [[ $# != 1 ]]; then
echo "Usage: $0 string-to-urlencode"
return 1
fi
local data="$(curl -s -o /dev/null -w %{url_effective} --get --data-urlencode "$1" "")"
if [[ $? == 0 ]]; then
echo "${data##/?}"
fi
return 0
}
https://gist.github.com/westfly/ed7e25ee4353751d94132f92837a7074

Looks good. Shellcheck complains about the printf printing the character directly. Easily solvable to be shellcheck-clean:

<             [a-zA-Z0-9.~_-]) printf "%s" "$c" ;;
---
>             [a-zA-Z0-9.~_-]) printf "$c" ;;

Great script! Thanks!

For fish users

function urlencode
  set str (string join ' ' $argv)

  for c in (string split '' $str)
    if string match -qr '[a-zA-Z0-9.~_-]' $c
      env LC_COLLATE=C printf "$c"
    else
      env LC_COLLATE=C printf '%%%02X' "'$c"
    end
  end
end

function urldecode
    set url_encoded (string replace -a '+' ' ' $argv[1])
    printf '%b' (string replace -a '%' '\\x' $url_encoded)
end

urlencode() {
    # urlencode <string>
    old_lc_collate=$LC_COLLATE
    LC_COLLATE=C
    
    local length="${#1}"
    for (( i = 0; i < length; i++ )); do
        local c="${1:i:1}"
        case $c in
            [a-zA-Z0-9.~_-]) printf "$c" ;;
            ' ') printf "%%20" ;;
            *) printf '%%%02X' "'$c" ;;
        esac
    done
    
    LC_COLLATE=$old_lc_collate
}

paxsalis version works with bash like charm, but not with bourne (

That snippet worked with bourne

urlencode() {
    # urlencode <string>
    old_lc_collate=$LC_COLLATE
    LC_COLLATE=C
    local i=1
    local length="${#1}"
    while [ $i -le $length ]
    do
        local c=$(echo "$(expr substr $1 $i 1)")
        case $c in
            [a-zA-Z0-9.~_-]) printf "$c" ;;
            ' ') printf "%%20" ;;
            *) printf '%%%02X' "'$c" ;;
        esac
        i=`expr $i + 1`
    done

    LC_COLLATE=$old_lc_collate
}

On zsh I was getting unrecognized modifier 'i' until I changed the following line:

-      local c="${1:i:1}"
+      local c="${1:$i:1}"

amazing solution. thanks

On zsh I was getting unrecognized modifier 'i' until I changed the following line:

-      local c="${1:i:1}"
+      local c="${1:$i:1}"

@cdown, I think this should be replaced on your gist.

urldecode "abc%40abc.com" returns "abc%40abc.com".
Not working.

It works just fine.

$ urldecode() {
>     # urldecode <string>
> 
>     local url_encoded="${1//+/ }"
>     printf '%b' "${url_encoded//%/\\x}"
> }
$ urldecode "abc%40abc.com"
abc@abc.com

I've made a small screencast. Would you mind watching this, please?

https://www.dropbox.com/s/dul4wipk59o2ttk/urldecode_not_working_gist_1163649.webm?dl=0

No, It's your own gist. I've just modified a line 'local c="${1:$i:1}"' according to @krin-san. I did only because I was getting "unrecognized modifier 'i'" error on zsh.

Lovely! Thank you all!

@rajeshisnepali for urldecode to work with my zsh:
In zsh ${url_encoded//%/\\x} adds a \x to the end but ${url_encoded//\%/\\x} replaces % with \x.
lri - https://unix.stackexchange.com/questions/159253/decoding-url-encoding-percent-encoding

also, in urlencode, $i could be made local
local i length="${#1}"

@mountaineerbr Thanks for the information but I couldn't make the above script work.

But it worked using an alias from the python3 script 👍 (from the link above).

I'm seeing strings with 2 or more consecutive spaces get shrunk to 1 space. So %20%20 or ' ' changes to only ' '. Any thoughts on how to not truncate these spaces?

@cjplay02 I'm pretty sure the issue is elsewhere.

$ urldecode() {
    # urldecode <string>

    local url_encoded="${1//+/ }"
    printf '%b' "${url_encoded//%/\\x}"
}
$ urldecode 'foo%20%20%20%20%20bar'
foo     bar

Good call @cdown. I had the urldecode call in a command substitution - urldecoded=$(urldecode 's3://...'). Once I removed the function call from the command substitution, the spaces were retained from the encoding. Now I just need to find a better way to declare the result as a variable...

Edit. Double Quoting around the variable's presentation in downstream commands fixed my issue. Ie echo "$varname"

just a brief nod to mawk
which is five times faster in my tests
(indeed, often faster than sed)

I know it's not a de facto standard like bash
i.e. installed by default on so many systems
but it should be and it is on my systems

I also notice that bash seems to be catching up with ksh93

One line implementation, suitable for storing in .bashrc

urle () { [[ "${1}" ]] || return 1; local LANG=C i x; for (( i = 0; i < ${#1}; i++ )); do x="${1:i:1}"; [[ "${x}" == [a-zA-Z0-9.~_-] ]] && echo -n "${x}" || printf '%%%02X' "'${x}"; done; echo; }
urld () { [[ "${1}" ]] || return 1; : "${1//+/ }"; echo -e "${_//%/\\x}"; }

Thanks for it!

Thanks for this.
Could you please also license this code of yours?

Thanks for the script, but i don't know why when calling urlencode i got in the encoded data a : % at the end !

i had to add a check for systems where collate is not set

if [ -n "$old_lc_collate" ] ; then LC_COLLATE=$old_lc_collate ; fi