Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Bash urlencode and urldecode
urlencode() {
# urlencode <string>
old_lc_collate=$LC_COLLATE
LC_COLLATE=C
local length="${#1}"
for (( i = 0; i < length; i++ )); do
local c="${1:$i:1}"
case $c in
[a-zA-Z0-9.~_-]) printf '%s' "$c" ;;
*) printf '%%%02X' "'$c" ;;
esac
done
LC_COLLATE=$old_lc_collate
}
urldecode() {
# urldecode <string>
local url_encoded="${1//+/ }"
printf '%b' "${url_encoded//%/\\x}"
}
@Sleepingwell
Copy link

Sleepingwell commented Sep 16, 2020

Lovely! Thank you all!

Copy link

ghost commented Dec 12, 2020

@rajeshisnepali for urldecode to work with my zsh:
In zsh ${url_encoded//%/\\x} adds a \x to the end but ${url_encoded//\%/\\x} replaces % with \x.
lri - https://unix.stackexchange.com/questions/159253/decoding-url-encoding-percent-encoding

also, in urlencode, $i could be made local
local i length="${#1}"

@rajeshisnepali
Copy link

rajeshisnepali commented Dec 13, 2020

@mountaineerbr Thanks for the information but I couldn't make the above script work.

But it worked using an alias from the python3 script 👍 (from the link above).

@cjplay02
Copy link

cjplay02 commented Jan 29, 2021

I'm seeing strings with 2 or more consecutive spaces get shrunk to 1 space. So %20%20 or ' ' changes to only ' '. Any thoughts on how to not truncate these spaces?

@cdown
Copy link
Author

cdown commented Jan 29, 2021

@cjplay02 I'm pretty sure the issue is elsewhere.

$ urldecode() {
    # urldecode <string>

    local url_encoded="${1//+/ }"
    printf '%b' "${url_encoded//%/\\x}"
}
$ urldecode 'foo%20%20%20%20%20bar'
foo     bar

@cjplay02
Copy link

cjplay02 commented Jan 29, 2021

Good call @cdown. I had the urldecode call in a command substitution - urldecoded=$(urldecode 's3://...'). Once I removed the function call from the command substitution, the spaces were retained from the encoding. Now I just need to find a better way to declare the result as a variable...

Edit. Double Quoting around the variable's presentation in downstream commands fixed my issue. Ie echo "$varname"

@dicktyr
Copy link

dicktyr commented Jan 31, 2021

just a brief nod to mawk
which is five times faster in my tests
(indeed, often faster than sed)

I know it's not a de facto standard like bash
i.e. installed by default on so many systems
but it should be and it is on my systems

I also notice that bash seems to be catching up with ksh93

@GeekDuanLian
Copy link

GeekDuanLian commented Feb 9, 2021

One line implementation, suitable for storing in .bashrc

urle () { [[ "${1}" ]] || return 1; local LANG=C i x; for (( i = 0; i < ${#1}; i++ )); do x="${1:i:1}"; [[ "${x}" == [a-zA-Z0-9.~_-] ]] && echo -n "${x}" || printf '%%%02X' "'${x}"; done; echo; }
urld () { [[ "${1}" ]] || return 1; : "${1//+/ }"; echo -e "${_//%/\\x}"; }

@rojenzaman
Copy link

rojenzaman commented Jun 13, 2021

Thanks for it!

@SilviaIenciu
Copy link

SilviaIenciu commented Jan 7, 2022

Thanks for this.
Could you please also license this code of yours?

@ThePredators
Copy link

ThePredators commented Feb 6, 2022

Thanks for the script, but i don't know why when calling urlencode i got in the encoded data a : % at the end !

@ironbishop
Copy link

ironbishop commented Apr 4, 2022

i had to add a check for systems where collate is not set

if [ -n "$old_lc_collate" ] ; then LC_COLLATE=$old_lc_collate ; fi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment