Skip to content

Instantly share code, notes, and snippets.

@mika76
Created June 16, 2018 11:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mika76/d09e2b65159e435e7a4cc5b0299c3e84 to your computer and use it in GitHub Desktop.
Save mika76/d09e2b65159e435e7a4cc5b0299c3e84 to your computer and use it in GitHub Desktop.
-------------------------------------------------------------------------------
Base64 and related binary to 'printable' ASCII encoding.
-------------------------------------------------------------------------------
Base64...
Encodes a binary string into a printable form using a set of 64 characters
such that 3 binary charcaters become 4 printable characters
A-Z a-z 0-9 + /
Base64 on the other hand can output up to 2 extra '=' pading characters on
the final line (unused padding characters).
Problems however can develop if the base64 encoding is being used as
a filename, as the '/' is not permitted, and '+' represents space in URLs
and not a 'plus'.
"Modified Base64 for URL"
This replaces '+' and '/' with the charcaters '-' and '_' respectivally. It
also removes (deletes) any '=' padding on the end.
A-Z a-z 0-9 - _
Many other varients for the two special charcaters also exists. for example
encfs replaces these characters with '-' and ',' (which is which is not
known)
EncFS filename encoding
The base64 character set is used for the binary encrypted filenames,
but with the characters './' replaced with ',-' respectiveally.
The filename has a checksum added, and it is then encrypted.
The result is then base64 encoded.
Source file from EncFS (binary to base64 indexes, then to ascii)
http://code.google.com/p/encfs/source/browse/trunk/encfs/base64.cpp
UUencoding...
Was originally designed in the 1970's for sending binary data in mail and
usenet news articals. It is actually exactly the same as base64 encroding, but
using a different set of 64 characters, that are consecutive ASCII character
codes, where base64 charaectrs are not. (see perl implementation below).
That is it uses the charcaters.
space to _ (with an sometimes optional ` characater as a pad)
Lines are broken into lines with newline characters, and prefixed with a count
of the number of characters on the line, typically 'M' (45) characters.
The final line will be a single '`' on it own (zero characters on line) and
the whole encoding wrapped by 'begin' and 'end' constructs.
Note that while uuencode uses space and other punctunation characters,
base64 does not, which solves a problem that has always plagued uuencoding
when emailing binary files.
XXencoding...
An obsolete attempt to fix the UUencode method so space is not needed.
Characters set was
+ - 0-9 A-Z a-z with + also used as the pad
Password base64
(incomplete)
http://lists.fedoraproject.org/pipermail/389-users/2009-January/008805.html
LDAP and Unix shadow files use different form of base64, basically
just another shuffle of the characters used.
"A-Za-z0-9+/" --> "./0-9A-Za-z"
Base32
Case insensitive encoding. 5 binary characters as 8 printable characters
A-Z 0-7 and a pad of = (needing either 6,4,3,1 such pads)
Base16
Case insensitive encoding for binary data
Essentually it is just simply 'hex' encoding (using capitals)
You can find words which are also hexadecimal number in the dictionary.
For example...
egrep '^[a-f]{4}$' /usr/share/dict/words
deaf decaf feed beef decade faded defaced facade
You generally want a sting of 32 such characters
ModHex
A variation on Base16, designed for USB keyboard emulation.
See "yubikey" information.
-------------------------------------------------------------------------------
Programs available to handle.
Base64 Encoding...
mmencode file # metamail package
uuencode -m - # sharutils package (base64 option)
gmime-uuencode --base64 # Gmime package
perl -ne 'use MIME::Base64; print encode_base64($_);' file
b64encode # personal perl script (no wrapper)
openssl enc -base64 # no wrapper - multiple input/output methods
Base64 Decoding
mmencode -u mime_file
uudecode # wrap with begin-base64
gmime-uudecode --base64
perl -ne 'use MIME::Base64; print decode_base64($_);' mime_file
b64decode # personal perl script (no wrapper)
openssl enc -base64 -d # no wrapper - multiple input/output methods
See also this file from EncFS (ascii to base65 indexes then to binary)
http://code.google.com/p/encfs/source/browse/trunk/encfs/base64.cpp
Most of these packages add a start/end wrapper during encoding
and also needs the same wrapper around the data for decoding.
EG:
begin-base64 600 -
cGFzc3dkCg==
====
For Example remove and re-add the wrapper
> echo passwd | uuencode -m - | sed -n '1d;/^====$/!p' > secret
> cat secret
cGFzc3dkCg==
> ( echo 'begin-base64 600 -'; cat secret; echo '===='; ) | uudecode
passwd
-------------------------------------------------------------------------------
Perl and Base64
NOTE: as base64 is the same a uuencoding except that length character at the
start of the lines has been removed and the characters have been replaced
with alphanumerics instead of puncuation and space characters.
EG: and perl4 decoder...
perl -ne ' tr#A-Za-z0-9+/`##cd;
tr#A-Za-z0-9+/`# -_=#;
$len = pack("c", 32 + 0.75*length);
print unpack("u", $len.$_ );' mime_file
The final length of the base64 encoded sequence uses = signs at the end to
zero out unwanted characters. This makes it difficult for a simple
base64 encoder, but it is close to..
perl -e 'while(read(STDIN,$_,45)) {
$_=substr(pack("u",$_),1);
tr# -_`#A-Za-z0-9+/A#;
print $_; }' file
With any 'A' characters on the end converted to '=' characters instead.
The perl package "MIME::Base64" provides better and easier methods for
conversion between base64 and Binary. For example...
#!/usr/bin/perl
use MIME::Base64;
my $passwd = 'cGFzc3dkCg==';
my $decoded = &decode_base64($passwd);
print "$decoded\n";
Would return
passwd
-------------------------------------------------------------------------------
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment