Skip to content

Instantly share code, notes, and snippets.

@danielpclark
Created May 11, 2018 15:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save danielpclark/84e1efff8b470ef129c729c6c1fe62e7 to your computer and use it in GitHub Desktop.
Save danielpclark/84e1efff8b470ef129c729c6c1fe62e7 to your computer and use it in GitHub Desktop.
Started writing AES file parser. Doing a rewrite with `nom` so this is for reference.
extern crate byteorder;
use byteorder::{BigEndian,ByteOrder};
use std::io::prelude::*;
use std::fs::File;
use std::str;
// 3 Octets - 'AES'
// 1 Octet - 0x02 (Version)
// 1 Octet - Reserved (set to 0x00)
// .... Start of repeating extension block section
// 2 Octet - Length in octets (in network byte order) of an extension
// identifier and contents. If 0x0000, then no further
// extensions exist and the next octet is the start of the
// Initialization Vector (IV). Following an extension,
// this length indicator would appear again to indicate
// presence or absense of another extension and the size of
// any such extension.
// nn Octets - Extension identifier. This is either a URI or an
// identifier defined by the AES developer community and
// documented on the standard extensions page, either
// of which is terminated by a single 0x00 octet. All
// extension identifiers are case sensitive.
// Examples of URIs:
// http://www.aescrypt.com/extensions/creator/
// urn:oid:1.3.6.1.4.1.17090.55.14
// urn:uuid:85519EA3-1DA6-45b9-9041-8CD368D8C086
// Note:
// A URI was used to allow anybody to define extension
// types, though we should strive to define a standard
// set of extensions.
// Examples of standard extension identifiers:
// CREATED-DATE
// CREATED-BY
// A special extension is defined that has no name, but is
// merely a "container" for extensions to be added after the
// AES file is initially created. Such an extension avoids
// the need to read and re-write the entire file in order to
// add a small extension. Software tools that create AES
// files should insert a 128-octet "container" extension,
// placing a 0x00 in the first octet of the extension
// identifier field. Developers may then insert extensions
// into this "container" area and reduce the size of this
// "container" as necessary. If larger extensions are added
// or the "container" area is filled entirely, then reading
// and re-writing the entire file would be necessary to add
// additional extensions.
// nn Octets - The contents of the extension
// .... End of repeating extension block section
// 16 Octets - Initialization Vector (IV) used for encrypting the
// IV and symmetric key that is actually used to encrypt
// the bulk of the plaintext file.
// 48 Octets - Encrypted IV and 256-bit AES key used to encrypt the
// bulk of the file
// 16 octets - initialization vector
// 32 octets - encryption key
// 32 Octets - HMAC
// nn Octets - Encrypted message (2^64 octets max)
// 1 Octet - File size modulo 16 in least significant bit positions
// 32 Octets - HMAC
//
// Thus, the footprint of the file is at least 134 octets.
#[allow(dead_code)]
fn read_aes_from_bytes(file: &mut File) -> String {
let mut aes = [0; 3];
file.read_exact(&mut aes).ok().unwrap();
str::from_utf8(&aes).unwrap().to_string()
}
#[allow(dead_code)]
fn read_version_from_bytes(file: &mut File) -> u16 {
let mut version = [0; 1];
file.read_exact(&mut version).ok().unwrap();
let b16 = [0, version[0]];
BigEndian::read_u16(&b16)
}
#[allow(dead_code)]
fn skip_byte(file: &mut File) {
let mut b = [0; 1];
file.read_exact(&mut b).ok().unwrap();
}
#[allow(dead_code)]
fn read_extension_length_from_bytes(file: &mut File) -> u16 {
let mut ext_len = [0; 2];
file.read_exact(&mut ext_len).ok().unwrap();
BigEndian::read_u16(&ext_len)
}
#[allow(dead_code)]
fn read_extension_contents(file: &mut File, length: usize) -> String {
let mut buf = vec![0; length].into_boxed_slice();
file.read_exact(&mut buf).ok().unwrap();
str::from_utf8(&buf).unwrap().to_string()
}
#[allow(dead_code)]
fn is_uninitialized(s: &str) -> bool {
s.chars().all(|v| v == '\u{0}')
}
#[allow(dead_code)]
fn read_iv1(file: &mut File) -> [u8; 16] {
let mut iv = [0; 16];
file.read_exact(&mut iv).ok().unwrap();
iv
}
#[allow(dead_code)]
fn read_iv_and_key(file: &mut File) -> ([u8; 16], [u8; 32]) {
let mut iv = [0; 16];
let mut key = [0; 32];
file.read_exact(&mut iv).ok().unwrap();
file.read_exact(&mut key).ok().unwrap();
(iv, key)
}
#[allow(dead_code)]
fn hmac_sha256(file: &mut File) -> [u8; 32] {
let mut hmac = [0; 32];
file.read_exact(&mut hmac).ok().unwrap();
hmac
}
#[test]
fn it_works() {
let mut f = File::open("Cargo.toml.aes").ok().unwrap();
assert_eq!(read_aes_from_bytes(&mut f) ,"AES");
assert_eq!(read_version_from_bytes(&mut f) , 2);
skip_byte(&mut f);
assert_eq!(read_extension_length_from_bytes(&mut f), 24);
assert_eq!(read_extension_contents(&mut f, 24), "CREATED_BY\u{0}aescrypt 3.11");
assert_eq!(read_extension_length_from_bytes(&mut f), 128);
assert_eq!(is_uninitialized(&read_extension_contents(&mut f, 128)[..]), true);
assert_eq!(read_extension_length_from_bytes(&mut f), 0);
assert_eq!(read_iv1(&mut f),
[
62,
157,
216,
95,
118,
204,
208,
218,
238,
226,
244,
34,
91,
58,
171,
33
]
);
// do a password iv strech here
// 8192 iterations and such
assert_eq!(read_iv_and_key(&mut f),
(
[
194,
222,
196,
162,
115,
204,
217,
6,
67,
14,
237,
158,
66,
158,
17,
4
],
[
176,
252,
33,
248,
95,
109,
241,
143,
209,
125,
237,
98,
219,
203,
172,
199,
184,
246,
151,
7,
146,
71,
23,
205,
21,
44,
102,
124,
17,
100,
120,
225
]
)
);
// read HMAC-SHA256 of the encrypted iv and key
assert_eq!(hmac_sha256(&mut f),
[
225,
222,
77,
137,
137,
119,
65,
238,
87,
93,
58,
86,
16,
175,
148,
136,
208,
248,
214,
228,
237,
157,
207,
233,
26,
247,
232,
61,
22,
0,
100,
62
]
);
// compute actual HMAC-SHA256 of the encrypted iv and key
// HMAC check
// instantiate AES cipher
// decrypt main iv and key
// get internal iv and key
// instantiate another AES cipher
// instantiate actual HMAC-SHA256 of the ciphertext
// write file
// decrypt remaining ciphertext, until last block is reached
// last block reached, remove padding if needed
// read last block
// this is for empty files
// update HMAC
// read plaintext file size mod 16 lsb positions
// decrypt last block
// remove padding
// write decrypted data to output file
// read HMAC-SHA256 of the encrypted file
// HMAC check
}
@danielpclark
Copy link
Author

End comments are from looking at Python source code https://github.com/marcobellaccini/pyAesCrypt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment