Problem Overview

In ASM2, we are given assembly source code to reverse engineer. We know that the function takes two arguments: 0x4 and 0x2d

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

asm2:
	<+0>:	push   ebp
	<+1>:	mov    ebp,esp
	<+3>:	sub    esp,0x10
	<+6>:	mov    eax,DWORD PTR [ebp+0xc]
	<+9>:	mov    DWORD PTR [ebp-0x4],eax
	<+12>:	mov    eax,DWORD PTR [ebp+0x8]
	<+15>:	mov    DWORD PTR [ebp-0x8],eax
	<+18>:	jmp    0x50c <asm2+31>
	<+20>:	add    DWORD PTR [ebp-0x4],0x1
	<+24>:	add    DWORD PTR [ebp-0x8],0xd1
	<+31>:	cmp    DWORD PTR [ebp-0x8],0x5fa1
	<+38>:	jle    0x501 <asm2+20>
	<+40>:	mov    eax,DWORD PTR [ebp-0x4]
	<+43>:	leave
	<+44>:	ret

I had to learn how function arguments are managed on the stack, and it wasn't that bad! Before we walk through the assembly code and get the flag, let's explore arguments and the stack.

Arguments and the Stack…

Coming from a background of higher-level languages, I never really had to wonder how arguments in code like below was represented in assembly:

1
2

def check_number(first, second):
   return first + second

This is a good chance to learn! Let's see what fancy stuff is going on in the code.

Creating a New Stack Frame

The below sequence is a commonly-found sequence for many compilers. The push and mov instructions sets up a new stack frame.

The sub instruction creates space in memory for automatic local variables. You can read about this sequence here.

1
2
3
4

asm2:
	<+0>:	push   ebp
	<+1>:	mov    ebp,esp
	<+3>:	sub    esp,0x10 ;; 16 bytes

Now, we have a new stack frame with 16 bytes of memory allocated.

Accessing Function Parameters

I found out that there are different conventions for how arguments are passed onto the stack.

The assembly code looked like x86, so I assumed it was using the cdecl calling convention.

In cdecl, functions arguments are passed onto the stack from right to left.

So in the example below, the assembly representation would first push num3 onto the stack, then num2, and finally num1.

1
2
3

int add(int num1, int num2, int num3) {
   return num3 + num2 + num1;
}

For our CTF challenge, we now know that the last function parameter is the first thing pushed onto the stack.

Getting the 2nd Parameter

For this code segment, we get the value 12 bytes above ebp, the 2nd parameter, and push it into eax.

Afterwards, we move the value from eax 4 bytes below ebp, making it easier to call in our stack frame.

We know that the 2nd paramter is 0x2d, or 45.

1
2

	<+6>:	mov    eax,DWORD PTR [ebp+0xc]
	<+9>:	mov    DWORD PTR [ebp-0x4],eax

Getting the 1st Parameter

Likewise, we repeat the same process to get the first function parameter, 0x4, into the stack frame.

1
2

	<+12>:	mov    eax,DWORD PTR [ebp+0x8] ;;
	<+15>:	mov    DWORD PTR [ebp-0x8],eax ;;

Simple Calculations

Now that we know where our function parameter values are, we are left with a relatively straight-forward algorithm to follow:

1
2
3
4
5
6
7
8

	<+18>:	jmp    0x50c <asm2+31>
	<+20>:	add    DWORD PTR [ebp-0x4],0x1
	<+24>:	add    DWORD PTR [ebp-0x8],0xd1
	<+31>:	cmp    DWORD PTR [ebp-0x8],0x5fa1
	<+38>:	jle    0x501 <asm2+20>
	<+40>:	mov    eax,DWORD PTR [ebp-0x4]
	<+43>:	leave
	<+44>:	ret

The algorithm is something like:

  1. If the first function parameter is less than or equal to 24481

    1. Add 1 to the second function parameter
    2. Add 209 to the first function parameter
    3. Return to the less than or equal comparison

  2. If the first function parameter is > 24481, move the second function parameter into eax

    1. Return the value in eax

More or less, we can translate the assembly code here into this simple Python script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

arg_1 = 4
arg_2 = 45
counter = 0

while arg_1 <= 24481:
    arg_2 += 1
    arg_1 += 209
    counter += 1

print(f"looped {counter} times, arg1: {arg_1}, arg2: {arg_2}")

Running the Python Script

When we run the Python script, the output is: looped 118 times, arg1: 24666, arg2: 163

We know that the function returns the value of arg2, so the flag is simply the hexadecimal representation of 163, or 0xA3