Writeup: Inconspicuous Program - upCTF (Reverse)

Challenge Description

I found this file on one of our servers and even though its presence is suspicious there doesn't seem to be anything of note about it.

Category: Pwn / Reverse Engineering Flag: upCTF{I_w4s_!a110wed_t0_write_m4lw4r3}

Analysis

We're given a single ELF 64-bit binary called inconspicuous.

Initial Recon

$ file inconspicuous
inconspicuous: ELF 64-bit LSB executable, x86-64, dynamically linked, not stripped

$ strings inconspicuous
Enter password:
encrypted_bin
encrypted_bin_len
mmap
mprotect

Key observations:

• The binary prompts for a password
• It has symbols encrypted_bin and encrypted_bin_len — an embedded encrypted payload
• It uses mmap and mprotect — it will decrypt and execute code in memory

Reversing main

Disassembling main reveals the following logic:

1. Prompt for a password via fgets (max 64 bytes)
2. Strip the newline with strcspn
3. Compute a single-byte XOR key: key = strlen(password) + 0x10
4. XOR every byte of encrypted_bin (308 bytes at 0x403060) with this key
5. mprotect the region as executable
6. Jump into the decrypted shellcode, passing the password as an argument

// Pseudocode
char password[64];
fgets(password, 64, stdin);
password[strcspn(password, "\n")] = 0;

uint8_t key = strlen(password) + 0x10;
void *buf = mmap(NULL, encrypted_bin_len, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0);

for (int i = 0; i < encrypted_bin_len; i++)
    buf[i] = encrypted_bin[i] ^ key;

mprotect(buf, encrypted_bin_len, PROT_READ|PROT_EXEC);
((void(*)(char*))buf)(password);

Breaking the XOR

Since the entire payload is XORed with a single byte, we can brute-force all 256 possible keys and check which one produces valid x86-64 code.

with open('inconspicuous', 'rb') as f:
    data = f.read()

enc_data = data[0x2060:0x2060+308]  # encrypted_bin file offset

for key in range(256):
    dec = bytes([b ^ key for b in enc_data])
    if dec[0] == 0x55 and dec[1] == 0x48:  # push rbp; mov rbp, rsp
        print(f"Key 0x{key:02x} -> valid function prologue")
        print(f"Password length = {key - 0x10}")

Result: Key 0x1c (decimal 28) produces a valid function prologue (push rbp; mov rbp, rsp). This means the password is 12 characters long (28 - 16 = 12).

Decrypted Shellcode

Decrypting with key 0x1c reveals shellcode that:

1. Checks each byte of the password against the string s3lf_m0d_b1n
2. If the password is correct, uses a write syscall to print the flag

The flag is embedded directly in the shellcode:

upCTF{I_w4s_!a110wed_t0_write_m4lw4r3}

Solution Script

with open('inconspicuous', 'rb') as f:
    data = f.read()

enc_data = data[0x2060:0x2060+308]
key = 0x1c
dec = bytes([b ^ key for b in enc_data])

import re
for s in re.findall(b'[\x20-\x7e]{4,}', dec):
    print(s.decode())

Flag

upCTF{I_w4s_!a110wed_t0_write_m4lw4r3}