Photo by Compare Fibre / Unsplash

Writeup: JMP Custom Protocol - EspilonCTF (ESP)

Feri Harjulianto

Challenge Description

The CERT-CORP intercepted a strange firmware from an unknown router model built by the shady Jnouned Company. Analysts found it targets an ESP32-based prototype, but the system is protected by a locked UART console. Your mission: flash the firmware, break into the system, and unlock admin access.

Category: Reverse Engineering Flag: ESPILON{Jn0un3d_UDP_Pr0t0c0l}

Solution

Step 1: Firmware Reconnaissance

The challenge provides ESP32 firmware files (ESP-IDF v5.3.2, Xtensa LX6). String analysis of jnouned_router.bin (873 KB) reveals a custom network protocol used for covert data exfiltration:

Protocol: JMP on UDP:6999
JMP Server starting...
JMP Server listening on UDP:%d
JNOUNER_SECRET_EXFILTRATION
PROTOCOL INITIALIZATION LEAK
Exfiltration session started.
start_session  - Start data exfiltration session

This is the JMP protocol — a custom UDP-based exfiltration channel running on port 6999, complete with SHA256 authentication, session tokens, and block-based data transfer.

Step 2: Map the JMP Protocol from Firmware Strings

By extracting and ordering all JMP-related log strings, the full protocol flow emerges:

Server Initialization:

I: JMP Server starting...
I: JMP Server task started
I: JMP Server listening on UDP:6999

Protocol Leak (intentional info disclosure on init):

W: PROTOCOL INITIALIZATION LEAK
W: Protocol: JMP v1.0
W: Port: UDP/%d
W: Magic: 0x%08lX
W: Authentication Hash (SHA256):
W: Hint: Secret pattern is JNOUNER_SECRET_XXXX
W: Refer to RFC JMP for protocol details

Authentication Phase:

I: AUTH_REQUEST from %s:%d
I: [OK] AUTH SUCCESS - Token: 0x%08lx
W: [FAIL] AUTH FAILED - Invalid hash
W: AUTH_REQUEST too small: %d bytes

Data Transfer Phase:

I: DATA_REQUEST - Block ID: %d
I: [OK] Sending block %d/%d (%d bytes, checksum=0x%04X)
W: [FAIL] Block ID %d out of range
W: [FAIL] Invalid session (token=0x%08lx, expected=0x%08lx)
W: DATA_REQUEST too small: %d bytes
   Invalid session. Authenticate first.
   Block ID out of range

Step 3: Reconstruct the JMP Protocol Specification

From the firmware strings and disassembly, the JMP v1.0 protocol works as follows:

┌────────────────────────────────────────────────────────────┐
│                   JMP Protocol v1.0                        │
│                   UDP Port 6999                            │
├────────────────────────────────────────────────────────────┤
│                                                            │
│  Phase 1: Authentication                                   │
│  ┌──────────────────────────────────────────────────────┐  │
│  │ Client -> Server: AUTH_REQUEST                       │  │
│  │   [4B Magic] [1B cmd_type=0x01] [32B SHA256 hash]   │  │
│  │                                                      │  │
│  │ The SHA256 hash is computed over a secret string     │  │
│  │ matching pattern: JNOUNER_SECRET_XXXX                │  │
│  │ Actual secret: JNOUNER_SECRET_EXFILTRATION           │  │
│  │                                                      │  │
│  │ Server -> Client: AUTH_RESPONSE                      │  │
│  │   [4B session_token]                                 │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                            │
│  Phase 2: Data Exfiltration                                │
│  ┌──────────────────────────────────────────────────────┐  │
│  │ Client -> Server: DATA_REQUEST                       │  │
│  │   [4B Magic] [1B cmd_type=0x02] [4B token] [4B ID]  │  │
│  │                                                      │  │
│  │ Server -> Client: DATA_RESPONSE                      │  │
│  │   [Block data] [2B checksum]                         │  │
│  │                                                      │  │
│  │ Blocks are numbered 0..N, with per-block checksums   │  │
│  └──────────────────────────────────────────────────────┘  │
│                                                            │
└────────────────────────────────────────────────────────────┘

Key protocol details:

  • Magic: 4-byte header identifying JMP packets (leaked on init via Magic: 0x%08lX)
  • Authentication: SHA256 hash of JNOUNER_SECRET_EXFILTRATION (secret hinted as JNOUNER_SECRET_XXXX)
  • Session tokens: 32-bit token issued on auth success, required for all data requests
  • Data transfer: Block-based with sequential IDs and 16-bit checksums

Step 4: Trace the UART Command Interface

The UART console exposes the start_session command that triggers the exfiltration:

start_session  - Start data exfiltration session

On execution:

Exfiltration session started.
Protocol: JMP on UDP:6999

Or on failure:

Session already active.
Failed to start session.

This command initializes the JMP server task (jmp_server) which creates a FreeRTOS task listening on UDP port 6999.

Step 5: Extract the Encrypted Flags

The flags are encrypted with AES-256-CBC. The key is derived from 8 XOR-encrypted 4-byte blocks stored in DROM:

small_blocks = [0x3f412e54, 0x3f412e50, 0x3f412e4c, 0x3f412e48,
                0x3f412e44, 0x3f412e40, 0x3f412e3c, 0x3f412e38]
xor_keys     = [0x38, 0xe7, 0x91, 0xf7, 0x10, 0x6b, 0x74, 0x96]

aes_key = bytearray()
for addr, k in zip(small_blocks, xor_keys):
    foff = addr - 0x3f400020 + 0x20
    block = data[foff:foff+4]
    aes_key.extend(bytes([b ^ k for b in block]))

The encrypted flag data is organized as (16-byte IV, 32-byte ciphertext) pairs. Flag 3 uses:

  • IV: 16 bytes at 0x3f412d98
  • Ciphertext: 32 bytes at 0x3f412d78
from Crypto.Cipher import AES

iv = data[drom_offset(0x3f412d98):drom_offset(0x3f412d98)+16]
ct = data[drom_offset(0x3f412d78):drom_offset(0x3f412d78)+32]

cipher = AES.new(bytes(aes_key), AES.MODE_CBC, iv=iv)
plaintext = cipher.decrypt(ct)

# Strip PKCS7 padding (0x03 x 3)
pad = plaintext[-1]
plaintext = plaintext[:-pad]

print(plaintext.decode())  # ESPILON{Jn0un3d_UDP_Pr0t0c0l}

Flag 3 Context

ESPILON{Jn0un3d_UDP_Pr0t0c0l} — "Jnouned UDP Protocol" — refers to the custom JMP (Jnouned Message Protocol) discovered in the firmware. This is a covert exfiltration channel built into the router that:

  1. Runs a UDP server on port 6999
  2. Authenticates clients using SHA256 of a shared secret (JNOUNER_SECRET_EXFILTRATION)
  3. Issues session tokens for stateful data transfer
  4. Exfiltrates data in numbered blocks with checksums

The protocol intentionally leaks its own specification on initialization (PROTOCOL INITIALIZATION LEAK), including the magic value, port, and a hint toward the authentication secret — suggesting it was designed as a discoverable backdoor.

UART Console
    │
    ├── admin_login jnoun-admin-2022     (authenticate)
    ├── start_session                     (start JMP server)
    │       │
    │       └── JMP Server on UDP:6999
    │               │
    │               ├── AUTH_REQUEST  (SHA256 of JNOUNER_SECRET_EXFILTRATION)
    │               │       └── AUTH_RESPONSE (session token)
    │               │
    │               └── DATA_REQUEST  (token + block ID)
    │                       └── DATA_RESPONSE (block data + checksum)
    │
    └── flag                              (decrypt & display flags)

All Decrypted Flags

FLAG 0: ESPILON{Jn0un3d_4dM1N}
FLAG 1: ESPILON{802_11_tx_jnned}
FLAG 2: ESPILON{Adm1n_4r3_jn0uned}
FLAG 3: ESPILON{Jn0un3d_UDP_Pr0t0c0l}    <-- this flag

Flag

ESPILON{Jn0un3d_UDP_Pr0t0c0l}

Tools Used

  • Python 3 — binary parsing, scripting
  • Capstone — Xtensa LX6 disassembly
  • PyCryptodome — AES-256-CBC decryption
  • strings — protocol string extraction and analysis

Key Takeaways

  • Embedded firmware often contains custom network protocols. Extracting and ordering log format strings is an effective way to reconstruct the protocol state machine without full disassembly.
  • The PROTOCOL INITIALIZATION LEAK pattern is a deliberate CTF hint — real-world backdoors in router firmware are often more subtle, but the principle of looking for debug/log strings to reverse proprietary protocols applies broadly.
  • The authentication used SHA256 of a discoverable secret (JNOUNER_SECRET_EXFILTRATION, hinted as JNOUNER_SECRET_XXXX), demonstrating that security-through-obscurity in embedded protocols is trivially defeated by firmware analysis.
  • ESP-IDF's FreeRTOS task model (xTaskCreate for jmp_server) and lwIP UDP socket API are standard patterns to recognize when reversing ESP32 network code.
CTFReverseWriteup