CTF Writeup - Flare-On 2016 - 01: challenge1

• Name - challenge1
• Category - Reverse Engineering
• Points - 1
• Description - n/a
• Binary - Download here

Running the PE binary and giving it "1234567890" as input:

Command Prompt

C:\>challenge1.exe Enter password: 1234567890 Wrong password C:\>

Loading it in IDA, we realise that there's a lot going on before we get to the part where the "Enter password:" string is displayed. All this can be disregarded as it's not affected by our input; everything happens in sub_11C1420():

Only 2 functions (sub_11C1260() and sub_11C2C30) are called before the final branch is taken. Let's start with the one that's encountered first. This function takes 2 parameters, the number of bytes in our input, and the input itself. For each iteration of the inner main loop, the function takes 3 characters from our input, loads them into EAX (annotated in the image), does some operations on EAX and translates the characters from buffer byte_EF3000:

As seen in the image, the 3 bytes in EAX (3 chars from input) are shifted to the right by 18, 12, 6 and 0 bits before being translated. This means that each 3 characters from out input are mapped into 4 characters. At this point all alarms should start going off: the algorithm is Base64. If we let the algorithm finish, we get: JQFwKAR2Kwd5JZ==. Although we get what looks like Base64, it does not match the actual Base64 of our input '1234567890', which should be: MTIzNDU2Nzg5MA==. If we look at the translation buffer byte_EF3000, we notice something odd:

The index table has been mangled. So now we know what that the algorithm is a Base64 with a custom index table.

Moving on to the next function, sub_11C2C30, we get a char by char comparison of our (custom-)base64 output from the previous function and the string x2dtJEOmyjacxDemx2eczT5cVS9fVUGvWTuZWjuexjRqy24rV29q. As anticipated, if all the comparisons match, the program branches to the success message.

We create a small python program which reverts the resultant string back to the expected input using the custom base64 algorithm:

    import string
    import base64

    my_base64chars  = "ZYXABCDEFGHIJKLMNOPQRSTUVWzyxabcdefghijklmnopqrstuvw0123456789+/"
    std_base64chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"

    result_string = "x2dtJEOmyjacxDemx2eczT5cVS9fVUGvWTuZWjuexjRqy24rV29q"

    result_string = result_string.translate(string.maketrans(my_base64chars, std_base64chars))
    answer = base64.b64decode(result_string)

    print answer

Using the result as the password:

Command Prompt

C:\>challenge1.exe Enter password: sh00ting_phish_in_a_barrel@flare-on.com Correct! C:\>